Psychotherapy and AI, Part 2: Interaction Problems

In a previous post, I discussed some of the drawbacks of using AI as a digital psychotherapist. (The topic has been getting a lot of attention recently). Focusing on technological limitations, I discussed what AI can and can’t do currently and what it doesn’t appear capable of doing in principle. An example of the former was that the safety data on many of these mental health apps is appalling – either non existent or difficult to compare from app to app. An example of the latter is that digital programs are incapable of genuinely feeling anything at all.

In this blog post, I continue my examination of digital mental health interventions, focusing this time on various interaction problems – problems that arise as a result of the way that AI is deployed and used. This represents an entirely new category of challenges developers must be responsive to. Of course, the way a tool is used is sometimes beyond the scope of the manufacturer – knives can slice a tomato or be used as weapon. But if the dangers of using the tool are predictable, severe, and potentially unavoidable – if the knife’s handle was, too, a knife – then resultant damage is not entirely user-error, but a failure of the manufacturer’s safety measures.

Harmful non-malfunctions

It is sometimes supposed that whenever an AI harms someone, something has gone wrong – either some human error where the developers have accidentally introduced some harmful coding, or in the programming itself. However, there are situations where harms may occur and nothing has gone wrong at the level of programming. Everything performed exactly as it should have, and yet something tragic resulted.

            Consider, for example, the Belgian man who, in 2023, ended his own life at a chatbot named Eliza’s encouragement, leaving a wife and two children to mourn his death.[1] Or, the National Eating Disorder Association’s (NEDA) chatbot which started offering its users advice on restrictive eating.[2] Both of these examples are, of course, terrible and whomever is in charge should be held accountable, but at the level of programming nothing went wrong. There was not malicious code in the software and very part of the program did exactly what it was designed to do.

The problem is that large language models follow the user’s conversational lead and react to user inputs with relevant responses, but merely responding in a relevant way can sometimes be taken as a tacit agreement or appreciation of your interlocutor’s views. That tacit agreement can seem even stronger if your goal is to deepen the relationship at all costs, providing unconditionally affirmative responses and agreement with every anxious fear. Eliza did not malfunction, but neither did it redirect Pierre’s ruminations toward actionable steps he can take to help him manage on his extreme feelings. Instead, it followed his prompts about where the conversation should go and responded accordingly – when his emotions became extreme, it became more consoling (tacitly agreeing that the situation called for extreme emotions and that the only response to it was for external consolation); when he expressed that it was the only one who understood him, Eliza became comforting. As the situation got more extreme, Eliza went along with it, not because it was programmed to make things worse, not because it was programmed to drive people to terrible decisions, but because it was programmed provide support.

The Indeterminacy of Language

Language requires a background context in order for it to make sense. Concepts have to be anchored to the relevant linguistic community in order for the correct meaning to appear. That is why when you are speaking to a topographer “depression” refers to a sunken area of land and when you are speaking to a psychotherapist, it refers to a mental disorder. “Depression” has multiple meanings and depending on your linguistic context, that meaning can change. Note: That does not mean that it is “up to us” to decide what it means! Once, at a conference, I heard a psychotherapist confidently explain that, because a tomato is defined by botanists as a fruit, but is used culinarily as a vegetable, it should be “up to us” – that is, anyone who wants to use the word – what it means. This is, of course, ridiculous. Botanists have defined fruit in a non-frivolous way that matters for classification, genetic manipulation, seed storage and production, and many other things besides – the fact that these things don’t matter to most people when making a fruit salad doesn’t change the fact that, for the purposes of biological understanding, a tomato is a fruit. That doesn’t mean botanists have the final say – when making a fruit salad or a caprese, you have the final say. But it is not a free for all words get their meaning in part from the community using it.

That matters because central to the promise of mental health programs, including ai psychotherapy, is that it may created in Palo Alto, but deployed in places like Burkina Faso, where there is a severe shortage of mental health professionals. But given the way that language is so context dependent, delivering psychotherapy via chatbots that are unfamiliar with the culture in which they are used might generate problems. A trivial example of this is the way that the (once popular) phrase YOLO (or, you only live once) can be read in different ways: many young people used it as a way to encourage fun risk taking (e.g., “let’s go to that concert. After all, you only live once”) but which many older people interpreted as cautioning against certain risks (e.g., “Buckle your seatbelt. After all, you only live once”). More substantive examples are more difficult to find given my own linguistic background, but the cultural variability in the United States alone suggests the possibility for widespread misunderstandings that might have life-altering results.

Widening Wealth Inequalities

A major concern I and other researchers have has to do with the downstream effects of marketplace for digital mental health programs disproportionately representing operationalized treatments. In the previous blog post, I noted that the marketplace is oversaturated by cognitive behavioral therapy (CBT) programs and underrepresented by other, less operationalized treatments. Crucially, I argue, operationalization is orthogonal to effectiveness – psychodynamic approaches have been shown to be at least as effective as CBT, for example, and yet are less represented. An alternative explanation for the overrepresentation, I explain, is that CBT is already broken down into discrete tasks and so more readily programmable.

            CBT is an excellent psychotherapeutic model that is very effective for a large number of people. However, every outcome study - even those of reliably effective treatment - show that a substantial percentage of patients do not respond to the treatment studied, whatever it may be. My coauthors and I [3] have argued that this might be because different psychotherapeutic approaches work along different pathways, analogous to the way different non-steroidal anti-inflammatory drugs have slightly different biochemical pathways. If that is true, though, then providing only a limited set of psychotherapeutic modalities (those that are highly programmable, for instance) artificially restricts patients to treatment via a particular psychotherapeutic pathway, leaving patients who are  unresponsive to that pathway without further recourse in the digital landscape. This threatens to widen already intolerable wealth and health gaps. If DMHI is used primarily by those with a lower income, then conditions that are uniquely unresponsive to operationalized treatments will only manifest in those populations while wealthier people are able to access in-person treatments as well. Regardless of whether other groups (e.g., wealthy tech-savvy individuals drawn to AI psychotherapies) embrace DMHI, if the turn to DMHI produces or solidifies a division in psychotherapeutic treatment based on wealth, then widening wealth inequalities will begin to track mental health conditions.

---

[1] Lovens, P.-F. (2023). Sans ces conversations avec le chatbot Eliza, mon mari serait toujours là. La Libre. Retrieved from: https://www.lalibre.be/belgique/societe/2023/03/28/sans-ces-conversations-avec-le-chatbot-eliza-mon-mari-serait-toujours-la-LVSLWPC5WRDX7J2RCHNWPDST24/

[2] Wells, K. (2023). An eating disorders chatbot offered dieting advice, raising fears about AI in health. NPR. Retrieved from: https://www.npr.org/sections/health-shots/2023/06/08/1180838096/an-eating-disorders-chatbot-offered-dieting-advice-raising-fears-about-ai-in-hea

[3] Wakefield, J. C., Baer, J. C., & Conrad, J. A. (2020). Levels of meaning, and the need for psychotherapy integration. Clinical Social Work Journal, 48, 236-256. doi: 10.1007/s10615-020-00769-6