Something's Missing: What Psychotherapy Research Leaves Out

It is common to hear, in discussions about the value of psychotherapy research, that nomothetic outcomes leave out some indiscernible inter-personal human "magic". That there is something missing from psychotherapy research, which renders its findings essentially moot. So often do I hear this point in discussions, that I want to take it on and suggest that it presents less of a problem than is generally supposed.

When the "something missing" argument is wielded in a debate about research, two consequences usually seem to be implied:

1. That psychotherapy research cannot tell you very much about what psychotherapy is really like, and so should not be trusted in appraising whether it is helpful.

2. That psychotherapy research does a sort of crass violence to the psychotherapy relationship itself, and that psychotherapy researchers are naive to think they can capture something so delicate.

The "something's missing" argument is often stated as though it were a knock-out blow to the value of outcomes research. It isn't. That something should be left out whenever we attempt to measure or represent something else is a banal truism. It simply presents no problem to the project of learning about reality. I am sure I have quoted Paul Meehl on this question before. In his book on statistical prediction, he refutes those who claim that any aspect of human behaviour is too complex to be in principle predictable from regression models, because humans are "more than" the models in question:

"A cannon ball falling through the air is “more than” the equation S=½g, but this has not prevented the development of a rather satisfactory science of mechanics".

The same goes for all of science (the full reality of a Large Hadron collider is more than the sum of the research produced by the physicists who work with it, but the research they produce does not lack veracity or utility in virtue of that fact) and the humanities too (no quantity of historical books on the American Civil War will ever completely reconstruct the experience of someone who fought in it). In fact, it is inherent to representing a state of affairs in any form other than the original.

So yes, psychotherapy research has "something missing", but that is trivial and we have to either accept the limitation or offer solutions to it (which is to say, become methodologists rather than critics). The choice we have is not between trite, uninformative quantitative research and rich-full-blooded qualitative information, it is between some combination of those two approaches and sheer guesswork. 

Quantitative research does not just forget the magic of the interpersonal encounter, it factors it out in a bid to discover a separate numerical truth: how many people show some sort of measurable improvement (and how much of one)? This can look clumsy, but it is actually necessarily revealing to escape the persuasion of interpersonal charm and the therapeutic relationship. Think of a doctor like John Bodkins Adams, who appears to have been very successful interpersonally. He was sufficiently charming that he received money from many of his patients in their wills and became a extremely successful GP. Only something as crass as a body-count (Bodkin-Adams may have killed as many as 160 of his patients) revealed that something untoward was going on. 

We should think of psychotherapy outcome research as analogous to the body count. Without it, we are too apt to be misled by the charisma and good intentions of the therapy industry. 

Unpacking the "illnessy intuition"

Last summer I wrote a post examining why, in the face of fairly wide dissatisfaction, the concept of schizophrenia seems to show such tenacity. As part of that argument I invoked the idea of an intuition that people have about schizophrenia; namely that it seems there is often something "illnessy" there:

"There is a way in which the diagnosis is very convincing; on the face of it many people who meet criteria for schizophrenia seem to be seriously unwell and many of them will testify to that fact.

...even with skepticism about the DSM construct [of schizophrenia], many people's intuitions are that there is something illnessy about the experiences which commonly attract the diagnosis."

At the time, someone rightly pointed out that this is a weak point in the inferential chain.

Interesting argument. Not sure about the scientific validity of the "something illnessy" intuition, though. :-) http://t.co/FPSP8JdGd5

— Jane E.M. Callaghan (@JaneEMCallaghan) July 25, 2015

To some extent this does not matter, as my post was not an attempt to justify the continued "success" of the schizophrenia concept, but rather to explain it. Thus, I do not need to show that the "illnessy intuition" is scientifically valid, only that it has a hold on people's imaginations. However, if you detected a note of endorsement in my post, that is because it was there. I do not really believe in the notion of schizophrenia as illness, but unlike some critics, I do not think it is warranted to conclude that no-one who meets the diagnostic criteria is ill. In some cases the illnessy intuition is valid. Here's how.

Defining illness is slippery, and I am not going to get into philosophical debates here about how to arrive at a definition. For my purposes I don't need to. Some phenomena are so widely agreed to be illnesses that to describe them in any other way seems redundant. I could argue, for example, that leukemia is not an illness, but generally we accept it is because that framework has remained the most useful game in town.

The same appears to be true of some of the phenomena which can give rise to a diagnosis of schizophrenia. It has recently been suggested that some proportion of individuals with "schizophrenia" may actually have a form of NMDA-receptor encephalitis, a brain disorder which is treatable. If these people are not considered ill then two harms arise; they are denied effective medical treatment, and they are denied an adequate narrative account of their distress.

It is this fact that gives rise to the "illnessy intuition" in the case of schizophrenia, and for two reasons. The first is that some unknown (and possibly quite high) percentage of cases clearly arises from biological states of affairs which it will prove useful to describe as illnesses. Think not only of NMDA receptor encephalitis, but also of the conditions, and pharmacological treatments, which are yet to be discovered. The second is a sort of reasoning by analogy. If the set of symptoms associated with schizophrenia can be produced by an illness then it stands to reason that even cases which are not produced by the same phenomena might reasonably be construed as illness, pending a fuller explanation.

None of this speaks against the project of providing alternative non-medical narratives for understanding the suffering associated with psychosis. As I said in the original post, we don't yet know how (or if) all cases of schizophrenia will wind up being explained. 

Psychoanalysis and Schizophrenia?

I've just finished reading Christopher Bollas' newest book When the Sun Bursts, about his work providing psychoanalysis to people going through psychosis. I realised I feel very mixed about it, and I suspect this tells me about my ambivalence about psychoanalysis more generally.

Psychoanalysis and schizophrenia fell out with one another in the 1980s, when the idea that the latter was a brain based disorder began to usurp the notion (mainstream in the 1950s, 60s and 70s US) that it was a psychological reaction originating in family dynamics. Many things contributed to the therapeutic divorce. The overall background was the rise of biological psychiatry and the renewed interest in using the DSM as a systematic tool for empirical research, but several historical events also conspired to paint psychoanalysis in a particularly unfavourable light. One major blow was the Chestnut Lodge Follow Up Study, conducted by Thomas McGlashan, a psychiatrist with sympathies to psychoanalysis who became curious about how effective it was. He examined the long term outcomes of patients at Chestnut Lodge (pretty much the world center of psychoanalytic treatment for schizophrenia) and concluded they were not being helped by their treatment. Another was the Osheroff Case, in which a physician with severe depression was treated psychodynamically and tried to sue Chestnut Lodge for not deploying the most effective treatment.

The Osheroff case did not actually concern an individual with a diagnosis of schizophrenia, but the principle at stake (whether psychoanalysis was an effective treatment for a severe and enduring psychological difficulty) was highly relevant to that diagnosis. Furthermore the case concerned treatment at, again, Chestnut Lodge. Many histories of psychiatry cite these events as key factors in the decline of psychoanalysis as a treatment for schizophrenia. One such (Edward Dolnick's Madness on The Couch) reads as a stern polemic, taking psychoanalysis to task for victim blaming. Another, Jeff Lieberman's Shrinks (review by me here), is a good example of how contemptuous many psychiatrists are about Freudian therapies and ideas.

So in one sense, Bollas' book looks like an anachronism. Few now expect psychoanalysis to ameliorate psychosis anymore, and Bollas is aware of the of how widespread is this view. He skirts the issue:

But goes on to makes a claim (a few pages later) which is quite definitie in nature and would require just the sort of outcome study that Bollas has recently repudiated:

This sort of sophism is irritating; does Bollas want to claim his approach is effective, or does he not? It's tempting to dismiss him altogether at this point, but does reading someone like Bollas have anything to teach us? I think it might.

It is not totally outlandish to suggest that R.D. Laing wrote one of the 20th century's best books about psychosis. The Divided Self is an unparalleled masterpiece of phenomenology (read it if you haven't already). Yet at the same time, there is also something approaching consensus (from what I have read) that his major therapeutic innovation, the Philadelphia Association, was something of a failure. Sure you can read Mary Barnes and Joe Berke's favourable account of a "Journey Through Madness", but when it comes to a more overarching view of the project (as found in Daniel Burston's very respectful biography of Laing), it doesn't look like the majority of people were helped any more than they would have been without the Scottish guru.

What Laing offered the study of psychosis then was not a viable alternative therapeutics (ultimately it's not clear that his notion of psychosis, as a revelatory journey one must past through, offered the sort of safety and serenity that might be optimal in a residential setting) but a powerful vision of how we can approach highly disorganized and disoriented people as people, with a degree of empathy and openness to listening to their experiences. Whatever packaged and marketed therapies are regarded as appropriately evidenced and offered in healthcare services, this sort of humane engagement is not only highly desirable, it is unavoidable. Whether you are discussing a person's medication, their experiences or their occupational aspirations, you need some way of taking them seriously and understanding what it might be like to live in their head. Laing's description of ontological insecurity, followed by the development and ultimate collapse of a protective "false self" system allowed clinicians to at least imagine what might lead to behavior which is so confusing as to be routinely described as "mad".

It is this kind of imaginative material that Bollas offers, and it is what makes his new book worthwhile for clinicians, even if they don't follow him all (or even most of) the way. Bollas and many other analysts (and not just analysts) may eschew the value of evidence in psychotherapy, but that doesn't invalidate everything they have to say. Whatever you think of the benefits of psychoanalysis as a therapy, it still has potential as a mode of observation and phenomenological hypothesis forming. None other than Paul Meehl once wrote a very effective argument to the effect that the un-testability of many psychoanalytic assertions does not itself render them untrue or worthless. If you take empathetic and highly attentive individuals (psychoanalytic clinicians) and put them in a situation in which they observe people's verbal behaviour over long periods of time, it seems highly implausible that they wouldn't derive any very insightful and useful ideas about how minds work.

Bollas' book has many such ideas. He suggests that the symptom of hearing voices is the result of "despositing" unwanted parts of the self in the environment such that they start to talk back to you, representing aspects of your past in ways that demand to be listened to. He describes the experience of the separation of the "I" from the "me", such that people with psychosis may split their self and behave in remarkable ways once they have done so . He suggests that people in the grips of a psychosis may be so frightened by their thoughts that they take an (often bizarre seeming) action to prevent having them. This sort of deep meditation gives rise to some remarkably empathetic moments:

Such intuitive accounts may be "wrong" (in the sense that they don't really get at what individuals experience) but they strike me as preferable to a defensive dismissal of people's "crazy" experiences. Many mental health services for psychosis currently isolate patients and ignore their experiences, alienating rather than engaging them. If social isolation is an exacerbating factor in the deterioration of people's mental health, we need to find ways of spending time with such individuals, even at their most disorganized and frightening. At least Bollas (and he is in a long line of psychoanalytic clinicians on this score) is trying his damnedest to connect.

Some of his ideas I found intuitive and quite striking, others (the obscure theory of "Metasexuality") I found maddeningly arcane. As Meehl (and many others) have pointed out, there may be no decent way to adjudicate between them, but unless you subscribe to a sort of rigid Vienna-Circle logical positivism in which statements about the self can only ever be empty "metaphysics", they don't seem entirely worthless. Are we only interested in ideas that are testable scientifically? Surely not; many highly speculative and unverifiable ideas (the dialectic movement of history) are valuable, so long as you have a clear distinction in your mind between them and some notion of "truth". We value novelists and poets who can illuminate their inner worlds through their vivid writing, why not try and find some similar worth in the imagination of psychoanalysts?

If you are still queasy about such unscientific shenanigans, it's worth remembering that a phenomenological account is not inconsistent with an informed empirical one. I'll end with a quote from one of Paul Meehl's best papers:

In Defence of the Psyche

There is an admirable current tendency in my profession (described passionately and elegantly here by Masuma Rahim), toward an appreciation of the many environmental factors which contribute to bringing people into our offices. It's as though the headshrinkers are only just learning that people aren't just miserable in isolation; things happen and make us miserable. This truism is ubiquitous at the moment, and has even become something of a slogan:

The corollary of this environmental turn is an increasing appetite for the rejection of diagnosis or an "illness" way of thinking; an assertion that facts about the individual are ultimately less important than facts about their past. Despite feeling closely aligned with this politicised version of my profession, I also feel some unease. Is there a baby somewhere in the bathwater? I suggest there is, for despite the indisputable importance of the outer world we all also inhabit a unique inner world, the land of the psyche. If there is one idea psychologists should be interested in it is surely this.

Right now hundreds of thousands of people are flooding out of Syria, escaping from some of the most harrowing events imaginable. Their journeys to asylum will not necessarily be any better. While we know that many refugees survive and even thrive in their new lives, many others will be psychologically devastated. Nobody can say why that is but inter-individual differences in the psyche have to be important. This is more than just to say that some people get lucky in the great individual-differences lottery. The psyche is important in understanding why life's horrors so completely overwhelm some people, but is also what accounts for why something as impractical as psychotherapy could make a difference to the lives of those who have suffered them.

Why do I say any of this? Clinical psychology has spent most of its past in denial of the environment. If we overcorrect at the expense of the psyche, several risks emerge:

1. A focus on the environment to the exclusion of the psyche is liable to promote a sort of therapeutic nihilism. If we believe that people in dire material straits can only be helped by material changes to their lives, we risk neglecting them. Ultimately we are capable of astonishing changes to our own lives. How we think plays a huge role in doing so. 

2. Following on from the issue of therapeutic nihilism is the issue of professional burnout. If you believe meaningful change is possible only through means which are beyond you, cynicism and overwhelm will not be far away. Therapists who lose all faith in their capacity to help people are on the road to confirming their own worst fears. 

3. Too great an emphasis on the environment is a form of reductionism analogous to "biologising". Just as a restrictive "disease" model leads to the belief that all should get medication, a restrictive environmental model could lead to the neglect of individual differences in therapeutic need. Some people's misery is intelligible almost entirely in terms of things which have happened to them. For most of us, the struggle is in the complex and all-too-human dance between problems foisted on us and problems we make for ourselves. 

One model for what I am pointing to is bereavement counselling. While we recognise that not everyone who experiences death is in need of it, we readily accept that some of us can be so rocked by the resulting grief that it is helpful to see a therapist. This does not entail that only immortality can stem the tide of human misery. Some miseries are best adjusted to. Even those miseries which are not best adjusted to (such foes as discrimination, economic inequality and political violence) are nonetheless pervasive.

Perhaps there is an argument in the near vicinity that I will be accused of making: that psychologists should not be political, but should get on with the job in hand. I am not making that case at all. Psychologists should be highly political, just as should any profession which takes people seriously. To the extent that society makes victims of some of its members, we should change it. However if the history of humanity has anything to teach us, it's that suffering is inevitable. To think otherwise is not political, it's utopian.

Politics come in part from theories about human nature, and psychologists have those in spades. There is simply no incompatibility between believing in large scale political action, while simultaneously asserting the value of small-scale individual change. Just as our social world impacts on our psyche, so too our psyche impacts on our social world.

The Hickey-Lieberman Test

Phil Hickey has an interesting post over at his blog about what happened when Jeff Lieberman was asked if psychiatry over-medicated people:

Absolutely.  I had an experience with my own son.  I have two sons.  My older son was going to nursery school, and they said he’s not paying attention and were concerned.  ‘You should have him tested.’  We had him tested.  The neuropsychologist said, ‘Well there’s some kind of, you know, information processing problems, you should see a pediatric psychiatrist.’  I said, “Well, I am a psychiatrist, but I’ll take him to see a pediatric psychiatrist.’  We took him to see a pediatric psychiatrist, spent twenty minutes with him, and he started, you know, writing a prescription for Ritalin.  I said, ‘Why?’ and he said ‘Well, he’s got ADHD.’   I said, ‘I don’t think so.’
So, long story short, he ended up graduating from University of Pennsylvania, law school at Columbia, he’s in a top law firm.  So, yes, it happens, and part of that is social pressure.

There is something very telling about this story. I think one intuitively sides with Lieberman's sense that ADHD is an unnecessary label under the circumstances. But how can one avoid such undesirable clinical encounters? It's tempting to prescribe a healthy dose of "common sense", but this is a questionably useful. I am sure the paediatric psychiatrist in the story felt they were applying common sense in their work by applying a DSM diagnosis, so invoking it only leads to a conflict between two people's notions of what is meant by that rough and ready notion.

I want to propose we use Hickey's report of Lieberman's anecdote to formulate a test for ourselves as mental health professionals. I call this the "Hickey Lieberman test" to recognise the role of Jeff Lieberman in articulating the problem, and Phil Hickey in transcribing Lieberman's story. The Hickey-Lieberman test should be applied in any situation in which a psychiatric or mental health intervention (be it diagnosis, prescription, therapeutic plan, change in living circumstance, or really any substantive change) is being considered. It consists of four questions the intervening clinician(s) should ask themselves before taking action.

The Hickey Lieberman Test: 

1.How would I react if this intervention was to be applied to me, or to someone I cared about?
2. What would be the basis of that reaction?
3. If I would react negatively to this intervention, can I nonetheless justify it in terms of converging lines of evidence that it is an appropriate course of action?
4. If the answer to 3 is no, what would I change about this intervention to make it more reasonable for the person toward whom it is directed? 

The Hickey-Lieberman test is hardly water tight. If you agree with my reservations about "common-sense" then you will notice that this proposal also contains a great deal of subjectivity. However, a formal test does demand at least a moment's thought. When people act in ways they think are concordant with "common-sense", they may often be rationalising after the fact rather than thinking in advance. Much like the "reasonable person" test applied in legal settings, the Hickey-Lieberman test draws on the notion of a shared agreement about what it means to behave sensibly, which isbetter than nothing. 

Reasons and Causes

Some quick thoughts about an alternative way of weighing up the status of psychiatric problems. The conventional controversy is organized around the question of whether such and such a set of behaviours constitutes an "illness". Much ink gets spilled defining "illness" and then asking whether any given problem meets the criteria. Could we instead make the determination on the basis of the mechanisms that have given rise to any given problem?

To some extent a version of this already happens. People who advocate for the use of psychological formulation want us to ask "what has happened to you?" Not a bad way to go about things, but infections and closed head injuries happen to people, and they have a place in the world of illness/medicine. A finer grained distinction may follow from Karl Jasper's division between things that can be explained and things that can be understood. 

Into the former (at least for Jaspers) fall "ununderstandable" phenomena like delusional beliefs (psychopathological because incomprehensible), while into the latter category fall emotions that arise as responses to events (sadness in response to loss). I am not saying we have to agree with Jaspers about delusional beliefs here (this post is not a bid to police what is and what is not understandable) I am just saying that it is, in principle, a potentially helpful distinction.

It brings us on then to thinking about aetiology, which could be thought of in parallel terms of reasons and causes. I have reasons when something that has happened to me "makes sense" of my behaviour/feelings in light of some culturally shared system of meaning (i.e. depression in response to bereavement). We seek causes where we suspect we need to go down one level of explanation.

We might say that a person who is afraid of dying has reason to not see a small painting of a skull hanging on the wall in front of them. A person with a scotoma occluding their view of the painting has had their inability to see caused by a biological event. Under this scheme, problems which are primarily caused would belong mainly to "mechanistic" forms of cure, while events which have reasons would belong to more narrative/psychotherapeutic approaches.

The distinction already starts to break down of course. An individual with Parkinsons has had the shaking in their hands caused by dopamine dysregulation in their basal ganglia, but the slow pace of their walking might be something they have reason for ("I would fall over if I tried to go any faster"). Equally, consumption of large quantities of some substances will cause certain brutally physiological physical problems, but the consumption itself may have socially-comprehensible reasons (drinking to numb some emotional pain). The tangle of the mechanistic and hermeneutic approaches will not be dissolved, but at least we might have a better way of talking about it. 

Rethinking Therapist Drift

One of the strangest ways I have ever been evaluated in my career was while delivering a psychological intervention in a prison. My job was to stand in front of a room of young offenders and facilitate group discussions of their "thinking skills". The prison service has a clear idea of what this should look like, indeed so clear that every session I did was filmed. Auditors could then check that I was sticking to the protocol. 

This remarkable surveillance was an attempt to minimize what is commonly called "therapist drift", the process whereby a therapist ostensibly delivering an evidence-based therapy winds up doing something else instead. Psychotherapies are not easy things to administer, and in the face of diverse people and problems it's easy to see how one might end up straying from the guidelines outlined in therapy delivery manuals. But if you claim to be doing, say CBT when you aren't in fact doing anything of the sort, you might not get the same results.

For the most part, therapist drift is regarded as a bad thing. This makes sense (at least, it's internally consistent). Drift is a problem for people who are trying to research a therapy (because they don't end up testing what they intend to) and it's potentially a problem for people who are trying to deliver a therapy (because rather than delivering something that has been demonstrated to be effective, they do something which is not).

But there is a strand of thought in clinical psychology and psychotherapy that maintains suspicion about the notions of "adherence" and "drift", and of evidence-based therapy altogether. Critics of this stripe view evidence-based-approaches as overly rigid and formulaic, too focused on technique at the expense of relationship. There is some good quality criticism of manualized therapy (here's a good example), but also much exaggeration about manualized therapies making the process "robotic" (as though a set of instructive principles were incompatible with being human). 

Recently I looked into the topic (in a very non-systematic way) to see what research had found about the importance of adherence and drift. Most studies that have been done (and it is surprising how few there are that focus specifically on drift) seem to support the contention that "drifting" can lead to less impressive outcomes. However, one study had an intriguing result.

Examining CBT for panic disorder, researchers (Jonathan Huppert and colleagues) took measures of patient motivation (rated by the therapists) and adherence to the therapy manual (rated by listening to audiotapes of sessions). Perhaps counter-intuitively, the researchers found that among highly motivated patients, the therapist adherence did not have much impact on outcome (look at the graph and you can see the blue line only slopes upward a small amount; this difference was not found to be significant). However, among less motivated patients, adherence was associated with worse outcomes than drift.*

One possible explanation for this (one that Huppert and colleagues themselves suggest) is that patients with low motivation present an extra degree of complexity which cannot be adequately addressed by staying within the set protocol. An experienced therapist will depart from the standard protocol to address in some way the low motivation, before continuing with the planned process. Under this interpretation, the therapists who show the greater adherence with the "low-motivation" patients are paradoxically failing to do something with the low adherence therapists are succeeding at it. Although they are moving beyond the purview of the manual, it seems misguided to call this "drift". This squares with the extended discussion by Drew Westen and colleagues on the more tendentious implications of therapy manuals:

...manualization commits researchers to an assumption that is only appropriate for a limited range of treatments, namely that therapy is something done to a patient—a process in which the therapist applies interventions— rather than a transactional process in which patient and therapist collaborate. (p.639)

And yet still some idea of therapist "drift" seems important. Unless we believe there is no value to specific training for psychologists and psychotherapists, we want to have some reasonably defined sense of what we're up to; some sense of what it looks like to do the job properly. It is any deviation from this that can reasonably be considered drift. In other words: in the space between conforming, robot-like, to a predetermined protocol and doing whatever the hell you want, there lies a knowable range of skills which we ideally would want to adhere to. That set of skills is what constitutes being a good psychologist. This definition extends the realm of evidence based practice some way beyond the parameters of individual evidence based treatments. 

_________________________________________________________

*Though the authors note: "it is important to keep in mind that even when therapists were rated as less adherent, adherence was still rated as “good” or better, suggesting that therapists did not cease doing CBT for panic, but likely incorporated other strategies into their armamentarium" p.202