July 2004
392
The Psychologist Vol 17 No 7
The myth of
evidence-based
psychotherapy
E
VIDENCE-based psychotherapy
seems a reasonable aspiration.
There are many weird and
wonderful treatments in the field; surely it
makes sense to know whether or not they
work? This was the rationale behind the
Department of Health's Task Force on the
Psychotherapies, which I was briefly part
of. It led to an impressively documented
book by Fonagy and Roth (1996) in which
the research supporting various
psychotherapies was presented and
thoroughly analysed, and conclusions were
drawn about specific treatments for specific
conditions. Yet I want to argue that
evidence-based psychotherapy is a myth.
I am not against scientific research in
psychotherapy. On the contrary, I believe
that scientific advances in psychology and
related disciplines are important to the
development of psychological therapies.
But that is different from claiming that
what psychotherapists do is, or should be,
securely founded in evidence of
effectiveness; that, for example, we can
say with authority that depressed people
are most likely to benefit from cognitive
therapy or that research has shown that 90
per cent of people with panic attacks will
recover with anxiety management. Such
claims are in my view misleading and
simplistic, and it is this `outcome research'
I have a problem with. It does justice
neither to the complexity of people's
psychology nor to the intricacies of
psychotherapy.
The problem with psychiatric
diagnosis
To carry out outcome research one needs to
categorise conditions in a reliable and valid
way, for the obvious reason that research
findings need to be generalised.
Psychotherapy research uses one of two
parallel psychiatric classifications systems
­ the American Psychiatric Association's
Diagnostic and Statistical Manual (DSM)
or the World Health Organization's
International Classification of Diseases.
These classifications have been derived
from original diagnostic distinctions made
by Emil Kraepelin in the 19th century that,
as Richard Bentall (2003) has recently
shown, were fundamentally flawed. The
lines drawn to distinguish different
psychiatric conditions are far from firm.
There is a huge overlap of symptoms
between so-called illnesses. There is
a strong consensus element in defining
some conditions as a psychiatric illness,
most obvious when in 1974 members of
the American Psychiatric Association
voted that homosexuality should no longer
be in the DSM lexicon.
But the most important point is that
psychotherapists, anxious to prove that
what they do works, have bought into a
medicalised way of defining psychological
experience. They act as though it is correct
to state that people have depression or
anxiety or schizophrenia like they have
measles or diabetes or heart disease. But
for all of these conditions, including the
more extreme psychotic states, it is
impossible to divorce the condition from
the person. The experiences that lead
people to be diagnosed as `mentally ilľ are
experiences that all of us can have in some
form or at some stage. This is why, despite
over a century or more of research,
psychiatrists are no further forward in
defining and understanding ­ let alone
successfully treating ­ any major
psychiatric `illness'. If you don't believe
me, read Bentalľs excellent book, Part 1
in particular.
This uncomfortable fact opens up a
huge black hole for outcome research into
psychotherapy. For if one cannot validly
categorise experiences as illnesses, how
can we generalise what we find in any
research study? If so-called symptoms
spread across diagnostic categories like
spilled ink flowing over paper, how do we
reliably differentiate between conditions?
If psychological problems are not treatable
like medical ones, what sense does it make
JOHN MARZILLIER argues against `outcome research'.
People do not have depression or anxiety or schizophrenia like they have measles or diabetes
or heart disease
of the assertion that a treatment `works'?
This has always been obvious to anyone
who has practised as a psychotherapist as
opposed to researching into it. People who
come for help for a specific problem
almost invariably have other problems
often to do with their personality, social
conditions, emotional experiences or way
of life.
As an illustration, take the two
examples of phobic anxiety described in
the box, both people I was seeing at the
time of writing this article. They have
different personalities, experiences and
circumstances and respond differently to
therapy. What psychological sense is there
is in throwing these people together as
though their problems were solely defined
by their `symptoms'?
The limitations of outcome
research
In a survey of the Psychotherapy Division
of the American Psychological Association
about what psychotherapists found most
useful to their practice, 48 per cent said
clinical experience; only 4 per cent said
research. Research publications came a
lowly eighth out of ten most important
influences on clinicians (Morrow-Bradley
& Elliott, 1986).
This is not perversity on the part of
psychotherapists. Drawing conclusions
from outcome research into psychotherapy
is highly problematic. Not only is it
regularly shown in the academic literature
that any given outcome study has
significant faults, usually fatal to any
conclusions, but also initially positive
findings for a therapeutic approach can be
reversed a few years later. This happened
to Carl Rogers's claims for empathy,
warmth and genuineness as the core
conditions for all therapies. A 1971 review
of research gave them the seal of approval,
only to be retracted in a revised assessment
six years later (Mitchell et al., 1977).
Moreover, paradoxically, the better
quality the research, the less value is it to
clinicians. For example, the Sheffield
Psychotherapy Project had an exemplary
research design (Shapiro & Firth, 1987).
A selected sample of anxious and depressed
individuals from the professional and
managerial class was randomly assigned to
two contrasting therapies, exploratory and
prescriptive, in a crossover design, so that
after eight sessions of one therapy the
clients received eight sessions of the other.
Therapists were carefully trained, followed
specially written manuals and had all the
sessions recorded and assessed. A battery
of well-chosen assessments was used.
The researchers were able to compare
the effectiveness of the two therapies, the
results slightly favouring the prescriptive
therapies on some measures. To the
clinician however these results mean very
little. Not only was the subject sample
highly selected at the outset, but 50 per
cent of these did not get beyond the initial
assessment. No therapist would switch
therapies in mid-stream without recourse
to the clienťs needs. And what about the
effects of therapy manuals, continuous
recording and frequent assessments on the
whole process? The very procedures
designed to improve the research distanced
it from the realities of clinical work.
Few psychotherapists change their
practices merely because research studies
favour one therapy over another or show
that a particular therapy does not work. Are
they being blinkered? Perhaps. But mostly
their scepticism is justified. For example,
there has been a recent backlash against
psychological debriefing, the method of
providing group psychological support for
emergency personnel after trauma. It has
been suggested that not only does it not
work but also it can make people worse
(see Wessely & Deahl, 2003, for a debate).
Yet anyone studying the two controlled
trials (yes, just two trials) with negative
research findings will see a glaring lacuna.
Neither used a group procedure, the
essence of the psychological debriefing
approach: people were seen for individual
counselling. Do researchers want clinicians
to believe that individual counselling is the
same as group work? Should those of us
who use group debriefings stop doing so
on such flimsy and flawed evidence?
The aspiration to discover what works
is laudable. The reality is very different.
The whole business of categorising
people's problems into quasi-medical
and suspect diagnoses is psychologically
July 2004
393
The Psychologist Vol 17 No 7
Evidence-based psychotherapy
A TALE OF TWO PHOBIAS `JACKIE'
AND `PENNY'
Jackie, a young mother of a two-year-old, referred herself to me for treatment of her panic attacks. She had
a long-standing fear of people vomiting that was treated years ago, unsuccessfully, by behaviour therapy. She
was now concerned about her husband or daughter vomiting and engaged in a plethora of `safety
behaviours'. Further assessment revealed chronic social anxiety, lack of self-confidence, dislike of her physical
appearance and depression. She had been bullied at school.As the middle child she felt neglected by her
parents, her mother being particularly critical of her. Since the birth of her own child, her relationship with
her husband had become strained.When the husband attended, it was clear that Jackie's anxiety was far
more extensive than she had first presented. She hadn't taken her daughter out for two weeks. She shouted
at her husband if he talked to the next-door neighbour for fear that the neighbour would become too
friendly. She refused to see any of her in-laws.
I felt that it might be difficult to help Jackie. Her anxieties were extensive, and previous therapy had
failed to make an impact.Yet she was desperate.We began anxiety management. She did reasonably well
initially, but her general anxiety remains high. She clings on to therapy, finding solace in the weekly support. I
worry that when therapy ends, she may well lose the gains she has made, and I wonder how to prevent that
happening.
Penny, a postgraduate student in her early twenties, sought help because of panic attacks. Since returning
from her summer vacation she had experienced a number of panic attacks, once during an examination,
then when giving a paper, and again during a seminar. On each occasion she became worried that she would
need to leave to go to the loo but had never needed to do so. Characteristically, she had begun to be
anxious about her anxiety. She had read about anxiety management and wondered if it might be helpful. I
established that academically she was doing fine, having passed the exam, and she was enjoying doing her
doctoral work. She came over as a self-assured, lively and insightful person. She was in a long-term
relationship, which, while not perfect, was strong enough.There were past traumas that seemed a possible
precursor of her presenting anxiety.
At Penny's request we began anxiety management, and within two weeks her anxiety had disappeared.
Penny then talked about the trauma and later about her difficult relationship with her mother, her absent
father and her boyfriend. She flirted with me in the sessions and expressed an interest in becoming a clinical
psychologist.Aware of the transference elements, I suggested she might consider weekly psychodynamic
therapy, but after reflection, she decided not to take up my offer.
dubious, and outcome studies in
psychotherapy tell clinicians nothing of
value. In over 30 years of
psychotherapeutic work not one outcome
study has influenced my practice to any
significant degree. The most one can say,
cynically, is that it can be useful to cite
outcome studies when defending one's
practice against critics. The apogee of this
self-serving stance came in a review of
outcome research (Luborsky et al., 1975)
that concluded that all psychotherapies
were more or less equally effective, or as
their heading ironically stated, `Everybody
has won and all must have prizes'. It is of
course vacuous nonsense to assert that all
psychotherapies work equally well, though
useful if third parties like insurance
companies begin to cavil about paying for
psychotherapy.
An essentially personal
relationship
Psychologists have largely ignored the
essential component of psychotherapy,
the personal relationship. There has been
research seeking to link the personal
qualities of therapists to outcome, usually
unsuccessfully, but this is subject to the
same problems and criticisms as outcome
research. Like friendship, romance or
chatting to someone in your local shop,
psychotherapy is at heart a personal
transaction. It has some special features,
notably that it is a professional relationship,
that one person remains more or less a
stranger while the other may reveal very
intimate matters. But at heart, the personal
exchange defines psychotherapy. All else
flows from it.
Consider these micro-examples, all
taken from my own experience. A woman
whose husband committed suicide breaks
into uncontrollable sobbing. I feel
uncomfortable leaving her to cry without
making some attempt to console her.
A man, who is very acerbic in his manner,
ridicules all my therapeutic comments as
`typical rubbish that psychologists spouť,
making me feel angry and frustrated.
A woman, experienced in psychotherapy,
comes in with a gift of homegrown
tomatoes to thank me for my help and
is furious when I politely decline the gift.
A man, starting a course of exploratory
psychotherapy, wants me to start the
session off, claiming the silence at the
beginning is persecutory. A man writes
me a heartfelt letter wondering if he is
depressed and wanting to know more about
what depression is. I could go on. These
examples of the minutiae of the therapy
process illustrate how limited a textbook
or therapisťs manual is. In each case the
therapisťs personal response will greatly
influence the way the relationship develops.
Imagine the difference between hugging
the bereaved woman and sitting back and
saying nothing. Or accepting the gift of the
tomatoes. Or offering to start the session
oneself. The personalities of both the
therapist and the client exercise a huge
influence on what can be achieved and how.
How the therapist responds to clients is
as much dependent on his or her skills and
style as any technique. How can such
complexity be reduced to a set of
predetermined treatments or manualised
procedures? When I am asked to
recommend a therapist to someone, my
first thought is about the personal qualities
of the therapist. My next is about how
experienced they are and whether they
would suit the client and help them
with their problems. Not very scientific,
but pretty sensible all the same. Recently
I overheard a conversation between
psychologists in training in which one said
if she had therapy herself, she would go to
a psychodynamic therapist. Why, said the
other, if cognitive therapy is so successful?
Because with a psychodynamic therapist
you would be able to explore what really
mattered to you, you can tell your story.
Over years of working as a
psychotherapist I have come to realise that
what many people need is to tell their story.
I have become adept at attentive listening.
Not just to the litany of symptoms or
problems, but to the nature of the person,
the context, their hopes and desires. I try to
help them make sense of what is happening
to them, and not just label their experiences
as symptoms and try to get control of them.
I establish a carefully boundaried personal
relationship. I get to like my clients, some
of them more than others. Most I can help
in some way, judging by what they say to
me. Some I can't help, but not for want of
trying. I seek to behave honestly and treat
people in a way that merits the confidence
they show in me. Sometimes, despite
myself, I act as a quasi-doctor dispensing
advice and practical help, giving self-help
booklets or relaxation tapes. Sometimes
well-worn phrases or oft-repeated stories
fall from my lips, reassuring like the sound
of rushing water and probably as effective.
At other times I find myself engaged in
an engrossing struggle where emotions
run high and the relationship is repeatedly
challenged. These actions cannot be
reduced to a series of technical procedures,
nor can they be scientifically proved, just
as you cannot prove scientifically the value
of friendship or a good marriage.
You are probably thinking that I have
become one of those people who airily
claim that psychotherapy is an art and that
science has no part to play in it. Clearly
I do think there is an art to therapy, to do
with personalities, human relations and
engagement in the world. But this does not
mean that scientific research has no part to
play in exploring these processes. Graham
Davey (2003), in his Presidential Address,
outlined the way psychological research
can be conducted into psychopathology
to provide clinicians with insight into
underlying processes. In ruminative
anxiety, for example, psychologists have
examined the way certain `stop-rules' that
is, cognitions that influence whether
ruminations continue or not ­ are affected
July 2004
394
The Psychologist Vol 17 No 7
Evidence-based psychotherapy
Even something as small as the decision by a therapist to accept or decline a gift of
homegrown tomatoes can influence the way the relationship develops
by mood, thereby helping clinicians unpick
what happens in the complex transaction
between thoughts and feelings. It is useful
to know this, not because the research
provides definitive answers, but because it
offers a way of understanding a complex
process (ruminations) and some new
information (under conditions of high
anxiety a particular stop-rule leads to
further ruminations). In the experimental
lab hypotheses can be tested scientifically.
In therapy they can be tried out clinically.
The interaction between the two can lead
to both theory development and new
therapeutic practices. The psychotherapist
need not give up being a psychologist. This
sort of psychological research exploratory,
to do with processes, human
experience, personal interaction, underlying
mechanisms ­ is obviously relevant to
therapists since that is what they deal with
every day.
Many years ago Jerome Frank (1961)
pointed out the close parallels between
psychotherapy and other forms of
persuasion and healing, stressing what he
called `non-specific factors' as the main
agents of its effectiveness. The personal
power of the healer, the beliefs of the
therapist, the perceived credibility of the
knowledge base, the ritual of the
procedures, the place of safety where the
therapy takes place. All of these are of
course psychological, yet psychologists
have generally turned their backs on these
factors. Instead they rushed into the blind
alley of outcome research in the vain hope
that, like their medical colleagues, they
could claim scientific justification for the
practice of psychotherapy. Is it not about
time we acknowledged the error and admit
that evidence-based psychotherapy is a
myth?
 John Marzillier is a clinical
psychologist and psychotherapist in
Oxford. E-mail:
john.marzillier@ntlworld.com.
July 2004
395
The Psychologist Vol 17 No 7
Evidence-based psychotherapy
Bentall, R. (2003). Madness explained:
Psychosis and human nature.
London:Allen Lane.
Davey, G. (2003). Doing clinical
psychology research.The
Psychologist, 16, 412­416.
Fonagy, P. & Roth,A. (1996). What works
for whom? A critical review of
psychotherapy research. NewYork:
Guilford Press.
Frank, J. (1961). Persuasion and healing.
Baltimore, MD: Johns Hopkins
University Press.
Luborsky, L., Singer, B. & Luborsky, L.
(1975). Comparative studies of
psychotherapies: Is it true that
`Everyone has won and all must
have prizes?' Archives of General
Psychiatry, 32, 995­1008.
Mitchell, K.M., Bozarth, J.D. & Krauft,
C.C. (1977).A reappraisal of the
therapeutic effectiveness of
accurate empathy, non-possessive
warmth and genuineness. In A.S.
Gurman & A.M. Razin (Eds.)
Effective psychotherapy:A handbook
of research. NewYork: Pergamon
Press.
Morrow-Bradley, C. & Elliott, R. (1986).
Utilization of psychotherapy
research by practicing psychologists.
American Psychologist, 41, 188­197.
Shapiro, D. & Firth, J. (1987).
Prescriptive v. exploratory
psychotherapy: Outcomes of the
Sheffield Psychotherapy Project.
British Journal of Psychiatry, 151,
790­799.
Wessely, S. & Deahl, M. (2003).
Psychological debriefing is a waste
of time [Debate]. British Journal of
Psychiatry, 183, 12­14.
References