Evidence-Based Practice: Beyond Empirically Supported Treatments
Stanley B. Messer
Rutgers University
Must the clinician choose between a practice that is strictly objective and data based and one that is purely
subjective and experience based? Optimally, practitioners need to follow a model of evidence-based
psychotherapy practice, such as the disciplined inquiry or local clinical scientist model, that encompasses
a theoretical formulation, empirically supported treatments (ESTs), empirically supported therapy relationships,
clinicians’ accumulated practical experience, and their clinical judgment about the case at
hand. Some shortcomings of ESTs are reviewed, and a form of evidence for psychotherapy practice is
presented that entails the accumulation of systematic case studies published online. Practitioners can
contribute to such a database and be guided in their practice by those cases most relevant to their clients’
problems.
Mrs. T is a 42-year-old woman who is married with two teenage
sons and one preteen-age daughter. She is of European background
and grew up abroad. On September 11, 2001, Mrs. T witnessed the
second plane crash into the World Trade Center while knowing
that several of her close associates were in that building for a
meeting. At the time she had a very responsible human relations
position in a corporation in which she did the hiring. In this role,
she arranged the funeral services for two of the employees and
served as the liaison for their families, taking care of matters such
as insurance and death certificates. She was also an emotional
support for the bereaved families.
From a few months after that time and the subsequent loss of her
managerial job, she has had frequent crying spells, anxiety attacks,
and a sense of despondency. She finds that her thoughts are
disconnected, making her unable to focus on any task for very
long. Beset by physical problems such as high blood pressure and
a severe facial rash, Mrs. T has experienced chest pains and heart
palpitations causing her to feel as if she were having a heart attack.
Complaining of diminished libido, she has less interest in being
intimate with her husband, with whom there have been increased
marital tensions. Since shortly after 9/11 Mrs. T has been less
involved in social activities, preferring to be by herself. Her
feelings, except for grief, sadness, and irritability, have been
numbed.
Mrs. T came to see me in a very distraught, tearful state despite
a year having elapsed since 9/11. Referring to her experience at
that time, she told me that she had learned that one of her
associates was on fire when she left the building and died shortly
afterward. Mrs. T still imagines herself in conversation with the
deceased employees, who were also her friends. The 9/11 events
are replayed in her mind both in the waking state and in her
disturbing nightmares. Mrs. T believes that she was indirectly
responsible for her associates’ deaths, which has been one important
focus of therapy.
Because of the business downturn in New York City after 9/11,
her firm decided not to do any more hiring and Mrs. T was let go.
The loss of her high-paying, challenging position was a big blow
to her self-esteem. Although she made an effort to find other
employment, she was not successful, and she soon became unable
to pursue it further because of her increasingly debilitating symptoms.
Her financial situation deteriorated, made worse by her
husband’s losing his job as well. She first sought treatment with a
psychologist a few months after 9/11, which was helpful but which
she had to terminate when she no longer had insurance coverage.
She also has been seen by a psychiatrist who prescribed antidepressant
and antianxiety medications. Although Mrs. T is not yet
fully recovered after 1 1⁄2 years of therapy, she is much less subject
to anxiety and mood swings, is working part time, and is functioning
better on a day-to-day basis.
Is an Empirically Supported Treatment Sufficient
for Mrs. T?
For the past decade there has been a culture war raging over the
value and even ethical imperative of practicing empirically supported
treatments (ESTs). ESTs refer to those therapies that have
been shown to be efficacious in treating specific disorders, based
on the American Psychiatric Association’s (1994) Diagnostic and
Statistical Manual of Mental Disorders (4th ed.; DSM–IV), in two
randomized controlled trials (RCTs) or in a series of single-case
design experiments (Task Force on Promotion and Dissemination
of Psychological Procedures, 1995). On one side of the divide are
those who advocate that ESTs be preeminent in clinicians’ practice
(e.g., Chambless & Hollon, 1998; Sanderson & Rego, 2000); on
the other are those unwilling to cede the field to what are typically
DSM-oriented, manual-based, cognitive–behavioral therapies
(e.g., Bohart, 2000a; Levant, 2004). Mrs. T clearly suffers from
STANLEY B. MESSER received his PhD in clinical psychology from Harvard
University in 1968. He is a professor of clinical psychology and dean of the
Graduate School of Applied and Professional Psychology at Rutgers University.
A practicing psychologist, his interests include theoretical, empirical,
and philosophical aspects of psychotherapy.
I THANK Daniel Fishman for his help on earlier versions of this article.
CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Stanley
B. Messer, Rutgers University, Graduate School of Applied and Professional
Psychology, 152 Frelinghuysen Road, Piscataway, NJ 08854. Email:
smesser@rci.rutgers.edu
Professional Psychology: Research and Practice Copyright 2004 by the American Psychological Association
2004, Vol. 35, No. 6, 580–588 0735-7028/04/$12.00 DOI: 10.1037/0735-7028.35.6.580
580
posttraumatic stress disorder (PTSD) and probably fits other diagnostic
categories as well. Is there an EST for this disorder and, if
so, would it suffice to restore Mrs. T’s functioning? I return to
these questions after examining ESTs and the culture war between
more humanistic and more scientific psychological outlooks (see
Kimble, 1984).
I use the term culture war because the controversy taps into
broad worldviews in matters psychological that divide many applied
psychologists and that predate the current acrimony over
ESTs. These outlooks include but are not limited to the following
contrasting pairs, the first of each being the more “humanistic”:
subjectivism versus objectivism (Woolfolk, 1998), contextualism
versus atomism (Slife, 2004), hermeneutics versus universalism
(Messer, Sass, & Woolfolk, 1988; Slife, 2004), idiographic versus
nomothetic (Allport, 1937), and qualitative versus quantitative
method (Denzin & Lincoln, 2000). In reference to ESTs, what this
amounts to is whether RCTs and experimental, single-case studies
yield more useful information for practitioners than the combination
of philosophical outlook, theory, other research sources, and
practical experience on which most practitioners rely.
These dichotomies are overdrawn insofar as not every psychologist
falls neatly on one or the other side of the divide. In addition,
alternatives now available for practitioners to draw on for an
empirically oriented practice are studies on empirically supported
relationships, or ESRs (Norcross, 2002). The ESR literature focuses
on people, not disorders, namely, the role of the therapist, the
client, and their interaction in contrast to the ESTs’ emphasis on
techniques and treatment packages. For example, three general
elements of the therapeutic relationship that have been shown to
correlate with therapy outcome are the quality of the therapeutic
alliance, therapist empathy, and agreement and collaboration
around goals.
In addressing this culture war as it pertains to the practice of
psychotherapy, I first review shortcomings of ESTs in an effort to
demonstrate that their wholesale adoption by practitioners is not
warranted—although neither should they be ignored. In doing so,
I summarize and update some arguments that I have presented
elsewhere (e.g., Messer, 2001a, 2002). In the hope that we have
arrived at a point at which the battle has exhausted both sides and
some form of a truce, if not peace agreement, is possible, I discuss
two helpful ways of viewing the researcher–clinician divide.
These offer a viable middle ground for the practitioner interested
in evidence-based practice (the now preferred and broader term
than ESTs). They include Peterson’s (1991) disciplined inquiry
model and Stricker and Trierweiler’s (1995) concept of the local
clinical scientist. The case of Mrs. T is then revisited to demonstrate
the importance of going beyond ESTs in clinical practice.
The article points to some promising uses of the pragmatic case
study method that may also contribute to narrowing the gap
between research and practice, science and humanism. Finally,
implications for practice are spelled out.
Some Shortcomings of ESTs
Randomized Clinical Trials: The Gold Standard?
Luborsky et al. (1999) examined 29 RCTs comparing one therapy
with another and found a correlation of .85 between researchers’
therapy allegiance and outcome. That is, there was a very
substantial association between the researcher’s preferred therapy
model and the therapy that was more successful. It emerged
despite the fact that differences in efficacy between the therapies
were rather small and clinically insignificant to begin with. Because
behaviorally oriented researchers carry out the great majority
of research on ESTs, differences that are found between
cognitive–behavior therapy (CBT) and other therapies may be due
to researcher allegiance (in essence, a type of “experimenter bias”)
rather than to a feature specific to the behavioral ESTs. For
example, it may be the case that researchers’ theoretical persuasion
leads to their favorite therapy being administered in the studies
with more fidelity and enthusiasm than those with which it has
been compared.
The RCTs on which ESTs are based are problematic in certain
other respects. A case in point is Westen and Morrison’s (2001)
meta-analysis of high-quality RCTs of panic disorder, generalized
anxiety disorder, and depression. Most of the treatments were
cognitive–behavioral ESTs. They found a high exclusion rate from
clinical trials for all three disorders: “two thirds of patients who
present for treatment of the disorder are excluded, and the more
patients excluded and the more stringent the exclusion criteria, the
more successful the treatment” (p. 884). Could it be that ESTs are
based on rather select patient samples? If patients with comorbid
disorders (e.g., both a DSM–IV Axis I and Axis II disorder) are
frequently excluded, to what extent is the research on which ESTs
rest truly applicable to clinical practice where such patients are not
typically refused treatment? On the other hand, somewhat reassuring
in this connection are results reported by Shadish, Matt,
Navaro, and Phillips (2000) that there is a high degree of similarity
between the research samples and those seen in clinical practice.
Similarly, Stirman, DeRubeis, Brody, and Crits-Christoph (2003)
found that 80% of the outpatients treated in the community would
have been eligible for at least one published RCT—although it
should be noted that 58% of these patients had the milder diagnoses
such as adjustment disorder or dysthymia, which are poorly
represented in the psychotherapy outcome literature. Apparently,
the verdict is still out on the issue of the generalizability of ESTs
to normative practice.
Of equal concern is Westen and Morrison’s (2001) finding that
for depression and generalized anxiety disorder, the average patient
maintains a mild but clinically significant level of symptoms
after treatment with an EST. Furthermore, only 40% of these
patients who undertake these treatments (which include the dropouts)
gain from them. Of the patients with depression who completed
treatment, the number who improved and remained improved
after 1 to 2 years was about 37%. If the patients who began
treatment but did not complete it are included, the improvement
rate at 2 years is only 27%. Data for panic disorder are more
encouraging but not outstanding: Half of those patients who completed
treatment can expect to remain improved at 2 years. Comparable
data for generalized anxiety disorder are not available.
Note, too, that somewhere between one quarter and one half of
patients treated for these disorders seek further treatment within
12–18 months, and half do so within 2 years.
These studies suggest that there is reason to be cautious about
the claimed virtues of ESTs, or any therapy for that matter. Nathan
(2001) speculated that longer continuation of psychosocial treat-
581SPECIAL SECTION: EVIDENCE-BASED PRACTICE
ments for these disorders, as is often the case for treatment with
medication, might yield better results. Generally speaking, the
results of short-term therapy (15–25 sessions) are quite good at
termination and up to 1-year follow-up (e.g., Anderson & Lambert,
2001), but it may be that long-term therapy is needed to maintain
long-term gains, especially for the more severe mental disorders.
Validity of Non-EST Therapies
The fact that some therapies have been shown to be efficacious
in RCTs does not mean that others are invalid. Meta-analyses
(Smith & Glass, 1977; Wampold et al., 1997) and meta-analyses of
meta-analyses (called mega-analyses, e.g., Grissom, 1996; Luborsky
et al., 2002) have found very few differences among those
therapies referred to by Wampold (2001) as bona fide. By this
term, he means therapies that have a firm theoretical structure,
have been practiced extensively over time, and have a research
foundation even if this foundation does not include the RCTs as
defined by the Task Force on Promotion and Dissemination of
Psychological Procedures (1995). Psychodynamic therapy, clientor
person-centered therapy, and family or couples therapy are all
bona fide by this definition. Given the results of the meta-analyses,
it is very likely that bona fide therapies will be found as helpful as
the current ESTs for many diagnostic disorders and nondiagnosable
problems, although this remains to be proven. By my count,
41 of the 49 ESTs listed by the Task Force update (Chambless et
al., 1996) are either behavioral or cognitive–behavioral, but when
direct comparisons with other bona fide therapies have been made
in the meta-analyses mentioned above, typically the behavioral or
cognitive–behavioral therapies have not been found to be more
efficacious. These analyses are not cited in the approved EST lists
because the meta-analyses are often conducted on studies that have
not targeted one specific DSM-based disorder. In such studies,
diagnostic groups such as adjustment disorders, major depression,
and anxiety disorders have been combined, which has disqualified
them. Therefore, it is not the case that the non-CBT therapies are
without any type of empirical support for their efficacy.
Even in those instances in which there was an advantage to the
behaviorally based ESTs, closer inspection has revealed this to be
incorrect. Wampold, Minami, Baskin, and Tierney (2002), for
example, meta-analyzed therapies for depression and found CBT
to be superior to the noncognitive and nonbehavioral therapies.
When they separated the latter therapies into two groups—those
that were bona fide treatments and those that were not (such as
supportive counseling with no theoretical framework)—the superiority
of CBT turned out to be an artifact of including non–bona
fide therapies in the comparisons. In other words, CBT was not
significantly more beneficial than noncognitive and nonbehavioral
treatments that were intended to be therapeutic rather than merely
serving as a convenient control group for the researchers’ favored
therapy.
Process and Outcome Criteria for ESTs Versus Other
Therapies
On another front, there is a mismatch between the nature of the
treatment emphasis of ESTs and those therapies that are not
cognitive–behavioral. The latter, including such approaches as
psychodynamic, experiential, and existential, focus more on the
process of therapy compared with CBT, which sets its sights more
directly on presenting problems and outcomes per se (Gold, 1995).
Stated differently, in CBT the aim is to modify the psychiatric
disorder or its symptoms as directly and efficiently as possible.
The process-oriented therapies, on the other hand, view symptomatic
changes as occurring indirectly through exploration of emergent
themes, schemas, or unconscious motives and beliefs. They
seek to resolve complex intrapsychic conflicts, personality difficulties,
or interpersonal maladaptive patterns through insight and
affective experience in the therapy. Therapists encourage a process
that leads patients to an awareness of their potential for selfdirection
rather than focusing them along more preset lines in
accordance with the guidelines of a manual or specific technique.
There is a process of discovery and meaning-making to help enrich
the patient’s self-experience and awareness. Outcome criteria used
are different, too, encompassing variables not directly tied to
presenting symptoms and DSM–IV categories, such as attainment
of insight, a sense of agency, a firmer identity, higher self-esteem,
a greater recognition and better handling of feelings, greater ego
strength and self-cohesion, and increased pleasure and serenity in
living life (McWilliams, 1999).
Process therapies tend to emphasize the competence, skill, and
personal qualities of individual therapists and their impact on the
therapeutic alliance more so than specific techniques or ingredients
such as cognitive reframing or anger management that are characteristic
of the ESTs. These therapist and therapist–client variables
are sometimes referred to as common factors that cut across
the different therapies. Here is a brief summary of the empirical
findings regarding these two kinds of variables (Messer & Wampold,
2002):
Common factors and therapist variability far outweigh specific ingredients
in accounting for the benefits of psychotherapy. The proportion
of variance contributed by common factors such as placebo effects,
working alliance, therapist allegiance and competence are much
greater than the variance stemming from specific ingredients or effects.
(p. 23)
(For specific empirical studies documenting this conclusion, see
Wampold, 2001.)
A common factors or contextual approach (Frank & Frank,
1991; Wampold, 2001) does regard specific types of therapeutic
interventions as necessary to the conduct of therapy, but their
purpose is conceptualized quite differently than in the EST literature.
The purpose of such specific ingredients or techniques is to
construct a coherent treatment—be it cognitive–behavioral, psychoanalytic,
existential, or otherwise—that therapists believe in
and that provides a convincing rationale to clients. Furthermore,
these specific ingredients cannot be studied independently of the
healing context and atmosphere in which they occur.
In terms of clinical practice, the medical model on which ESTs
are based says, “Seek a therapist who uses techniques with demonstrated
ability to alleviate your condition,” whereas the contextual
(common factors) model advises, “Seek an interpersonally
competent therapist who uses a treatment approach you find compatible
with your worldview.” The former puts more reliance on
the value of targeted treatment ingredients, and the latter more on
the personal qualities of the therapist and the fit between the
582 MESSER
client’s and therapist’s worldviews as expressed in a theoretical
orientation.
Two Models of Clinical Practice
The Disciplined Inquiry Model
Does the clinician have to choose between ESTs and ESRs,
between a scientific and humanistic practice, and between reliance
on nomothetic findings and the idiographic richness of the individual
case? I do not believe so. Because of the complexity and
contextual features of each applied case, psychologists can only
practice responsibly by drawing on knowledge available from
many sources (e.g., Goodheart, 2004). What is the relationship,
then, between science and practice, and how does this bear on
evidenced-based practice?
Figure 1 illustrates Peterson’s (1991) disciplined inquiry model
(see also Fishman, 2001). The first step is assessment of the client,
which is based on a theory or “guiding conception.” The assessment
is then used by the practitioner to create a specific formulation
of the client’s situation, frequently involving a reframing of
the issues the client initially presented. Most relevant for the issue
of evidence-based practice, the assessment and formulation also
rely on clinicians’ knowledge of relevant empirical research and
their mental storehouse of similar cases. There are now several
empirically supported assessment approaches that are applicable to
the formulation of individual clinical cases. These include, for
example, the core conflictual relationship theme, cyclical maladaptive
patterns, and cognitive–behavioral case formulation using
functional analysis (Eells, 1997). All of these provide an empirically
supported framework leading to specific kinds of interventions.
Note that disciplined inquiry is a pluralistic model insofar as
it can accommodate multiple, empirically supported approaches.
In other words, the evidence-based clinician is not constrained by
empirical findings to one practice orientation only.
A formulation typically leads to a treatment plan for action.
Here, too, research findings are relevant for both selecting a
treatment and conducting it. Such research can include the ESTs.
Is there an EST that is targeted to the client’s problem, and can it
be administered within the theoretical approach guiding the practitioner?
This is much more likely to be the case for the therapist
who is cognitively or behaviorally oriented. However, as noted
earlier, there are other kinds of empirical findings to draw on
besides ESTs. It behooves practitioners of all theoretical stripes to
know what client and therapist factors, for example, have been
shown to correlate with therapy outcomes because these are often
more useful than diagnosis (Clarkin & Levy, 2004). It is true that
the link between cause and effect is not as strong in correlational
findings as it is in RCTs, because the latter include a comparison
or control group. Correlational data, however, do have evidentiary
standing by virtue of their pointing to an association between two
variables.
For example, there is strong empirical evidence that the therapeutic
alliance has a significant, reliable, albeit modest, relationship
to outcome (Martin, Garske, & Davis, 2000). In a review of
therapist characteristics and techniques that enhance the therapeutic
alliance, Ackerman and Hilsenroth (2003) found the following
personal attributes of the therapist to be important: being flexible,
honest, respectful, trustworthy, confident, warm, interested, and
Figure 1. Professional activity as disciplined inquiry. From “Connection and Disconnection of Research and
Practice in the Education of Professional Psychologists,” by D. R. Peterson, 1991, American Psychologist, 46,
p. 426. Copyright 1991 by the American Psychological Association. Adapted with permission. (See also
Fishman, 2001.)
583SPECIAL SECTION: EVIDENCE-BASED PRACTICE
open. Apropos of techniques, the following kinds of intervention
contributed positively to the alliance: exploration, reflection, noting
past therapy success, accurate interpretation, facilitating the
expression of affect, and attending to the patient’s experience. The
findings were not specific to only one kind of therapy, although
there were few studies that included CBT. Knowing these findings
should help therapists maximize their chances of preventing dropouts
and enhancing outcomes.
Finally, a monitoring evaluation is conducted that, in practice, is
most typically based on a discussion between client and practitioner
of what has been accomplished. Testing may be used as part of
a more formal evaluation. If either client or practitioner considers
the outcome insufficient and expects further efforts to lead to
greater improvement, the case may be reformulated and new
interventions attempted either by the same or another practitioner.
If both the client and the practitioner consider the outcome satisfactory,
the client’s needs are met and the therapist proceeds to
termination and a concluding evaluation. The case then becomes
part of the psychologist’s accumulated repertoire of clinical experience,
which influences how future cases are treated. Results that
differ from expectations may require a change or accommodation
of the guiding conception.
The Local Clinical Scientist
Parallel to the disciplined inquiry model, Stricker and Trierweiler
(1995) proposed the term local clinical scientist to describe
the mode of practice of a scientifically oriented psychologist. A
local clinical scientist is one who “brings the attitudes and knowledge
base of the scientist to bear on the problems that must be
addressed by the clinician in the consulting room” (p. 995). In
supporting the synergy between science and practice, they cited
Flexner’s (1925) comments on medical education:
At bottom the intellectual attitude of the two are—or should be—
identical: Neither investigator nor practitioner should be blinded by
prejudices or jump to conclusions; both should observe, reflect, conclude,
try, and, watching results, continuously reapply the same method
until the problem in hand has been solved or abandoned. (p. 4)
They stressed that it is just as unscientific to apply presumably
generalizable empirical findings (read, ESTs) to a local situation
where they do not quite fit (e.g., Mrs. T) as to ignore them when
they do. Understanding a local situation is at least as important as
knowing something about clients or techniques in general.
Along these lines, note the similarity of the following two
statements from leading educators of applied psychologists who
differ in theoretical orientation:
Local clinical scientists amass whatever [scientific] data are relevant,
combine these with the observations of the immediate setting and with
experience gathered from years of local practice and put it all together
in the service of providing assistance to those in need. They become
Sherlock Holmes of the consulting room: learned and astute observers,
consummate logicians, and effective agents in the local situation.
(Stricker & Trierweiler, 1995, p. 998)
Nevertheless, hope for distinctive, closely specifiable, demonstrably
effective, dependably generalizable methods of treating all the conditions
that clinicians encounter seem out of the question. Variations
among disorders are too great, variations in treatment are too often
required, and the responsibilities of practice require that the unique
specifics as well as the common features of every case be thoroughly
examined, fully described, and managed to the greatest possible
benefit of each client. (Peterson, 2004, p. 204)
Of course, both models need to be subjected to their own disciplined
inquiry and empirical test.
If, in fact, applied psychologists need to know the specifics
about individual cases and not only generalized group findings,
what kinds of methods and sources of data might supplement and
even improve on evidence from traditional group designs, process–
outcome correlations, and Client ϫ Technique interactions? One
answer is intensive case studies that strive to preserve a scientific
attitude even if the traditional level of control and ability to
generalize is much less than in RCTs. The point made by advocates
of these methods, however, is that they are better suited to the
exigencies of practice than are the RCTs, and that through the
accumulation of such cases lawful trends can be discerned. This
will also be a vehicle to study the effectiveness of the disciplined
inquiry and local clinical scientist models.
Revisiting Mrs. T
Before I describe some new, scientifically based approaches to
the case study, let us return to the case of Mrs. T. Is there an EST
that is suitable for her? Not exactly. The PTSD treatments that are
considered “probably efficacious” typically pertain to combat veterans
or to the aftermath of sexual assault. In addition, many PTSD
studies apply only to treatment that takes place soon after the
trauma (Litz, Gray, Bryant, & Adler, 2002). Therefore, one difficulty
with ESTs, which tend to rely on DSM–IV diagnoses such as
PTSD, is that they may not fit the case at hand very well. Nevertheless,
the empirical psychological research on the treatment of
anxiety and specifically PTSD points to two important treatment
factors: anxiety management and exposure, whether real or imagined
(Keane & Barlow, 2002). I used these principles in helping
Mrs. T come to face and start to master the 9/11 trauma. Incidentally,
as Keane and Barlow pointed out, it was Pierre Janet and
Sigmund Freud who most influenced these CBT approaches to
PTSD, so that it was not difficult for me as an “assimilative”
psychodynamic therapist to adapt them to the therapy. (Assimilative
integration refers to the incorporation of techniques or perspectives
from one mode of treatment into one’s “home” therapy;
Messer, 2001b).
Although these procedures were of some help to Mrs. T, they
were only partially successful. A primary question that arose in my
mind was why her symptoms had persisted so long and with such
virulence. Was there something more to it than a natural, human
response to disaster? Why did she feel so very responsible for the
fate of her employees who were simply going about their business
in a usual way? After all, she did not deliberately send them to
their death and was only “responsible” to the extent that she had
hired them and made the case for keeping one of them on when the
boss wanted to let her go.
When I asked Mrs. T to tell me what came to mind about her
feeling responsible for the employees’ demise, she informed me
that she had long believed that she possessed magical powers, that
584 MESSER
she was, in her words, “a small deity.” I asked her for examples of
what she meant. In response, she told me how she is able to locate
objects even years after others have lost them simply by holding
something of theirs in her hand. As another example, on three
separate occasions she had heard a knock on the door although no
one was there and knew instantly who in the family had died.
It became clear to both of us that her sense of specialness (or
grandiosity) had played an important role in her slow recovery
from the trauma. If she could foresee the future, she posited, why
had she not prevented her associates from going into the World
Trade Center on that fateful day? When I pointed out to her that
her excessive guilt over their deaths was closely linked to her
belief in her special powers, she responded ruefully that if she were
more humble she might not be suffering so much. This dynamic
also helped her to understand that her sadness was not related only
to the 9/11 losses but also to the diminution of her secret sense of
being extremely powerful and special. At first, she experienced
this revelation as a considerable narcissistic blow (to use the
language of self psychology). With further exploration and reflection
about this unusual feature of her personality, her narcissism
diminished to some degree as she became more accepting of
herself as just an ordinary mortal. This also helped her to recognize
her own role in her strong reaction to 9/11. Although this insight
caused her grief, it also produced relief at its exposure in a safe
therapeutic setting.
In brief, other features of the therapy included my encouraging
expression of her mixture of feelings about her marital relationship
and her coming to the realization that she no longer wanted the
kind of high-powered job she once enjoyed. That job had meant
giving up what she now saw as precious time with her growing
children and having too little time for herself and her husband. (For
a fuller account of Mrs. T’s therapy in which I drew on the four
“visions of reality”—tragic, comic, romantic, and ironic—see
Messer, in press). The point to be made here is that an EST, while
of some help in this case, could not by itself cover other ground
that truly mattered to this woman. Hers was not a case of pure or
even typical PTSD, as is true of many cases that are dually or triply
diagnosed and multifaceted. In fact, the complexity of people’s
lives is not readily captured by diagnoses altogether. There are
many pieces to this case that, to be treated optimally, call for
knowledge of ESTs, conceptualization and recognition of narcissism,
a degree of assimilative integration, and at least some of the
relationship qualities listed above that contribute to a therapeutic
alliance.
What are the new methods available that are designed to capture
the formulation, process, and outcome of a therapy in a way that
preserves the important features that I just illustrated with the case
of Mrs. T? I discuss this in the next section.
Intensive Case Study: The Practitioner’s Friend
The Pragmatic Case Study Method
Fishman (1999, 2001; Fishman & Messer, 2004) has proposed
that the basic unit of knowledge in applied psychology—whose
ultimate purpose is to improve the condition of individuals,
groups, families, organizations, or communities—should be the
individual case. His pragmatic case study method (PCSM) refers
to systematic, largely qualitative case studies that are focused on
practical results. The case study has a venerable history in psychology,
especially in psychoanalysis, but has been widely criticized
for the many sources of bias that reduce its utility as a basis
for a cumulative psychological science (e.g., Messer & McCann,
in press; Spence, 1993). Fishman has argued that two features of
his method mitigate these concerns. The first is that in the PCSM
the case presentation is neither fragmentary, as in the use of case
vignettes, nor free flowing according to the preferences of the
author, but has a specified structure, one in fact that follows
Peterson’s disciplined inquiry model described earlier. This model
has the virtue of being developed on the basis of empirical studies
of how effective professionals in diverse fields actually operate
(e.g., Scho¨n, 1987). That is, the model possesses ecological validity.
In addition, the PCSM method has the potential to reduce
therapists’ reporting bias, such as their overemphasis on more
recent versus earlier information. It does so by encouraging systematic,
reflective processing of audiotaped material or extensive
progress notes and collecting quantitative feedback from client
questionnaires. This process would help alleviate cognitive bias,
which experimental studies have shown to affect clinical judgment
(Garb, 1998).
A pragmatic clinical case would be written up with the following
headings: Case Context and Method, The Client, Guiding
Conception With Research and Clinical Experience Support, Assessment
of the Client’s Presenting Problems and Goals, Formulation
and Treatment Plan, Course of Therapy, Therapy Monitoring
and Use of Feedback Information, and Concluding Evaluation
of the Therapy’s Process and Outcome. To situate the case normatively,
one may also use standardized, quantitative, client- or
therapist-completed measures. Fishman (1999, 2001) described
each of these sections in considerable detail, although the specifics
of the method are still evolving to accommodate diverse theoretical
approaches to case write-ups. For example, the Assessment
and Formulation sections describe the process of integrating contextual
information about the patient with the results of the assessment.
This would be carried out within the framework of the
guiding conception, leading to a formulation of the client’s problem
and an individualized treatment plan. One of the sources for
such case write-ups could be a database of the individual client
case studies that make up the samples in RCTs because the kind of
data collected lend themselves to this format (Fishman, 2001).
The common framework for case write-ups sets up the second
major feature of the PCSM that is designed to help create a
cumulative science of cases. Although one cannot generalize from
a single case study, the collection of many such cases allows for
inductive generalizations to other, similar settings. This can happen
by organizing case studies with similar presenting problems
and intervention approaches into searchable databases, akin to
legal databases used by lawyers and judges. As the number of
cases increases, so does the probability that selected cases in the
database will be contextually and pragmatically relevant to a new
target case. This approach requires the use of online capacities in
the form of a peer-reviewed, case-based journal, in which accumulating
case studies form a searchable database. My colleagues
(Dan Fishman, Ron Miller, Peter Nathan) and I have developed
such a journal. Titled Pragmatic Case Studies in Psychotherapy
and encompassing both case studies and articles on case study
585SPECIAL SECTION: EVIDENCE-BASED PRACTICE
method, it will be edited by Daniel Fishman and published jointly
by the Rutgers University Libraries and the Rutgers Graduate
School of Applied and Professional Psychology. One of the advantages
of such a journal is that it makes it much more possible
for practitioners to contribute to the literature than has been the
case to date. (The journal is located at the following Web site:
http://pcsp.libraries.rutgers.edu.) Another specific, related approach
that falls under the umbrella of PCSM follows.
Hermeneutic Single-Case Efficacy Design
This is an approach to assessing the outcome of psychotherapy
that goes beyond the general findings of RCTs (Elliott, 2001). It
tries to delve more thoroughly than do RCTs into the question of
which aspects of therapy a client found most helpful. The rationale
for the hermeneutic single-case efficacy design (HSCED) is that
traditional measures of therapy outcome are neither penetrating
enough nor specific enough to individual cases to yield a sufficiently
nuanced picture of what has changed and why. HSCED
attempts to use quantitative and qualitative information to create a
rich case record, which provides direct and indirect evidence for
the causal influence of therapy on client outcome. In other words,
it relies on thick description (Geertz, 1973) rather than controlled
research design, and interpretive rather than experimental procedures
(Elliott, 2001). The method is referred to as “hermeneutic”
because, in Elliott’s words, “it attempts to construct a plausible
understanding of the influence processes in complex, ambiguous
sets of information about a client’s therapy” (p. 317).
How does one make a reasonable case for claiming that a client
probably improved and that therapy was probably the cause?
Elliott (2001) stated two conditions: There must be one or more
pieces of positive evidence linking therapy to observed client
change, and there must be indications that plausible nontherapy
explanations are insufficient to explain the change. Typically the
method calls on client, therapist, and researcher input. Examples of
the positive evidence for change are the following: (a) The client
explicitly attributes change to therapy, (b) the client describes
helpful aspects of therapy linked to change, and (c) examination of
the weekly data reveals covariation between in-therapy processes
and week-to-week shifts in client problems. The combination of
weekly measures of “helpful aspects of therapy” and client difficulties
or goals provide the necessary information to make such
inferences.
There are also stringent criteria by means of which the researcher
attempts to refute these claims and to argue that therapy
outcome is due to nontherapy events, which gives this method a
scientific cast. The following are some examples: (a) The changes
are negative, irrelevant, or trivial; (b) the changes are due to
statistical artifacts or random error; (c) the client wants to make the
therapist feel good or tries to justify ending therapy, or (d) life
events outside therapy. After these data are collected, it is necessary
to interpret and weigh them as they may conflict with each
other. This requires an integration of the data and a determination
of where the weight of the evidence lies. One interesting addition
to this method that has been put into practice is having a panel of
affirmative and skeptic teams evaluate the evidence and present
the case for and against change. A research jury makes the final
determination (Bohart, 2000b). Elliott and his colleagues are currently
preparing an extended example of such an adjudicated
HSCED. (For another, related approach to single-case studies, see
Schneider’s, 2001, multiple-case depth research model.)
Implications for Practice
The culture war between proponents of ESTs and other forms of
treatment has been useful in bringing to the fore various arguments
and counterarguments about what the scientific practice of clinical
and counseling psychology can and should be. The issue need not
be framed so starkly as science versus humanism or the use of
evidence versus clinicians’ subjectivism and clinical judgment.
ESTs, for example, provide one kind of relevant data on which
clinicians can rely, but they are not and cannot be the sole source
of evidence-based practice. As I have tried to demonstrate, both
from the literature and in the case of Mrs. T, there are serious
limitations to ESTs at the same time as there are other sources of
scientific and not-so-scientific evidence available to clinicians, all
with their own strengths and limitations that are just as crucial to
daily practice. They can, at the very least, complement ESTs to
provide sound evidence-based treatments. ESRs are one such
source of evidence, as are the effects of the interaction of clients
and techniques and process–outcome correlational data more generally,
which it behooves clinicians to know.
Regarding future development, systematic, in-depth collections
of case studies, which can include both qualitative and quantitative
information, hold considerable promise. One need not be a researcher
to contribute to this literature (see http://pcsp.libraries
.rutgers.edu). All such sources of data can be brought to bear on
practice within the framework of the disciplined inquiry and local
clinical scientist concepts. These models are the most inclusive and
pluralistic ways of conceiving of evidence-based practice available
and can encompass the results of many extant psychotherapy
research strategies. Examples include research on the efficacy of
therapy (from RCTs in the lab) and its effectiveness (outcomes in
the field), process research (especially process–outcome relations
and client by treatment interactions), and pragmatic, hermeneutic,
and in-depth case studies. As practitioners, we cannot manage
without nomothetic and idiographic data, findings based on quantitative
and qualitative method, and a mixture of scientific and
humanistic outlooks, which are psychology’s dual heritage.
References
Ackerman, S. J., & Hilsenroth, M. J. (2003). A review of therapist characteristics
and techniques positively impacting the therapeutic alliance.
Clinical Psychology Review, 23, 1–33.
Allport, G. W. (1937). Personality: A psychological interpretation. New
York: Holt.
American Psychiatric Association. (1994). Diagnostic and statistical manual
of mental disorders (4th ed.). Washington, DC: Author.
Anderson E. M., & Lambert, M. J. (2001). A survival analysis of clinically
significant change in outpatient psychotherapy. Journal of Clinical Psychology,
57, 875–888.
Bohart, A. C. (2000a). Paradigm clash: Empirically supported treatments
versus empirically supported psychotherapy practice. Psychotherapy
Research, 10, 488–493.
Bohart, A. C. (2000b, June). A qualitative adjudicational model for assessing
psychotherapy outcome. Paper presented at the meeting of the
Society for Psychotherapy Research, Chicago.
586 MESSER
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported
therapies. Journal of Consulting and Clinical Psychology, 66, 7–18.
Chambless, D. L., Sanderson, W. C., Shoham, V., Johnson, S., Pope, K.,
Crits-Christoph, P., et al. (1996). An update on empirically validated
therapies. The Clinical Psychologist, 49, 5–18.
Clarkin. J. F., & Levy, K. N. (2004). The influence of client variables on
psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook
of psychotherapy and behavior change (pp. 194–226). New York:
Wiley.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (2000). Handbook of qualitative
research (2nd ed.). Thousand Oaks, CA: Sage.
Eells, T. (Ed.). (1997). Handbook of psychotherapy case formulation. New
York: Guilford Press.
Elliott, R. (2001). Hermeneutic single-case efficacy design: An overview.
In K. J. Schneider, J. F. T. Bugental, & J. F. Pierson (Eds.), The
handbook of humanistic psychology (pp. 315–324). Thousand Oaks, CA:
Sage.
Fishman, D. B. (1999). The case for pragmatic psychology. New York:
New York University Press.
Fishman, D. B. (2001). From single case to database: A new method for
enhancing psychotherapy, forensic, and other psychological practice.
Applied and Preventive Psychology, 10, 275–304.
Fishman, D. B., & Messer, S. B. (2004). Case-based studies as a source of
unity in applied psychology. In R. J. Sternberg (Ed.), The unification of
psychology: Prospect or pipedream? (pp. 37–59). Washington, DC:
American Psychological Association.
Flexner, A. (1925). Medical education: A comparative study. New York:
Macmillan.
Frank, J. D., & Frank, J. B. (1991). Persuasion and healing: A comparative
study of psychotherapy (3rd ed.). Baltimore, MD: Johns Hopkins University
Press.
Garb, H. N. (1998). Studying the clinician: Judgment research and psychological
assessment. Washington, DC: American Psychological As-
sociation.
Geertz, C. (1973). The interpretation of cultures. New York: Basic Books.
Gold. J. (1995). The place of process-oriented psychotherapies in an
outcome-oriented psychology and society. Applied and Preventive Psychology,
4, 61–74.
Goodheart, C. D. (2004). Evidence-based practice and the endeavor of
psychotherapy. The Independent Practitioner, 24, 6–10.
Grissom, R. J. (1996). The magic number .7 ϩ –.2: Meta-meta-analysis of
the probability of superior outcome in comparisons involving therapy,
placebo, and control. Journal of Consulting and Clinical Psychology, 64,
973–982.
Keane, T. M., & Barlow, D. H. (2002). Posttraumatic stress disorder. In
D. H. Barlow (Ed.), Anxiety and its disorders (2nd ed., pp. 418–453).
New York: Guilford Press.
Kimble, G. A. (1984). Psychology’s two cultures. American Psychologist,
39, 833–839.
Levant, R. F. (2004). The empirically-validated treatments movement: A
practitioner/educator perspective. Clinical Psychology: Science and
Practice, 11, 219–224.
Litz, B. T., Gray, M. J., Bryant, R. A., & Adler, A. B. (2002). Early
intervention for trauma: Current status and future directions. Clinical
Psychology: Science and Practice, 9, 112–134.
Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D.,
Johnson, S., et al. (1999). The researcher’s own therapy allegiances: A
“wild card” in comparisons of treatment efficacy. Clinical Psychology:
Science and Practice, 6, 95–106.
Luborsky, L., Rosenthal, R., Diguer, L., Andrusyna, T. P., Berman, J. S.,
Levitt, J. T., et al. (2002). The Dodo Bird is alive and well—mostly.
Clinical Psychology: Science and Practice, 9, 2–12.
Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the
therapeutic alliance with outcome and other variables: A meta-analytic
review. Journal of Consulting and Clinical Psychology, 68, 438–450.
McWilliams, N. (1999). Psychoanalytic case formulation. New York:
Guilford Press.
Messer, S. B. (2001a). Empirically supported treatments: What’s a nonbehaviorist
to do? In B. D. Slife, R. N. Williams, & S. H. Barlow (Eds.),
Critical issues in psychotherapy: Translating ideas into practice (pp.
3–19). Thousand Oaks: CA: Sage.
Messer, S. B. (2001b). Introduction to the special issue on assimilative
integration. Journal of Psychotherapy Integration, 11, 1–4.
Messer, S. B. (2002). Empirically supported treatments: Cautionary notes.
Medscape General Medicine. Retrieved from http://www.medscape
.com/viewarticle/445082
Messer, S. B. (in press). Psychotherapy integration based on contrasting
visions of reality. In G. Stricker & J. Gold (Eds.), Casebook of psychotherapy
integration. Washington, DC: American Psychological Associ-
ation.
Messer, S. B., & McCann, L. (in press). Research perspectives on the case
study: Single-case method. In J. S. Auerbach, K. N. Levy, & C. E.
Schaffer (Eds.), Relatedness, self-definition, and mental representation:
Essays in honor of Sidney J. Blatt. London: Brunner-Routledge.
Messer, S. B., Sass, L. A., & Woolfolk, R. L. (1988). Hermeneutics and
psychological theory: Interpretive perspectives on personality, psychotherapy,
and psychopathology. New Brunswick, NJ: Rutgers University
Press.
Messer, S. B., & Wampold, B. E. (2002). Let’s face facts: Common factors
are more potent than specific therapy ingredients. Clinical Psychology:
Science and Practice, 9, 21–25.
Nathan, P. E. (2001). Deny nothing, doubt everything: A comment on
Westen and Morrison. Journal of Consulting and Clinical Psychology,
69, 900–903.
Norcross, J. C. (2002). Psychotherapy relationships that work. New York:
Oxford University Press.
Peterson, D. R. (1991). Connection and disconnection of research and
practice in the education of professional psychologists. American Psychologist,
46, 422–429.
Peterson, D. R. (2004). Science, scientism, and professional responsibility.
Clinical Psychology: Science and Practice, 11, 196–210.
Sanderson, W. C., & Rego, S. A. (2000). Movement toward evidencebased
psychotherapies. Trends in Evidence-Based Neuropsychiatry, 2,
50–55.
Schneider, K. J. (2001). Multiple-case depth research. In K. J. Schneider,
J. F. T. Bugental, & J. F. Pierson (Eds.), The handbook of humanistic
psychology (pp. 305–314). Thousand Oaks, CA: Sage.
Scho¨n, D. A. (1987). The reflective practitioner. San Francisco: Jossey-
Bass.
Shadish, W. R., Matt, G. E., Navaro, A. M., & Phillips, G. (2000). The
effects of psychological therapies under clinically representative conditions:
A meta-analysis. Psychological Bulletin, 126, 512–529.
Slife, B. D. (2004). Theoretical challenges to therapy practice and research:
The constraint of naturalism. In M. J. Lambert (Ed.), Bergin and Garfield’s
handbook of psychotherapy and behavior change (pp. 44–83).
New York: Wiley.
Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy
outcome studies. American Psychologist, 32, 752–760.
Spence, D. P. (1993). Traditional case studies and prescriptions for improving
them. In N. E. Miller, L. Luborsky, J. P. Barber, & J. P.
Docherty (Eds.), Psychodynamic treatment research: A handbook for
clinical practice (pp. 37–52). New York: Basic Books.
Stirman, S. W., DeRubeis, R. J., Brody, P. E., & Crits-Christoph, P. (2003).
Are samples in randomized controlled trials of psychotherapy representative
of community outpatients? A new methodology and initial findings.
Journal of Consulting and Clinical Psychology, 71, 963–972.
587SPECIAL SECTION: EVIDENCE-BASED PRACTICE
Stricker, G., & Trierweiler, S. J. (1995). The local clinical scientist: A
bridge between science and practice. American Psychologist, 50, 995–
1002.
Task Force on Promotion and Dissemination of Psychological Procedures.
(1995). Training in and dissemination of empirically-validated psychological
treatments. The Clinical Psychologist, 48, 3–23.
Wampold, B. E. (2001). The great psychotherapy debate. Hillsdale, NJ:
Erlbaum.
Wampold, B. E., Minami, T., Baskin, T. W., & Tierney, S. C. (2002). A
meta-(re)analysis of the effects of cognitive therapy versus “other therapies”
for depression. Journal of Affective Disorders, 68, 159–165
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., &
Ahn, H. (1997). A meta-analysis of outcome studies comparing bona
fide psychotherapies: Empirically “all must have prizes.” Psychological
Bulletin, 122, 203–215.
Westen, D., & Morrison, K. (2001). A multidimensional meta-analysis of
treatments for depression, panic, and generalized anxiety disorder: An
empirical examination of the status of empirically supported treatments.
Journal of Consulting and Clinical Psychology, 69, 875–899.
Woolfolk, R. L. (1998). The cure of souls: Science, values and psychotherapy.
San Francisco: Jossey-Bass.
Received February 10, 2004
Revision received July 1, 2004
Accepted July 27, 2004 Ⅲ
New Editors Appointed, 2006–2011
The Publications and Communications Board of the American Psychological Association announces
the appointment of seven new editors for 6-year terms beginning in 2006. As of January
1, 2005, manuscripts should be directed as follows:
• Experimental and Clinical Psychopharmacology (www.apa.org/journals/pha.html), Nancy K.
Mello, PhD, McLean Hospital, Massachusetts General Hospital, Harvard Medical School, 115
Mill Street, Belmont, MA 02478-9106.
• Journal of Abnormal Psychology (www.apa.org/journals/abn.html), David Watson, PhD, Department
of Psychology, University of Iowa, Iowa City, IA 52242-1407.
• Journal of Comparative Psychology (www.apa.org/journals/com.html), Gordon M. Burghardt,
PhD, Department of Psychology or Department of Ecology & Evolutionary Biology, University
of Tennessee, Knoxville, TN 37996.
• Journal of Counseling Psychology (www.apa.org/journals/cou.html), Brent S. Mallinckrodt,
PhD, Department of Educational, School, and Counseling Psychology, 16 Hill Hall, University
of Missouri, Columbia, MO 65211.
• Journal of Experimental Psychology: Human Perception and Performance (www.apa.org/
journals/xhp.html), Glyn W. Humphreys, PhD, Behavioural Brain Sciences Centre, School of
Psychology, University of Birmingham, Edgbaston, Birmingham B15 2TT, United Kingdom.
• Journal of Personality and Social Psychology: Attitudes and Social Cognition section
(www.apa.org/journals/psp.html), Charles M. Judd, PhD, Department of Psychology, University
of Colorado, Boulder, CO 80309-0345.
• Rehabilitation Psychology (www.apa.org/journals/rep.html), Timothy R. Elliott, PhD, Department
of Psychology, 415 Campbell Hall, 1300 University Boulevard, University of Alabama,
Birmingham, AL 35294-1170.
Electronic submission: As of January 1, 2005, authors are expected to submit manuscripts
electronically through the journal’s Manuscript Submission Portal (see the Web site listed above
with each journal title).
Manuscript submission patterns make the precise date of completion of the 2005 volumes uncertain.
Current editors, Warren K. Bickel, PhD, Timothy B. Baker, PhD, Meredith J. West, PhD, Jo-Ida C.
Hansen, PhD, David A. Rosenbaum, PhD, Patricia G. Devine, PhD, and Bruce Caplan, PhD,
respectively, will receive and consider manuscripts through December 31, 2004. Should 2005
volumes be completed before that date, manuscripts will be redirected to the new editors for
consideration in 2006 volumes.
588 MESSER