Dodo bird verdict

The Dodo bird verdict (or Dodo bird conjecture) is a controversial topic in psychotherapy,[1] referring to the claim that all empirically validated psychotherapies, regardless of their specific components, produce equivalent outcomes. It is named after The Dodo character of Alice in Wonderland. The conjecture was introduced by Saul Rosenzweig in 1936, drawing on imagery from Lewis Carroll's novel Alice's Adventures in Wonderland, but only came into prominence with the emergence of new research evidence in the 1970s.[2]

The importance of the continuing debate surrounding the Dodo bird verdict stems from its implications for professionals involved in the field of psychotherapy and the psychotherapies made available to clients.

History

Carroll, Lewis, "Chapter 3: The Caucus Race and a Long Tale", Alice's Adventures in Wonderland, Everybody has won, and all must have prizes.

The Dodo bird verdict terminology was coined by Saul Rosenzweig in 1936 to illustrate the notion that all therapies are equally effective.[3][4][5][6] Rosenzweig borrowed the phrase from Lewis Carroll's 1865 book, Alice in Wonderland, wherein a number of characters become wet and, in order to dry themselves, the Dodo Bird decided to issue a competition: everyone was to run around the lake until they were dry. Nobody cared to measure how far each person had run, nor how long. When they asked the Dodo who had won, he thought long and hard and then said, "Everybody has won and all must have prizes." In the case of psychotherapies, Rosenzweig argued that common factors were more important than specific technical differences, so that (on the Dodo bird conjecture) all therapies are winners; they all produce equally effective outcomes.[2]

The Dodo bird debate only took flight in 1975 when Lester Luborsky, Barton Singer and Lise Luborsky reported the results of one of the first comparative studies demonstrating few significant differences in the outcomes among different psychotherapies.[5] This study spurred a plethora of new studies in both opposition to and support of the Dodo bird verdict.[7]

The Dodo bird debate, in brief, is focused on whether or not the specific components of different treatments lead some treatments to outperform other treatments for specific disorders. Supporters of the Dodo bird verdict contend that all psychotherapies are equivalent because of "common factors" that are shared in all treatments (i.e., having a relationship with a therapist who is warm, respectful, and has high expectations for client success).[8][9] In contrast, critics of the Dodo bird verdict would argue that the specific techniques used in different therapies are important, and all therapies do not produce equivalent outcomes for specific disorders.

Support

Common factors theory states that if certain therapies in psychology are equally effective, it is because of the common factors they share. The most important causal agents in treatment are the common factors; the specific techniques that are unique to treatment strategies have only minor importance.[10][11][12][13][14]

There is research to support common factors theory. One common factor is the client–therapist interaction, also known as the therapeutic alliance. A 1992 paper by Lambert showed that nearly 40 percent of the improvement in psychotherapy is from these client–therapist variables.[15] Other researchers have further analyzed the importance of client–therapist variables in treatment. They found that improvement in the patient was due to extra-therapeutic factors, for example patients' thought processes. Data shows that patients with more positive attitudes will have a better chance of experiencing clinical improvement, regardless of the therapist's actions.[4][5][16] Furthermore, in a meta-analysis of many studies of psychotherapy, Wampold et al. 2002, found that 7% of the variability in treatment outcome was due to the therapeutic alliance whereas 1% of the variability was due to a specific treatment.[17][18][19][20] The therapist's attitude is also a very important causal agent in positive patient change. Najavits and Strupp (1994) demonstrated that a positive, warm, caring, and genuine therapist generated statistically significant differences in patient outcome.[21] Wampold et al. 2002, also found that nearly 70% of the variability in treatment outcome was due to the therapist's attitude toward the efficacy of the treatment.[17] Specific components of therapy are concluded to be relatively frivolous when compared with the more profound and directly patient affecting common factors.

Researchers have studied common factors in detail. Grencavage and Norcross (1990) identified 35 common factors in published sources.[22] The identified common factors were categorized into five main groups: client characteristics, therapist qualities, change processes, treatment structures and relationship elements. Examples of some of the common factors included within these broad categories are persuasion, a healing setting, engagement, the use of rituals and techniques, suggestion, and emotional learning. Tracey et al. 2003, examined deeper relationships among the categories and common factors. They concluded that there are two dimensions of therapy: feeling and thinking. Within each of the two dimensions are three clusters: bond, information, and role.[23]

Data provide evidence for the Dodo bird verdict.[24] Generally speaking, common factors are responsible for patient improvement instead of any component of a specific treatment.[9][24] Researchers such as Wampold and Luborsky continue to support this claim in recent articles published in the Behavioral Therapist. Wampold et al. 2010 refutes claims made by Siev et al. 2009 that Wampold 2009 made errors in research. Wampold et al. 2009, suggests that people need to "accept the importance of the alliance and therapists and remain committed to developing and improving treatments."[24] Wampold continues by saying that techniques could be beneficial in psychotherapy because they are the easiest variables to manipulate. These variables can act to change alliance and other common factors. Common factors can then be closely monitored and carefully distributed to patients via the therapist.[24]

Some researchers have pointed out that there are many reasons to study common factors among different psychotherapies, and some of those reasons may have nothing to do with the Dodo bird verdict.[25] Regardless of whether or not certain psychotherapies are roughly equally effective, studying the commonalities among treatments can lead to a better understanding of why the treatments are effective.[25]

Opposition

In opposition to the Dodo bird verdict, there are a growing number of studies demonstrating that some treatments produce better outcomes for particular disorders when compared to other treatments.[26][27] Here, in contrast to the common factor theory, specific components of the therapy have shown predictive power.[28][29][30] The most compelling evidence against the Dodo bird verdict is illustrated by the research done on anxiety disorders. Many studies have found specific treatment modalities to be beneficial when treating anxiety disorders, specifically cognitive behavioral therapy (CBT). Albeit, other studies do not show evidence for cognitive-behavioural and focus on different topics. [31][32] CBT uses techniques from both cognitive therapy and behavioral therapy to modify maladaptive thoughts and behaviors.[33][34][35] One challenge to the opposition, however, is that some studies were conducted with waiting lists or against medication and criticism can arise as the therapeutic relationship is known to be a factor which influences outcomes. This means the control group may not experience the human condition, whereby the experimental group have an advantage when analysing differences. In addition, comparator trials with other modalities have been known to have unfair training standards between therapies, with some counsellors being given brief workshops over several days to evaluate the efficacy of an approach during trials. Humanistic therapies are also notably difficult to examine due to having less measurable factors that do not fare well under randomised controlled trials. Many of the more extensive studies using different research methodologies with larger sample sizes are overlooked for the perceived gold standard in research. Cognitive-behavioural therapy has indeed been the most examined counselling method; however, to visualise this, you may wish to see them as the big fish in a small pond. Numerous meta-analyses have shown that CBT yields significantly superior results in the treatment of psychological disorders, most notably, anxiety disorders. However, CBT also plays a positive role in treating depression, eating disorders, substance abuse disorders, and obsessive-compulsive disorder.[36][37][38][39][40][41][42] In meta-analytic reviews, there is generally a larger effect size when CBT is used to treat anxiety disorders. Recent studies show that when treating generalized anxiety disorder, CBT generated an effect size of 0.51, which indicates a medium effect. That is a much larger effect compared to supportive therapy and other treatments.[37][38] Similarly, when treating social anxiety disorder, CBT produced an effect size of 0.62, which again supports the evidence that CBT does in fact yield significantly better results than other therapies.[37][43] Those supporting the Dodo bird verdict often use meta-analyses to compare multiple treatments in order to illustrate that these treatments have no really significant differences. Some critics claim that if comparisons are made between treatments that already have empirical support (empirically supported treatments, or ESTs) then this will tend to reduce estimates of variability. Nonetheless, Wampold's criteria for including treatments in meta analyses of treatment variability do not require prior empirical support; rather, they must be "intended to be therapeutic".

Research supporting differences among different treatments for specific disorders is now reflected in the latest EST guidelines.[30] ESTs are developed from multiple control trial tests to evaluate which therapies yield the best results for specific disorders. These disorders include, but are not limited to, major depressive disorder, anxiety disorder, panic disorder, and OCD.[30][44] According to the Ethics Code of the American Psychological Association (APA), psychologists and therapists have an obligation to avoid harming their clients in any way.[44][45] ESTs are a major component in this movement. By using specific therapies clinically proven to improve certain disorders, therapists are avoiding further damage to their patients. However, supporting the idea of ESTs inevitably implies that some therapies are in fact more efficacious than others for particular disorders.[46]

There is also research suggesting that some treatments might be harmful. Indeed, "if psychotherapy is powerful enough to do good, it may be powerful enough to do harm".[47] Some psychotherapies can be labeled as unhelpful, meaning they give no assistance, while others fall under the category of harmful, meaning they are actually damaging or dangerous to the patient.[47] When identifying a "harmful" treatment it is important to note the distinction between "harm that can be caused by a disorder and harm that can be caused by the application of a treatment".[47] The negative outcomes of some psychotherapies are becoming an alarming reality in the field of clinical psychology.[48][49] Studies have shown that individuals exhibited negative responses to treatment in some substance abuse work,[44] some types of grief therapy,[47][50] and certain therapeutic techniques with trauma and PTSD patients.[47][51][52] While those studies that support the Dodo bird verdict focus on the importance of building a client–therapist relationship, some studies have "identified a number of other relationship factors that may interfere with or negatively impact therapeutic change".[48][53][54][55] The emerging evidence to the effect that there are possibly harmful psychotherapies is not only contradictory to the "all therapies are equal" stance of the Dodo bird verdict, but may also point out problems implicating the APA's Code of Ethics. Many meta-analyses show that there are treatments that do yield more positive outcomes for specific disorders. However, proof that some treatments actually harm patients instead of helping them further erodes support for the Dodo bird verdict.

Issues in testing

A considerable amount of the controversy about the Dodo bird verdict relates to the meta-analytic methods used in research.[56] These generate a lack of clear psychotherapeutic evidence for the support of either side in the debate.[56] Meta-analytic studies have compared the effect sizes of different treatments, but have not been reliable in finding a consistent effect size.[7] This could be because of several confounding variables. For example, many researchers are said to "have an agenda" when conducting meta-analyses, cherry-picking experiments they want to use in their study in order to produce the results they want. This pre-determined skewing results in unfair representations of the research.[57] "Agenda"-based meta-analyses are confounded with the researcher's political, social, and economical opinions. Since psychologists are given the power to choose which studies are used in a meta-analysis, personal biases are involved, and the meta-analysis will produce biased results if the researcher is not careful in controlling for his or her own opinions.[58]

Researchers opposing the Dodo bird verdict have found Dodo bird supporters' meta-analyses to be "agenda"-based and highly subjective.[57] Arguments have proposed that meta-analyses could possibly produce misleading results because of the type of studies combined in the comparison.[59][60][61] In Paul Crits-Christoph's review of Wampold et al.'s (1997) comparative study, a work that supported the Dodo bird verdict, he concluded that out of the 114 articles used in the study, 79 of them involve similar comparative tests.[62] Some meta-analyses constructed are not sensitive to the subtle distinctions between treatment effects, especially among comparative studies of highly similar treatments.[59][63][64]

Dodo bird supporting researchers have found anti-Dodo bird supporters' research to also be "agenda"-based. For example, Wampold (2009) found Siev et al.'s (2007) study whose research for significance of CBT (Cognitive Behavior Therapy) versus RT (Rational Therapy) was resting on one experiment with an uncharacteristically large effect size (1.02) by Clark et al. (1994).[24] Wampold found this effect size to be invalid because of the internal biases of the study. When this flawed experiment was removed from the analysis, the effect size was not statistically significant for the use of CBT over RT in panic disorder therapy. Against this research, in support of the anti-Dodo bird verdict, Chambless (2002) stated that "errors in data analysis, exclusion of research on many types of clients, faulty generalization to comparisons between therapies that have never been made, and erroneous sorts of treatments for all sorts of problems can be assumed to represent the difference between any two types of treatment for a given problem."[65]

Clearly, if meta-analyses are to be taken seriously, they need to be conducted carefully and objectively. In support of the anti-Dodo bird side, Hunsley (2007) says that when "measurement quality is controlled for and when treatments are appropriately categorized, there is consistent evidence in both treatment outcome and comparative treatment research that cognitive and behavioral treatments are superior to other treatments for a wide range of conditions, in both adult and child samples."[56] This suggests that if and when variables are controlled, there is appreciable evidence for the superiority of cognitive and behavioral treatments.

Importance

The outcome of the Dodo bird debate has extremely important implications for the future of clinical psychology. For one, policymakers have to know how effective each existing psychotherapy is in order to be able to decide which therapies should be supported. This controversy may also lead governmental officials to cut funding for psychological treatments and other public health measures.[66][67][68]

Perhaps the greatest illustration of the current state of the Dodo bird verdict is seen in the meta-analyses of Wampold and Barlow and the responses to it. In these meta-analyses, researchers on both sides point out the weaknesses and inconsistencies in their opponents' positions. Although both sides are trying to improve psychology in their respective ways, the disagreement about and lack of consistent evidence for the Dodo bird verdict may in fact be the cause of increased public doubt about the field. The conclusion of the debate could nationally dictate which therapists and which procedures will remain financially supported. For example, if the Dodo bird verdict is thought to be true regarding different psychotherapies, then many clinicians would feel free to use any therapy they see fit to employ. However, if the Dodo bird verdict is proven to be false, then clinicians would likely have to use empirically supported therapies when treating their clients.

Fuelling the debate have been alternative visions or paradigms of mental illness. Those believing in a medical model of mental illness and cure see the Dodo bird verdict as necessarily untrue – even absurd – whatever the evidence supporting it.[10] Those who see therapy as context-based – as relying on a shared frame of reference or context between client and therapist for optimum results[2] – will almost equally automatically welcome the Dodo bird verdict.[10]

See also

• Anti-psychiatry
• Carl Rogers
• Evidence-based medicine
• H. J. Eysenck
• Peer review
• Placebo
• Self-help groups for mental health
• Working alliance

References

1. For an overview of the controversy, see: "Special section on the Dodo Bird Verdict". Clinical Psychology: Science and Practice. 1 (9): 2–34. March 2002. doi:10.1111/cpsp.2002.9.issue-1. And: Budd, Rick; Hughes, Ian (December 2009). "The Dodo Bird Verdict—controversial, inevitable and important: a commentary on 30 years of meta-analyses". Clinical Psychology & Psychotherapy. 16 (6): 510–522. doi:10.1002/cpp.648. PMID 19728292.
2. Bentall, Richard P (2009), Doctoring the mind: is our current treatment of mental illness really any good?, New York: New York University Press, pp. 248–249.
3. Rosenzweig, Saul (1936). "Some implicit common factors in diverse methods of psychotherapy". American Journal of Orthopsychiatry. 6 (3): 412–15. doi:10.1111/j.1939-0025.1936.tb05248.x.
4. Luborsky, L (1999). "The researcher's own therapeutic allegiances: a 'wild card' in comparisons of treatment efficacy". Clinical Psychology: Science and Practice. 6: 49–62. doi:10.1093/clipsy/6.1.95.
5. Luborsky, L; Singer, B; Luborsky, L (1975), "Is it true that 'everyone has won and all must have prizes?'", Archives of General Psychiatry, 32 (8): 995–1008, doi:10.1001/archpsyc.1975.01760260059004, PMID 239666.
6. Luborsky, L; Rosenthal, R; Diguer, L; Andrusyna, TP; Berman, JS; Levitt, JT; Seligman, DA; Krause, ED (2002), "The dodo bird verdict is alive and well–mostly", Clinical Psychology: Science and Practice, 9: 2–12, CiteSeerX 10.1.1.596.7529, doi:10.1093/clipsy.9.1.2.
7. Tarrier, N (2002), "Yes, cognitive behaviour therapy may be all you need", British Medical Journal, 324: 291–92.
8. Frank, JD (1961), Persuasion and healing: A comparative study of psychotherapy.
9. Wampold, Bruce E (2007), "Psychotherapy: the humanistic (and effective) treatment", American Psychologist, 62 (8): 857–73, doi:10.1037/0003-066x.62.8.857, PMID 18020771.
10. Wampold, Bruce E (2001), The great psychotherapy debate: models, methods, and findings, Mahwah, NJ: L. Erlbaum, pp. 36, 96.
11. Horvath, AO; Bedi, RP (2002), "The alliance", in Norcross, JC (ed.), Psychotherapy relationships that work: therapist contributions and responsiveness to patients, New York: Oxford University Press, pp. 37–70.
12. Martin, DJ; Garske, JP; Davis, MK (2000), "Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review", Journal of Consulting and Clinical Psychology, 68 (3): 438–50, doi:10.1037/0022-006x.68.3.438, PMID 10883561.
13. Baldwin, SA; Wampold, Bruce E; Imel, ZE (2007), "Untangling the alliance–outcome correlation: exploring the relative importance of therapist and patient variability in the alliance", Journal of Consulting and Clinical Psychology, 75 (6): 842–52, doi:10.1037/0022-006x.75.6.842, PMID 18085902, S2CID 14232545.
14. Klein, DN; Schwartz, JE; Santiago, NJ; Vivian, D; Vocisano, C; Castonguay, LG; et al. (2003), "Therapeutic alliance in depression treatment: Controlling for prior change and patient characteristics", Journal of Consulting and Clinical Psychology, 71 (6): 997–1006, doi:10.1037/0022-006x.71.6.997, PMID 14622075.
15. Lambert, M (1992), "Implications for outcome research for psychotherapy integration", in Norcross, JC; Goldstein, MR (eds.), Handbook of psychotherapy integration, New York: Basic Books, pp. 94–129.
16. Wampold, Bruce E (1997), "Methodological problems in identifying efficacious psychotherapies", Psychotherapy Research, 7: 21–43, doi:10.1080/10503309712331331853.
17. Wampold, Bruce E; Minami, T; Baskin, TW; Tierney, SC (2002), "A meta-(re)analysis of the effects of cognitive therapy versus 'other therapies' for depression", Journal of Affective Disorders, 68 (2–3): 159–65, doi:10.1016/s0165-0327(00)00287-1, PMID 12063144.
18. Wampold, Bruce E; Mondin, GW; Moody, M; Ahn, H (1997), "The flat earth as a metaphor for the evidence for uniform efficacy of bona fide psychotherapies: reply to Crits-Christoph (1997) and Howard et al. (1997)", Psychological Bulletin, 122 (3): 226–30, doi:10.1037/0033-2909.122.3.226.
19. Wampold, Bruce E; Mondin, GW; Moody, M; Stich, F; Benson, K; Ahn, H (1997), "A meta-analysis of outcome studies comparing bona fide psychotherapies: empirically, 'All must have prizes'", Psychological Bulletin, 122 (3): 203–15, doi:10.1037/0033-2909.122.3.203.
20. Wampold, Bruce E; Serlin, RC (2000), "The consequences of ignoring a nested factor on measures of effect size in analysis of variance", Psychological Methods, 5 (4): 425–33, doi:10.1037/1082-989x.5.4.425, PMID 11194206.
21. Najavits, LM; Strupp, HH (1994), "Differences in the effectiveness of psychodynamic therapies: a process-outcome study", Psychotherapy, 31 (1): 114–23, doi:10.1037/0033-3204.31.1.114.
22. Grencavage, Lisa M; Norcross, John C (1990). "Where are the commonalities among the therapeutic common factors?". Professional Psychology: Research and Practice. 21 (5): 372–378. doi:10.1037/0735-7028.21.5.372.
23. Tracey, Terence JG (2003), "Concept mapping of therapeutic common factors", Psychotherapy Research, 13 (4): 401–13, doi:10.1093/ptr/kpg041, PMID 21827252.
24. Wampold, BE; Imel, ZE; Miller, D (2009), "Barriers to the dissemination of empirically supported treatments: Matching evidence to messages", The Behavior Therapist, 32: 144–55.
25. McAleavey, Andrew A.; Castonguay, Louis G (2015). "The process of change in psychotherapy: common and unique factors". In Gelo, Omar CG; Pritz, Alfred; Rieken, Bernd (eds.). Psychotherapy research: foundations, process, and outcome. New York: Springer. pp. 293–310 [294]. doi:10.1007/978-3-7091-1382-0_15. ISBN 9783709113813. OCLC 900722269. Though many authors view outcome equivalence as the main reason to study common factors in psychotherapy, we cheerfully disagree. Regardless of outcome, it is noncontroversial to say that psychotherapies of many origins share several features of process and content, and it follows that better understanding the patterns of these commonalities may be an important part of better understanding the effects of psychotherapies. That is, irrespective of whether some psychotherapies are equivalent to others in symptomatic outcome, understanding what part of clients' improvement is due to factors that are shared by several approaches appears to us to be a conceptually and clinically important question.
26. Siev, J; Chambless, DL (2007), "Specificity of treatment effects: Cognitive therapy and re- laxation for generalized anxiety and panic disorders", Journal of Consulting and Clinical Psychology, 75 (4): 513–22, doi:10.1037/0022-006x.75.4.513, PMID 17663606.
27. Siev, J; Chambless, DL (2009). "The dodo bird, treatment technique, and disseminating empirically supported treatments". The Behavior Therapist. 32: 69–75.
28. DeRubeis, RJ; Brotman, MA; Gibbons, CJ (2005). "A conceptual and methodological analysis of the nonspecifics argument". Clinical Psychology: Science and Practice. 12 (2): 174–83. doi:10.1093/clipsy.bpi022.
29. Clark, DM; Ehlers, A; Hackmann, A; McManus, F; Fennell, M; Grey, N (2006), et al., "Cognitive therapy versus exposure and applied relaxation in social phobia: A randomized controlled trial", Journal of Consulting and Clinical Psychology, 74 (3): 568–78, doi:10.1037/0022-006x.74.3.568, PMID 16822113.
30. Chambless, Dianne L; Hollon, SD (1998), "Defining empirically supported therapies", Journal of Consulting and Clinical Psychology, 66 (1): 7–18, CiteSeerX 10.1.1.586.4638, doi:10.1037/0022-006x.66.1.7, PMID 9489259.
31. Siev, Jedidiah; Huppert, Jonathan; Chambless, Dianne (Jan 2010), "Treatment specificity for panic disorder: a reply to Wampold, Imel, and Miller (2009)", The Behavior Therapist, 33 (1): 12–14.
32. Watson, D; O'Hara, MW; Stuart, S (2008). "Hierarchical structures of affect and psychopathology and their implications for the classification of emotional disorders". Anxiety and Depression. 25 (4): 282–88. doi:10.1002/da.20496. PMID 18415951.
33. Arch, JJ; Craske, MG (2009). "First-line treatment: a critical appraisal of cognitive behavioral therapy developments and alternatives". Psychiatric Clinics of North America. 32 (3): 525–47. doi:10.1016/j.psc.2009.05.001. PMID 19716989.
34. Hassett, A; Gevirtz, R (2009). "Nonpharmacologic treatment for fibromyalgia: Patient education, cognitive-behavioral therapy, relaxation techniques, and complementary and alternative medicine". Rheumatic Disease Clinics of North America. 35 (2): 393–407. doi:10.1016/j.rdc.2009.05.003. PMC 2743408. PMID 19647150.
35. Beck, AT (1964). "Thinking and depression: theory and therapy". Archives of General Psychiatry. 10 (6): 561–71. doi:10.1001/archpsyc.1964.01720240015003. PMID 14159256.
36. Butler, AC; Chapman, JE; Forman, EM; Beck, AT (2006), "The empirical status of cognitive-behavioral therapy: a review of meta-analyses", Clinical Psychology Review, 26 (1): 17–31, CiteSeerX 10.1.1.413.7178, doi:10.1016/j.cpr.2005.07.003, PMID 16199119
37. Hofmann, SG; Smits, JA (2008). "Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials". Journal of Clinical Psychiatry. 69 (4): 621–32. doi:10.4088/JCP.v69n0415. PMC 2409267. PMID 18363421.
38. Otte, C (2011). "Cognitive behavioral therapy in anxiety disorders: Current state of the evidence". Dialogues in Clinical Neuroscience. 13 (4): 413–21. PMC 3263389. PMID 22275847.
39. Driessen, E; Hollon, S.D. (2010). "Cognitive behavioral therapy for mood disorders: efficacy, moderators and mediators". Psychiatric Clinics of North America. 33 (3): 537–55. doi:10.1016/j.psc.2010.04.005. PMC 2933381. PMID 20599132.
40. Murphy, R; Straebler, S; Cooper, Z; Fairburn, CG (2010). "Cognitive behavioral therapy for eating disorders". Psychiatric Clinics of North America. 33 (3): 611–27. doi:10.1016/j.psc.2010.04.004. PMC 2928448. PMID 20599136.
41. McHugh, RK; Hearon, BA; Otto, MW (2010). "Cognitive behavioral therapy for substance use disorders". Psychiatric Clinics of North America. 33 (3): 511–25. doi:10.1016/j.psc.2010.04.012. PMC 2897895. PMID 20599130.
42. Boileau, B (2011). "A review of obsessive-compulsive disorder in children and adolescents". Dialogues in Clinical Neuroscience. 13 (4): 401–11. PMC 3263388. PMID 22275846.
43. Norton, P.J.; Price, E.C. (2007). "A meta-analytic review of adult cognitive-behavioral treatment outcomes across the anxiety disorders". The Journal of Nervous and Mental Disease. 195 (6): 521–31. doi:10.1097/01.nmd.0000253843.70149.9a. PMID 17568301.
44. Lilienfeld, O.S. (2007). "Psychological treatments that cause harm". Perspectives on Psychological Science. 2 (1): 53–70. CiteSeerX 10.1.1.531.9405. doi:10.1111/j.1745-6916.2007.00029.x. PMID 26151919.
45. Psychiatric Association, American (2002). "Ethical principles of psychologists and code of conduct". American Psychologist. 57 (12): 1067–73. doi:10.1037/0003-066x.57.12.1060.
46. Barlow, DH (2004). "Psychological treatments". American Psychologist. 59 (9): 869–78. doi:10.1037/0003-066x.59.9.869. PMID 15584821.
47. Dimidjian, S; Hollon, DS (2010). "How would we know if psychotherapy were harmful?". American Psychologist. 65 (1): 34–49. doi:10.1037/a0017299. PMID 20063907.
48. Castonguay, GL; Boswell, FJ; Constantino, JM; Goldfried, RM; Hill, EC (2010). "Training implications of harmful effects of psychological treatments". American Psychologist. 65 (1): 34–49. doi:10.1037/a0017330. PMID 20063908.
49. Bergin, AE (1971). The evaluation of therapeutic outcomes. Handbook of psychotherapy and behavior change, 217–70
50. Neimeyer, RA (2000). "Searching for the meaning of meaning: grief therapy and the process of reconstruction". Death Studies. 24 (6): 541–58. doi:10.1080/07481180050121480. PMID 11503667.
51. McNally, RJ; Bryant, RA; Ehlers, A (2003). "Does early psychological intervention promote recovery from posttraumatic stress?" (PDF). Psychological Science in the Public Interest. 4 (2): 45–79. doi:10.1111/1529-1006.01421. PMID 26151755.
52. Nishith, P; Resick, PA; Griffin, MG (2002). "Pattern of change in prolonged exposure and cognitive-processing therapy for female rape victims with posttraumatic stress disorder". Journal of Consulting and Clinical Psychology. 70 (4): 880–86. doi:10.1037/0022-006x.70.4.880. PMC 2970921. PMID 12182271.
53. Gelso, CJ; Latts, MG; Gomez, MJ; Fassinger, RE (2002). "Countertransference management and therapy outcome: an initial evaluation". Journal of Clinical Psychology. 58 (7): 861–67. doi:10.1002/jclp.2010. PMID 12205726.
54. Hill, CE; Mahalik, JR; Thompson, BJ (1989). "Therapist self-disclosure". Psychotherapy. 26 (3): 290–95. CiteSeerX 10.1.1.463.2175. doi:10.1037/h0085438.
55. Norcross, JC (2002), Psychotherapy relationships that work: therapist contributions and responsiveness to patients, New York: Oxford University Press.
56. Hunsley, J; Di Giulio, G (2002), "Dodo bird, phoenix, or urban legend? The question of psychotherapy equivalence", The Scientific Review of Mental Health Practice, 1: 11–22.
57. Barlow, DH (2010), "The dodo bird– again– and again", The Behavior Therapist, 33: 15–16.
58. Ferguson, C; Brannick, M (2012), "Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses", Psychological Methods, 17 (1): 120–28, doi:10.1037/a0024445, PMID 21787082.
59. Norcross, JC (1995), "Dispelling the Dodo bird verdict and the exclusivity myth in psychotherapy", Psychotherapy, 32 (3): 500–4, doi:10.1037/0033-3204.32.3.500.
60. Reid, WJ (1997), "Evaluating the Dodo bird verdict: do all interventions have equivalent outcomes?", Social Work Research, 21: 5–16, doi:10.1093/swr/21.1.5.
61. Wilson, GT; Rachman, SJ (1983), "Meta-analysis and the evaluation of psychotherapy outcome: Limitations and liabilities", Journal of Consulting and Clinical Psychology, 51: 54–64, doi:10.1037/0022-006x.51.1.54.
62. Crits-Christoph (1997), "Limitations of the Dodo bird verdict and the role of clinical trials in psychotherapy research: comment on Wampold et al. (1997)", Psychological Bulletin, 122 (3): 216–20, doi:10.1037/0033-2909.122.3.216.
63. Kazdin, AE; Bass, D (1989), "Power to detect differences between alternative treatments in comparative psychotherapy outcome research", Journal of Consulting and Clinical Psychology, 57: 138–47, doi:10.1037/0022-006x.57.1.138.
64. Norcross, JC; Rossi, JS (1994), "Looking weakly in all the wrong places? Comment of Shapiro et al. (1994)", Journal of Consulting and Clinical Psychology, 62 (3): 535–38, doi:10.1037/0022-006x.62.3.535.
65. Chambless, Dianne (2002), "Beware the Dodo Bird: the dangers of overgeneralization", Clinical Psychology: Science and Practice (commentaries), 9: 13–16, doi:10.1093/clipsy/9.1.13.
66. Beecham, J; Hallam, A; Knapp, M; Baines, B; Fenyo, A; Ashbury, M (1997), "Costing care in hospital and in the community", in Leff, J (ed.), Care in the community: illusion or reality?, pp. 93–108.
67. Glasgow, RE; Klesges, LM; Dzewaltowski, DA; Bull, SS; Esta-brooks, P (2004), "The future of health behavior change research: What is needed to improve translation of research into health promotion practice?", Annals of Behavioral Medicine, 27 (1): 3–12, doi:10.1207/s15324796abm2701_2, PMID 14979858.
68. Woolf, SH (2008), "The meaning of translational research and why it matters", Journal of the American Medical Association, 299 (2): 211–13, doi:10.1001/jama.2007.26, PMID 18182604.