Studies in Psychology 1304-4680 2602-2982 İstanbul Üniversitesi 10.26650/SP2021-959170 The Replication Crisis and Psychotherapy Efficacy Studies Tekrarlama Krizi ve Psikoterapi Etkililik Çalışmaları https://orcid.org/0000-0002-2440-3977 Türkarslan Kutlu Kağan ORTA DOĞU TEKNİK ÜNİVERSİTESİ 04 29 2022 42 1 1 41 06 29 2021 03 05 2022 Copyright © 1956, Psikoloji Çalışmaları 1956 Psikoloji Çalışmaları

In the last ten years, the replication crisis, which has manifested itself and shaken the field of psychology profoundly, has led researchers in every subfield of psychology to reconsider research findings and general assumptions. Though late to this trend, clinical psychology researchers have started to participate in these discussions. Psychotherapy efficacy studies, carried out over many years, constitute a major experimental area of clinical psychology. Psychotherapy efficacy studies are conducted to evaluate the effects of psychological treatments on various mental disorders. The findings of the replication crisis indicate that randomized controlled trials, which are frequently used to examine the efficacy of psychotherapies and meta-analyses that help evaluate the results of randomized controlled trials, have several methodological problems and biases. Randomized controlled trials have problems and biases such as low sample size, allegiance bias, selective outcome reporting, strictness of inclusion or exclusion criteria, using wait-list as a control group, problems of randomization and blinding, exclusion of drop-out subjects from the analyses, and ignoring psychotherapists effects. The main methodological issues of meta-analyses are related to publication bias and the inclusion of low-quality studies in meta-analyses. Research over the years has demonstrated that psychotherapies are effective tools for treating various mental disorders. However, there are two critical points to note. First, the studies showing the efficacy of psychotherapies barely meet current scientific methodological standards. Second, the real-life effectiveness of psychotherapies may be lower than those found in clinical trials. This review article aims to increase awareness regarding replication crisis within the field of clinical psychology and strengthen the methodological approach of future studies by addressing the methodological problems of and suggesting solutions for psychotherapy efficacy studies/meta-analyses

Son 10 yıldır psikoloji alanında kendini gösteren ve alanı derinden sarsan tekrarlama krizi, psikolojinin her alt alanını, araştırma bulgularını ve genel kabullerini yeniden değerlendirmesi için harekete geçirmiştir. Klinik psikoloji alanının geç de olsa tekrarlama krizi tartışmalarına katılmaya başladığı görülmektedir. Uzun yıllardır gerçekleştirilen psikoterapi etkililik (efficacy) çalışmaları, klinik psikoloji içindeki başlıca deneysel alanlardan birini oluşturmaktadır. Psikoterapi etkililik çalışmaları çeşitli psikolojik tedavilerin psikolojik rahatsızlıklar üzerindeki etkilerini değerlendirmek için gerçekleştirilmektedir. Tekrarlama krizi bağlamında ortaya çıkan bulgular, psikoterapilerin etkililiklerini değerlendirmek için sıkça kullanılan seçkisiz kontrollü çalışmaların ve bu çalışmaların sonuçlarının toplu olarak değerlendirilmesine yardımcı olan meta-analizlerin pek çok yöntemsel problem ve yanlılık içerdiğini ortaya çıkarmıştır. Seçkisiz kontrollü çalışmalar düşük örneklem sayısı, bağlılık yanlılığı, seçici sonuç raporlama, örnekleme dahil etme ya da dışarıda bırakma kriterlerinin katılığı, kontrol grubu olarak bekleme listesi kullanımı, seçkisizleştirme ve körleştirme sorunları, çalışmayı bırakan katılımcıların analizlere dahil edilmemesi ve psikoterapist etkilerinin ihmal edilmesi gibi problemler ve yanlılıklar içermektedir. Meta-analizler ile ilişkili başlıca sorunlar ise yayın yanlılığı ve düşük kaliteli çalışmaların meta-analizlere dahil edilmesidir. Tüm bu durumlar psikoterapilerin olumlu etkilerini olduğundan daha yüksek gösterirken, edinilen bulguların sağlıklı bir şekilde değerlendirilmesini engellemektedir. Uzun yıllardır gerçekleştirilen çalışmalar, psikoterapilerin çeşitli psikolojik rahatsızlıkların tedavisinde tercih edilebilecek etkili araçlar olduğunu göstermektedir. Ancak dikkat edilmesi gereken iki önemli husus vardır. Bunlardan ilki psikoterapilerin etkililiğini gösteren çalışmaların yöntemsel anlamda güncel bilimsel standartları yakalamakta zorlanmasıdır. İkinci önemli husus ise psikoterapilerin gerçek etkilerinin çalışmalarda bulunan etkilerden daha az olabileceğidir. Bu derleme makalesinin amacı, psikoterapi etkililik çalışmalarını ve meta-analizleri etkileyen yöntemsel problemlere ve onların çözümlerine değinerek klinik psikoloji alanında bu konuda ortaya çıkan farkındalığın arttırılmasına ve gerçekleştirilecek çalışmaların yöntemsel olarak güçlendirilmesine yardımcı olmaktır.

 Psikoterapi tekrarlama krizi seçkisiz kontrollü çalışma meta-analiz yanlılık dodo kuşu yargısı Psychotherapy replication crisis randomized controlled trail meta-analysis bias dodo bird verdict Abraham, W. T., ve Russell, D. W. (2008). Statistical power analysis in psychological research. Social and Personality Psychology Compass, 2(1), 283-301. https://doi.org/10.1111/j.1751-9004.2007.00052.x google scholar Addis, M. E. ve Krasnow, A. D. (2000). A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology, 68(2), 331-339. https://doi.org/10.1037/0022-006X.68.2.331 google scholar APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. The American Psychologist, 61(4), 271-285. https://doi.org/10.1037/0003-066X.61.4.271 google scholar Arnberg, A. ve Öst, L.-G. (2014). CBT for children with depressive symptoms: A meta-analysis. Cognitive Behaviour Therapy, 43(4), 275-288. https://doi.org/10.1080/16506073.2014.947316 google scholar Baardseth, T. P., Goldberg, S. B., Pace, B. T., Wislocki, A. P., Frost, N. D., Siddiqui, J. R., Lindemann, A. M., Kivlighan, D. M. III, Laska, K. M., Del Re, A. C., Minami, T., & Wampold, B. E. (2013). Cognitive-behavioral therapy versus other therapies: Redux. Clinical Psychology Review, 33(3), 395-405. https://doi.org/10.1016Zj.cpr.2013.01.004 google scholar Berkeljon, A., & Baldwin, S. A. (2009). An introduction to meta-analysis for psychotherapy outcome research. Psychotherapy Research, 19(4-5), 511-518. https://doi.org/10.1080/10503300802621172 google scholar Bradley, H. A., Rucklidge, J. J. ve Mulder, R. T. (2017). A systematic review of trial registration and selective outcome reporting in psychotherapy randomized controlled trials. Acta Psychiatrica Scandinavica, 135(1), 65-77. https://doi.org/10.1111/acps.12647 google scholar Budd, R. ve Hughes, I. (2009). The Dodo Bird Verdict—controversial, inevitable and important: A commentary on 30 years of meta-analyses. Clinical Psychology & Psychotherapy, 16(6), 510-522. https://doi.org/10.1002/cpp.648 google scholar Button, K. S., loannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J. ve Munafö, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365-376. https://doi.org/10.1038/nrn3475 google scholar Carroll, H. A., Toumpakari, Z., Johnson, L. ve Betts, J. A. (2017). The perceived feasibility of methods to reduce publication bias. PloS One, 12(10), e0186472. https://doi.org/10.1371/journal. pone.0186472 google scholar Chambless, D. L. ve Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7-18. https://doi.org/10.1037/0022-006X.66.1.7 google scholar Christley, R. M. (2010). Power and Error: Increased Risk of False Positive Results in Underpowered Studies. The Open Epidemiology Journal, 3(1), 16-19. http://dx.doi. org/10.2174/1874297101003010016 google scholar Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi. org/10.1037//0033-2909.112.1.155 google scholar Cooper, A. A. ve Conklin, L. R. (2015). Drop out from individual psychotherapy for major depression: A meta-analysis of randomized clinical trials. Clinical Psychology Review, 40, 57-65. https://doi. org/10.1016/j.cpr.2015.05.001 google scholar Coyne, J. C. ve Kok, R. N. (2014). Salvaging psychotherapy research: A manifesto. Journal of Evidence-Based Psychotherapies, 14(2), 105-124. google scholar Craigie, M. A. ve Nathan, P. (2009). A nonrandomized effectiveness comparison of broad-spectrum group CBT to individual CBT for depressed outpatients in a community mental health setting. Behavior therapy, 40(3), 302-314. https://doi.org/10.1016/j.beth.2008.08.002 google scholar Crameri, A., von Wyl, A., Koemeda, M., Schulthess, P. ve Tschuschke, V. (2015). Sensitivity analysis in multiple imputation in effectiveness studies of psychotherapy. Frontiers in Psychology, 6, 1042. https://doi.org/10.3389/fpsyg.2015.01042 google scholar Cristea, I. A., Stefan, S., Karyotaki, E., David, D., Hollon, S. D. ve Cuijpers, P. (2017). The effects of cognitive behavioral therapy are not systematically falling: A revision of Johnsen and Friborg (2015). Psychological Bulletin, 143(3), 326-340. https://doi.org/10.1037/bul0000062 google scholar Cuijpers, P. ve Cristea, I. A. (2016). How to prove that your therapy is effective, even when it is not: A guideline. Epidemiology and Psychiatric Sciences, 25(5), 428-435. https://doi.org/10.1017/ S2045796015000864 google scholar Cuijpers, P., Cristea, I. A., Karyotaki, E., Reijnders, M. ve Huibers, M. J. H. (2016). How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence. World Psychiatry, 15(3), 245-258. https://doi.org/10.1002/wps.20346 google scholar Cuijpers, P., Karyotaki, E., Reijnders, M. ve Ebert, D. D. (2018). Was Eysenck right after all? A reassessment of the effects of psychotherapy for adult depression. Epidemiology and Psychiatric Sciences, 28(1), 21-30. https://doi.org/10.1017/S2045796018000057 google scholar Cuijpers, P., Karyotaki, E., Reijnders, M. ve Ebert, D. D. (2019). Is psychotherapy effective? Pretending everything is fine will not help the field forward. Epidemiology and Psychiatric Sciences, 28(3), 356-357. https://doi.org/10.1017/S204579601800080X google scholar Cuijpers, P., Reijnders, M. ve Huibers, M. J. H. (2019). The Role of Common Factors in Psychotherapy Outcomes. Annual Review of Clinical Psychology, 15(1), 207-231. https://doi.org/10.1146/annurev-clinpsy-050718-095424 google scholar Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). The effects of psychotherapy for adult depression are overestimated: A meta-analysis of study quality and effect size. PsychologicalMedicine, 40(2), 211-223. https://doi.org/10.1017/S0033291709006114 google scholar de Felice, G., Giuliani, A., Halfon, S., Andreassi, S., Paoloni, G. ve Orsucci, F. F. (2019). The misleading Dodo Bird verdict. How much of the outcome variance is explained by common and specific factors? New Ideas in Psychology, 54, 50-55. https://doi.org/10.1016/j.newideapsych.2019.01.006 google scholar Del Re, A. C., Flückiger, C., Horvath, A. O., Symonds, D. ve Wampold, B. E. (2012). Therapist effects in the therapeutic alliance-outcome relationship: A restricted-maximum likelihood meta-analysis. Clinical Psychology Review, 32(7), 642-649. https://doi.org/10.1016/j.cpr.2012.07.002 google scholar Donnon, T. (2012). Experimental or RCT research designs: A crisis of nomenclature in medical education. Canadian Medical Education Journal, 3(2), 82-84. google scholar Driessen, E., Hollon, S. D., Bockting, C. L. H., Cuijpers, P. ve Turner, E. H. (2015). Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-Analysis of US National Institutes of Health-Funded Trials. PloS One, 10(9), e0137864. https://doi.org/10.1371/journal.pone.0137864 google scholar Dumas-Mallet, E., Button, K. S., Boraud, T., Gonon, F. ve Munafö, M. R. (2017). Low statistical power google scholar in biomedical science: A review of three human research domains. Royal Society Open Science, 4(2), 160254. https://doi.org/10.1098/rsos.160254 google scholar Easterbrook, P. J., Gopalan, R., Berlin, J. A. ve Matthews, D. R. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867-872. https://doi.org/10.1016/0140-6736(91)90201-Y google scholar Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16(5), 319-324. https://doi.org/10.1037/h0063633 google scholar Eysenck, H. J. (1964). The outcome problem in psychotherapy: A reply. Psychotherapy: Theory, Research & Practice, 1(3), 97-100. https://doi.org/10.1037/h0088591 google scholar Fanelli, D. (2011). Negative results are disappearing from most disciplines and countries. Scientometrics, 90(3), 891-904. https://doi.org/10.1007/s11192-011-0494-7 google scholar Faul, F., Erdfelder, E., Lang, A.-G., ve Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191. https://doi.org/10.3758/BF03193146 google scholar Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5), 532-538. https://doi.org/10.1037/a0015808 google scholar Fensterheim, H. ve Raw, S. D. (1996). Psychotherapy Research Is Not Psychotherapy Practice. Clinical Psychology: Science and Practice, 3(2), 168-171. https://doi.org/10.1111/j.1468-2850.1996. tb00067.x google scholar Fernandez, E., Salem, D., Swift, J. K. ve Ramtahal, N. (2015). Meta-analysis of dropout from cognitive behavioral therapy: Magnitude, timing, and moderators. Journal of Consulting and Clinical Psychology, 83(6), 1108-1122. https://doi.org/10.1037/ccp0000044 google scholar Flint, J., Cuijpers, P., Horder, J., Koole, S. L. ve Munafö, M. R. (2015). Is there an excess of significant findings in published studies of psychotherapy for depression? Psychological Medicine, 45(2), 439446. https://doi.org/10.1017/S0033291714001421 google scholar Fonagy, P., Luyten, P. ve Bateman, A. (2017). Treating Borderline Personality Disorder With Psychotherapy: Where Do We Go From Here? JAMA Psychiatry, 74(4), 316-317. https://doi. org/10.1001/jamapsychiatry.2016.4302 google scholar Freire, E. S. (2006). Randomized Controlled Clinical Trial in Psychotherapy Research: An Epistemological Controversy. Journal of Humanistic Psychology, 46(3), 323-335. https://doi. org/10.1177/0022167806286276 google scholar Friborg, O. ve Johnsen, T. J. (2017). The effect of cognitive-behavioral therapy as an antidepressive treatment is falling: Reply to Ljotsson et al. (2017) and Cristea et al. (2017). Psychological Bulletin, 143(3), 341-345. https://doi.org/10.1037/bul0000090 google scholar Furukawa, T. A., Noma, H., Caldwell, D. M., Honyashiki, M., Shinohara, K., Imai, H., Chen, P., Hunot, V., & Churchill, R. (2014). Waiting list may be a nocebo condition in psychotherapy trials: A contribution from network meta-analysis. Acta Psychiatrica Scandinavica, 130(3), 181192. https://doi.org/10.1111/acps.12275 google scholar Gelman, A. ve Carlin, J. (2014). Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors. Perspectives on Psychological Science, 9(6), 641-651. https://doi. org/10.1177/1745691614551642 google scholar Gelo, O. C. G., Lagetto, G., Dinoi, C., Belfiore, E., Lombi, E., Blasi, S., Aria, M., & Ciavolino, E. (2020). Which methodological practice(s) for psychotherapy science? A systematic review and a proposal. Integrative Psychological & Behavioral Science, 54(1), 215-248. https://doi.org/10.1007/ s12124-019-09494-3 google scholar Glass, G. V. (1982). Meta-analysis: An Approach to the Synthesis of Research Results. Journal of research in science teaching, 19(2), 93-112. google scholar Goldfried, M. R. (2020). The field of psychotherapy: Over 100 years old and still an infant science. Clinical Psychology in Europe, 2(1), e2753. https://doi.org/10.32872/cpe.v2i1.2753 google scholar Goldfried, M. R. ve Wolfe, B. E. (1998). Toward a more clinically valid approach to therapy research. Journal of Consulting and Clinical Psychology, 66(1), 143-150. https://doi.org/10.1037/0022-006X.66.1.143 google scholar Gonzalez-Blanch, C. ve Carral-Fernandez, L. (2017). Cage up Dodo, please! The tale of all psychotherapies being equally effective. Papeles del Psicologo, 38(2), 94-106. https://doi. org/10.23923/pap.psicol2017.2828 google scholar Hansen, W. B. ve Collins, L. M. (1994). Seven ways to increase power without increasing N. NIDA Research Monograph, 142, 184-195. google scholar Hengartner, M. P. (2018). Raising awareness for the replication crisis in clinical psychology by focusing on inconsistencies in psychotherapy research: How much can we rely on published findings from efficacy trials? Frontiers in Psychology, 9, Article 256. https://doi.org/10.3389/fpsyg.2018.00256 google scholar Hoertel, N., de Maricourt, P., Katz, J., Doukhan, R., Lavaud, P., Peyre, H. ve Limosin, F. (2014). Are participants in pharmacological and psychotherapy treatment trials for social anxiety disorder representative of patients in real-life settings? Journal of Clinical Psychopharmacology, 34(6), 697-703. https://doi.org/10.1097/JCP.0000000000000204 google scholar Hoertel, N., Lopez, S., Wang, S., Gonzalez-Pinto, A., Limosin, F. ve Blanco, C. (2015). Generalizability of pharmacological and psychotherapy clinical trial results for borderline personality disorder to community samples. Personality Disorders: Theory, Research, and Treatment, 6(1), 81-87. https:// doi.org/10.1037/per0000091 google scholar Hopewell, S., Clarke, M. ve Mallett, S. (2005). Grey Literature and Systematic Reviews. H. R. Rothstein & A. J. Sutton (Ed.), Publication Bias in Meta-Analysis içinde (s. 49-72). John Wiley & Sons, Ltd. https://doi.org/10.1002/0470870168.ch4 google scholar Hotopf, M. (2002). The pragmatic randomised controlled trial. Advances in Psychiatric Treatment, 8(5), 326-333. doi:10.1192/apt.8.5.326 google scholar Hsu, L. M. (1992). Random sampling, randomization, and equivalence of contrasted groups in psychotherapy outcome research. A. E. Kazdin (Ed.), Methodological issues & strategies in clinical research içinde (s. 91-105). American Psychological Association. https://doi.org/10.1037/10109-018 google scholar Johns, R. G., Barkham, M., Kellett, S. ve Saxon, D. (2019). A systematic review of therapist effects: A critical narrative update and refinement to Baldwin and Imel’s (2013) review. Clinical Psychology Review, 67, 78-93. https://doi.org/10.1016/j.cpr.2018.08.004 google scholar Johnsen, T. J. ve Friborg, O. (2015). The effects of cognitive behavioral therapy as an anti-depressive treatment is falling: A meta-analysis. Psychological Bulletin, 141(4), 747-768. https://doi. org/10.1037/bul0000015 google scholar Jones, J. L. ve Mehr, S. L. (2007). Foundations and assumptions of the scientist-practitioner model. American Behavioral Scientist, 50(6), 766-771. https://doi.org/10.1177/0002764206296454 google scholar Kelley, K. ve Preacher, K. J. (2012). On effect size. Psychological Methods, 17(2), 137-152. https://doi. org/10.1037/a0028086 google scholar Kendall, J. (2003). Designing a research project: Randomised controlled trials and their principles. Emergency Medicine Journal: EMJ, 20(2), 164-168. https://dx.doi.org/10.1136%2Femj.20.2.164 google scholar Kiai, A. (2019). To protect credibility in science, banish “publish or perish”. Nature Human Behaviour, 3(10), 1017-1018. https://doi.org/10.1038/s41562-019-0741-0 google scholar Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R. ve Williamson, P. R. (2010). The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ, 340, c365. https://doi.org/10.1136/bmj.c365 google scholar Lambert, M. J. (2013). The Efficacy and Effectiveness of Psychotherapy. M. J. Lambert (Ed.), Bergin and Garfields Handbook of Psychotherapy and Behavior Change (6. Bs) içinde (s. 169-218). John Wiley & Sons, Ltd. google scholar Landman, J. T. ve Dawes, R. M. (1982). Psychotherapy outcome: Smith and Glass’ conclusions stand up under scrutiny. American Psychologist, 37(5), 504-516. https://doi.org/10.1037/0003-066X.37.5.504 google scholar Laska, K. M., Gurman, A. S. ve Wampold, B. E. (2014). Expanding the lens of evidence-based practice in psychotherapy: A common factors perspective. Psychotherapy, 51(4), 467-481. https://doi. org/10.1037/a0034332 google scholar Leichsenring, F., Abbass, A., Hilsenroth, M. J., Leweke, F., Luyten, P., Keefe, J. R., Midgley, N., Rabung, S., Salzer, S. ve Steinert, C. (2017). Biases in research: Risk factors for non-replicability in psychotherapy and pharmacotherapy research. Psychological Medicine, 47(6), 1000-1011. https:// doi.org/10.1017/S003329171600324X google scholar Leichsenring, F., Abbass, A., Hilsenroth, M. J., Luyten, P., Munder, T., Rabung, S. ve Steinert, C. (2018). “Gold standards,” plurality and monocultures: The need for diversity in psychotherapy. Frontiers in Psychiatry, 9, 159. https://doi.org/10.3389/fpsyt.2018.00159 google scholar Leichsenring, F. ve Rabung, S. (2011). Long-term psychodynamic psychotherapy in complex mental disorders: Update of a meta-analysis. The British Journal of Psychiatry, 199(1), 15-22. https://doi. org/10.1192/bjp.bp.110.082776 google scholar Leykin, Y. ve DeRubeis, R. J. (2009). Allegiance in Psychotherapy Outcome Research: Separating Association From Bias. Clinical Psychology: Science and Practice, 16(1), 54-65. https://doi. org/10.1111/j.1468-2850.2009.01143.x google scholar Lilienfeld, S. O. (2014). The Dodo Bird verdict: Status in 2014. The Behavior Therapist, 37(4), 91-95. google scholar Linardon, J., Kothe, E. J. ve Fuller-Tyszkiewicz, M. (2019). Efficacy of psychotherapy for bulimia nervosa and binge-eating disorder on self-esteem improvement: Meta-analysis. European Eating Disorders Review, 27(2), 109-123. https://doi.org/10.1002/erv.2662 google scholar Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S., Halperin, G., Bishop, M., Berman, J. S., & Schweizer, E. (1999). The researcher’s own therapy allegiances: A “wild card” in comparisons of treatment efficacy. Clinical Psychology: Science and Practice, 6(1), 95-106. https://doi.Org/10.1093/clipsy.6.1.95 google scholar Luborsky, L., Rosenthal, R., Diguer, L., Andrusyna, T. P., Berman, J. S., Levitt, J. T., Seligman, D. A., & Krause, E. D. (2002). The dodo bird verdict is alive and well--mostly. Clinical Psychology: Science and Practice, 9(1), 2-12. https://doi.org/10.1093/clipsy.9.1.2 google scholar Luborsky, L., Singer, B. ve Luborsky, L. (1975). Comparative studies of psychotherapies: Is it true that “everyone has won and all must have prizes”? Archives of General Psychiatry, 32(8), 995-1008. https://doi.org/10.1001/archpsyc.1975.01760260059004 google scholar Marcus, D. K., O’Connell, D., Norris, A. L. ve Sawaqdeh, A. (2014). Is the Dodo bird endangered in the 21st century? A meta-analysis of treatment comparison studies. Clinical Psychology Review, 34(7), 519-530. https://doi.org/10.1016/j.cpr.2014.08.001 google scholar Maxwell, S. E., Lau, M. Y. ve Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70(6), 487-498. https://doi. org/10.1037/a0039400 google scholar Miguel, C., Karyotaki, E., Cuijpers, P. ve Cristea, I. A. (2021). Selective outcome reporting and the effectiveness of psychotherapies for depression. World Psychiatry, 20(3), 444-445. https://dx.doi. org/10.1002%2Fwps.20900 google scholar Miller, L. E., ve Stewart, M. E. (2011). The blind leading the blind: Use and misuse of blinding in randomized controlled trials. Contemporary Clinical Trials, 32(2), 240-243. https://doi. org/10.1016/j.cct.2010.11.004 google scholar Morrison, K. H., Bradley, R. ve Westen, D. (2003). The external validity of controlled clinical trials of psychotherapy for depression and anxiety: A naturalistic study. Psychology and Psychotherapy: Theory, Research and Practice, 76(2), 109-132. https://doi.org/10.1348/147608303765951168 google scholar Mulder, R., Singh, A. B., Hamilton, A., Das, P., Outhred, T., Morris, G., Bassett, D., Baune, B. T., Berk, M., Boyce, P., Lyndon, B., Parker, G., & Malhi, G. S. (2018). The limitations of using randomised controlled trials as a basis for developing treatment guidelines. Evidence-based mental health, 21 (1), 4-6. https://doi.org/10.1136/eb-2017-102701 google scholar Munder, T., Brütsch, O., Leonhart, R., Gerger, H. ve Barth, J. (2013). Researcher allegiance in psychotherapy outcome research: An overview of reviews. Clinical Psychology Review, 33(4), 501-511. https://doi.org/10.1016/j.cpr.2013.02.002 google scholar Munder, T., Flückiger, C., Leichsenring, F., Abbass, A. A., Hilsenroth, M. J., Luyten, P., Rabung, S., Steinert, C., & Wampold, B. E. (2019). Is psychotherapy effective? A re-analysis of treatments for depression. Epidemiology and psychiatric sciences, 28(3), 268-274. https://doi.org/10.1017/ S2045796018000355 google scholar Münch, R. (2014). Academic capitalism: Universities in the global struggle for excellence. Routledge. google scholar Nahum, D., Alfonso, C. A. ve Sönmez, E. (2019). Common Factors in Psychotherapy. A. Javed ve K. N. Fountoulakis (Ed.), Advances in Psychiatry içinde (s. 471-481). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-70554-5_29 google scholar Niemeyer, H., Musch, J. ve Pietrowsky, R. (2012). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for schizophrenia. Schizophrenia Research, 138(2), 103-112. https://doi.org/10.1016/j.schres.2012.03.023 google scholar Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., ... Yarkoni, T. (2015). SCIENTIFIC STANDARDS. Promoting an open research culture. Science (New York, N.Y.), 348(6242), 1422-1425. https://doi.org/10.1126/science.aab2374 google scholar Open Science Collaboration. (2015). PSYCHOLOGY. Estimating the reproducibility of psychological science. Science, 349, Article aac4716. https://doi.org/10.1126/science.aac4716 google scholar O’rourke, K. (2007). An historical perspective on meta-analysis: Dealing quantitatively with varying study results. Journal of the Royal Society of Medicine, 100(12), 579-582. https://doi. org/10.1177/0141076807100012020 google scholar Owen, J., Drinane, J. M., Idigo, K. C. ve Valentine, J. C. (2015). Psychotherapist effects in meta-analyses: How accurate are treatment effects? Psychotherapy, 52(3), 321-328. https://doi. org/10.1037/pst0000014 google scholar Peng, C.-Y. J. ve Chen, L.-T. (2014). Beyond Cohen’s d: Alternative Effect Size Measures for Between-Subject Designs. The Journal of Experimental Education, 82(1), 22-50. https://doi.org/10.1080/00 220973.2012.745471 google scholar Persons, J. B. (1991). Psychotherapy outcome studies do not accurately represent current models of psychotherapy: A proposed remedy. American Psychologist, 46(2), 99-106. https://doi.org/10.1037/0003-066X.46.2.99 google scholar Persons, J. B., Roberts, N. A., Zalecki, C. A. ve Brechwald, W. A. (2006). Naturalistic outcome of case formulation-driven cognitive-behavior therapy for anxious depressed outpatients. Behaviour Research and Therapy, 44(7), 1041-1051. https://doi.org/10.1016/j.brat.2005.08.005 google scholar Persons, J. B. ve Silberschatz, G. (1998). Are results of randomized controlled trials useful to psychotherapists? Journal of Consulting and Clinical Psychology, 66(1), 126-135. https://doi. org/10.1037/0022-006X.66.1.126 google scholar Philips, B., & Falkenström, F. (2021). What Research Evidence Is Valid for Psychotherapy Research?. Frontiers in psychiatry, 11, 625380. https://doi.org/10.3389/fpsyt.2020.625380 google scholar Rifkin, A. (2007). Randomized controlled trials and psychotherapy research [Editorial]. The American Journal of Psychiatry, 164(1), 7-8. https://doi.Org/10.1176/appi.ajp.164.1.7 google scholar Rosenzweig, S. (1954). A transvaluation of psychotherapy: A reply to Hans Eysenck. The Journal of Abnormal and Social Psychology, 49(2), 298-304. https://doi.org/10.1037/h0061172 google scholar Rossi, J. S. (2013). Statistical power analysis. J. A. Schinka, W. F. Velicer ve I. B. Weiner (Ed.), Handbook ofpsychology: Research methods in psychology içinde (s. 71-108). John Wiley & Sons, Inc.. google scholar Rothstein, H. R. ve Hopewell, S. (2009). Grey literature. H. Cooper, L. V. Hedges ve J. C. Valentine (Ed.), The handbook of research synthesis and meta-analysis içinde (s. 103-125). Russell Sage Foundation. google scholar Rubio-Aparicio, M., Marin-Martinez, F., Sanchez-Meca, J. ve L6pez-L6pez, J. A. (2018). A methodological review of meta-analyses of the effectiveness of clinical psychology treatments. Behavior Research Methods, 50(5), 2057-2073. https://doi.org/10.3758/s13428-017-0973-8 google scholar Sakaluk, J. K., Williams, A. J., Kilshaw, R. E. ve Rhyner, K. T. (2019). Evaluating the evidential value of empirically supported psychological treatments (ESTs): A meta-scientific review. Journal of Abnormal Psychology, 128(6), 500-509. https://doi.org/10.1037/abn0000421 google scholar Schafer, T. ve Schwarz, M. A. (2019). The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.00813 google scholar Shapiro, D. A. ve Shapiro, D. (1982). Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychological Bulletin, 92(3), 581-604. https://doi.org/10.1037/0033-2909.92.3.581 google scholar Shean, G. (2012). Some limitations on the external validity of psychotherapy efficacy studies and suggestions for future research. American Journal of Psychotherapy, 66(3), 227-242. https://doi. org/10.1176/appi.psychotherapy.2012.66.3.227 google scholar Shean, G. (2016). Psychotherapy Outcome Research: Issues and Questions. Psychodynamic Psychiatry, 44(1), 1-24. https://doi.org/10.1521/pdps.2016.44.1.1 google scholar Shedler, J. (2018). Where Is the Evidence for “Evidence-Based” Therapy? Psychiatric Clinics ofNorth America, 41(2), 319-329. https://doi.org/10.1016/j.psc.2018.02.001 google scholar Shelby, L. B. ve Vaske, J. J. (2008). Understanding meta-analysis: A review of the methodological literature. Leisure Sciences, 30(2), 96-110. https://doi.org/10.1080/01490400701881366 google scholar Shinohara, K., Tajika, A., Imai, H., Takeshima, N., Hayasaka, Y. ve Furukawa, T. A. (2015). Protocol registration and selective outcome reporting in recent psychiatry trials: New antidepressants and cognitive behavioural therapies. Acta Psychiatrica Scandinavica, 132(6), 489-498. https://doi. org/10.1111/acps.12502 google scholar Smit, Y., Huibers, M. J. H., Ioannidis, J. P. A., van Dyck, R., van Tilburg, W. ve Arntz, A. (2012). google scholar The effectiveness of long-term psychoanalytic psychotherapy—A meta-analysis of randomized controlled trials. Clinical Psychology Review, 32(2), 81-92. https://doi.org/10.1016/j. cpr.2011.11.003 google scholar Smith, M. L. ve Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32(9), 752-760. https://doi.org/10.1037/0003-066X.32.9.752 google scholar Sohn, D. (1996). Publication bias and the evaluation of psychotherapy efficacy in reviews of the research literature. Clinical Psychology Review, 16(2), 147-156. https://doi.org/10.1016/0272-7358(96)00005-0 google scholar Remission in CBT for adult anxiety disorders: A meta-analysis. Clinical Psychology Review, 61, 1-8. https://doi.org/10.1016/j.cpr.2018.03.002 google scholar Stricker, J. ve Günther, A. (2019). Scientific misconduct in psychology: A systematic review of prevalence estimates and new empirical data. Zeitschrift für Psychologie, 227(1), 53-63. http:// dx.doi.org/10.1027/2151-2604/a000356 google scholar Strupp, H. H. (1963). The outcome problem in psychotherapy revisited. Psychotherapy: Theory, Research & Practice, 1(1), 1-13. https://doi.org/10.1037/h0088565 google scholar Tackett, J. L., Brandes, C. M., King, K. M. ve Markon, K. E. (2019). Psychology’s replication crisis and clinical psychological science. Annual Review of Clinical Psychology, 15, 579-604. https://doi. org/10.1146/annurev-clinpsy-050718-095710 google scholar Tackett, J. L., Lilienfeld, S. O., Patrick, C. J., Johnson, S. L., Krueger, R. F., Miller, J. D., Oltmanns, T. F., & Shrout, P. E. (2017). It’s time to broaden the replicability conversation: Thoughts for and from clinical psychological science. Perspectives on PsychologicalScience, 12(5), 742-756. https://doi. org/10.1177/1745691617690042 google scholar Task Force on Promotion and Dissemination of Psychological Procedures Division of Clinical Psychology. (1995). Training in and dissemination of empirically-validated psychological treatments: Report and recommendations. The Clinical Psychologist, 48(1), 3-23. google scholar Thase, M. E. (1999). What is the investigator allegiance effect and what should we do about it? Clinical Psychology: Science and Practice, 6(1), 113-115. https://doi.org/10.1093/clipsy.6.1.113 google scholar Tracey, T. J. G., Wampold, B. E., Lichtenberg, J. W. ve Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69(3), 218-229. https://doi.org/10.1037/ a0035099 google scholar Tolin, D. F. (2010). Is cognitive-behavioral therapy more effective than other therapies?: A meta-analytic review. Clinical Psychology Review, 30(6), 710-720. https://doi.org/10.1016/j.cpr.2010.05.003 google scholar Tolin, D. F., McKay, D., Forman, E. M., Klonsky, E. D. ve Thombs, B. D. (2015). Empirically supported treatment: Recommendations for a new model. Clinical Psychology: Science and Practice, 22(4), 317-338. https://doi.org/10.1111/cpsp.12122 google scholar van der Zweerde, T., Bisdounis, L., Kyle, S. D., Lancee, J. ve van Straten, A. (2019). Cognitive behavioral therapy for insomnia: A meta-analysis of long-term effects in controlled studies. Sleep Medicine Reviews, 48, 101208. https://doi.org/10.1016/j.smrv.2019.08.002 google scholar Vevea, J. L. ve Woods, C. M. (2005). Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychological Methods, 10(4), 428-443. google scholar von Wolff, A., Jansen, M., Hölzel, L. P., Westphal, A., Harter, M. ve Kriston, L. (2014). Generalizability of findings from efficacy trials for chronic depression: An analysis of eligibility criteria. Psychiatric Services, 65(7), 897-904. https://doi.org/10.1176/appi.ps.201300309 google scholar Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Routledge. google scholar Wampold, B. E. (2015). How important are the common factors in psychotherapy? An update. World Psychiatry, 14(3), 270-277. https://doi.org/10.1002/wps.20238 google scholar Wampold, B. E. (2019). The basics of psychotherapy: An introduction to theory and practice (2. Bs.). American Psychological Association. https://doi.org/10.1037/0000117-000 google scholar Wampold, B. E., Hollon, S. D. ve Hill, C. E. (2011). Unresolved questions and future directions in psychotherapy research. History of psychotherapy: Continuity and change (2. Bs) içinde (s. 333356). American Psychological Association. https://doi.org/10.1037/12353-011 google scholar Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K. ve Ahn, H. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially, “all must have prizes.” Psychological Bulletin, 122(3), 203-215. https://doi.org/10.1037/0033-2909.122.3.203 google scholar Weinberger, J. (1995). Common factors aren’t so common: The common factors dilemma. Clinical Psychology: Science and Practice, 2(1), 45-69. https://doi.Org/10.1111/j.1468-2850.1995. tb00024.x google scholar Westbrook, D. ve Kirk, J. (2005). The clinical effectiveness of cognitive behaviour therapy: Outcome for a large sample of adults treated in routine practice. Behaviour Research and Therapy, 43(10), 1243-1261. https://doi.org/10.1016/j.brat.2004.09.006 google scholar Westen, D. ve Bradley, R. (2005). Empirically supported complexity: Rethinking evidence-based practice in psychotherapy. Current Directions in Psychological Science, 14(5), 266-271. https:// doi.org/10.1111/j.0963-7214.2005.00378.x google scholar Westen, D. ve Morrison, K. (2001). A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: An empirical examination of the status of empirically supported therapies. Journal of Consulting and Clinical Psychology, 69(6), 875-899. https://doi. org/10.1037/0022-006X.69.6.875 google scholar Westen, D., Novotny, C. M. ve Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130(4), 631-663. https://doi.org/10.1037/0033-2909.130.4.631 google scholar Wierzbicki, M. ve Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice, 24(2), 190-195. https://doi.org/10.1037/0735-7028.24.2.190 google scholar Wojnarowski, C., Firth, N., Finegan, M. ve Delgadillo, J. (2019). Predictors of depression relapse and recurrence after cognitive behavioural therapy: A systematic review and meta-analysis. Behavioural and Cognitive Psychotherapy, 47(5), 514-529. https://doi.org/10.1017/S1352465819000080 google scholar