Meta-analytic studies on whether psychotherapy is more effective than no therapy at all reveal that:

Abstract

Objective: In the past decades, many different types of psychotherapy for adult depression have been developed. Method: In this meta-analysis we examined the effects of 15 different types of psychotherapy using 385 comparisons between a therapy and a control condition: Acceptance and commitment therapy, mindfulness-based cognitive behavior therapy (CBT), guided self-help using a self-help book from David Burns, Beck’s CBT, the “Coping with Depression” course, two subtypes of behavioral activation, extended and brief problem-solving therapy, self-examination therapy, brief psychodynamic therapy, non-directive counseling, full and brief interpersonal psychotherapy, and life review therapy. Results: The effect sizes ranged from g = 0.38 for the “Coping with Depression” course to g = 1.10 for life review therapy. There was significant publication bias for most therapies. In 70% of the trials there was at least some risk of bias. After adjusting studies with low risk of bias for publication bias, only two types of therapy remained significant (the “Coping with Depression” course, and self-examination therapy). Conclusions: We conclude that the 15 types of psychotherapy may be effective in the treatment of depression. However, the evidence is not conclusive because of high levels of heterogeneity, publication bias, and the risk of bias in the majority of studies.

Obiettivi: Nei decenni passati, sono stati sviluppati molti tipi differenti di psicoterapia per la depressione degli adulti. Metodo: In questa meta-analisi abbiamo esaminato gli effetti di 15 differenti tipi di psicoterapia usando 385 confronti tra una terapia e una condizione di controllo: acceptance and commitment therapy, terapia cognitivo-comportamentamentale basata sulla mindfulness, auto-aiuto guidato usando un libro di auto-aiuto di David Burns, la CBT di Beck, il corso “Coping con depressione”, due sottotipi di attivazione comportamentale, terapia problem-solving breve e estesa, terapia auto-esame, terapia psicodinamica breve, counseling non direttivo, psicoterapia interpersonale completa e breve, e terapia life review. Risultati: Gli effect sizes vanno da g=.38 per il corso “Coping con Depression” a g=1.10 per la terapia life review. C’è un significativo bias di pubblicazione per la maggior parte delle terapie. Nel 70% dei trials c'era almeno qualche rischio di bias. Dopo studi di adattamento con basso rischio di errori per bias di pubblicazione, solo due tipi di terapia rimangono significative (il corso “Coping con Depression” e la terapia auto-esame). Conclusioni: Concludiamo che i 15 tipi di psicoterapia potrebbe essere efficace nel trattamento della depressione. Comunque, l'evidenza non è definitiva a causa dei livelli elevati di eterogeneità, bias di pubblicazione, e il rischio di bias nella maggioranza degli studi.

Objetivo: Nas últimas décadas, muitos tipos diferentes de psicoterapia para depressão em adultos foram desenvolvidos. Método: Nesta meta-análise, examinamos os efeitos de 15 tipos diferentes de psicoterapia usando 385 comparações entre uma terapia e uma condição de controle: terapia de aceitação e compromisso, terapia cognitivo comportamental (TCC) baseada em mindfulness, auto-ajuda guiada - livro de ajuda de David Burns, TCC de Beck, o curso "Lidando com a depressão", dois subtipos de ativação comportamental, terapia estendida e breve para solução de problemas, terapia de auto-exame, terapia psicodinâmica breve, terapia psicodinâmica breve, aconselhamento não-diretivo, psicoterapia interpessoal completa e breve e terapia de revisão de vida. Resultados: os tamanhos dos efeitos variaram de g=0,38 para o curso "Lidar com a Depressão" até g=1,10 para a terapia de revisão da vida. Houve um viés de publicação significativo para a maioria das terapias. Em 70% dos estudos, havia pelo menos algum risco de viés. Após o ajuste de estudos com baixo risco de viés para viés de publicação, apenas dois tipos de terapia permaneceram significativos (o curso “Lidar com a Depressão” e terapia de auto-exame). Conclusões: Concluímos que os 15 tipos de psicoterapia podem ser eficazes no tratamento da depressão. No entanto, as evidências não são conclusivas devido aos altos níveis de heterogeneidade, viés de publicação e risco de viés na maioria dos estudos.

目的：過去數十年已經發展出許多不同類型的成人憂鬱症心理治療方法。方 法：在這項後設分析中，我們針對有治療組和控制組設計的研究，包含385 種治療 情境，用以檢視15 種不同類型心理治療的效果：接受與承諾治療、以正念為基礎 的認知行為治療(CBT)、採用David Burns 自助書籍的引導式自我協助、 Beck 的 CBT、「戰勝憂鬱」課程、行為激發的兩種子類型、廣義與狹義的問題解決治療、 自我檢視治療、短期心理動力治療、非指導式諮商、全面和短期的人際互動心理治 療，以及生命回顧治療。結果：效果值從「戰勝憂鬱」課程的g = 0.38 到g=1.10 的 生命回顧治療。大多數的療法存在著明顯的出版偏誤，且在70％的實驗中，都至少 有些偏誤的風險。在針對出版偏誤較低的研究進行調整後，只剩兩種類型的治療仍 舊有效（「戰勝憂鬱」課程和自我檢視治療）。結論：我們得到的結論是，這15 種 心理療法可能對治療憂鬱症有效。但是由於大多數研究的異質性、出版偏誤，以及 存在著偏見的風險均高，因此證據尚無定論。

Clinical or methodological significance of this article: Although many types of psychotherapy for adult depression have been examined in randomized trials, only a relatively small number of generic types of psychotherapy have been examined in meta-analyses. In this meta-analysis, we examined the effects of fifteen more specific types of therapy. This is important for clinical practice because these therapies can be used in routine care. Although most of them may be effective, the research is limited by high levels of heterogeneity, publication bias, and the risk of bias in the majority of studies.

Several different types of psychotherapies have been found to be effective in the treatment of adult depression, including cognitive behavior therapy (CBT) (Cuijpers, Sijbrandij, et al., 2013; Furukawa et al., 2014), behavioral activation therapy (BAT) (Dimidjian, Barrera Jr, Martell, Muñoz, & Lewinsohn, 2011; Ekers, Richards, & Gilbody, 2008; Shinohara et al., 2013), interpersonal psychotherapy (IPT) (Churchill et al., 2010; Cuijpers, Donker, Weissman, Ravitz, & Cristea, 2016), problem-solving therapy (PST) (Cuijpers, de Wit, Kleiboer, Karyotaki, & Ebert, 2017; Malouff, Thorsteinsson, & Schutte, 2007), non-directive counseling (Cuijpers et al., 2012) and brief psychodynamic therapy (Driessen et al., 2013). Meta-analyses of trials directly comparing these therapies with each other, typically indicate that there are no or no major differences between the effects of these therapies (Barth et al., 2013; Cuijpers, 2017; Cuijpers, van Straten, Andersson, & van Oppen, 2008).

How to define these different types of therapies is not straightforward, however. There are no “official” definitions for these types of therapy. On the one hand, there are the very broad generic categories of psychodynamic, cognitive–behavioral and humanistic categories of psychotherapy (Wampold & Imel, 2015). These categories are, however, not well delineated, and there are all kinds of therapies that are not easily captured in one of these categories, such as IPT or couple therapy. On the other end of the spectrum of definitions are the therapies that are defined by the use of a specific manual, such as CBT according to the manual of Beck, Rush, Shaw, and Emery (1979), or IPT according to the manual of Klerman, Weissman, Rounsaville, and Chevron (1984). But most reports on psychological treatments only describe the actual therapies very briefly, which makes it impossible to make clear categories of therapies. Furthermore, in many studies, the authors refer to a specific manual, but also report that they have made adaptations to a specific population, setting or aims of the intervention, or they have inserted parts from other therapies in the original therapy. This makes it even more complicated to define and operationalize types of therapies.

Most meta-analyses have solved this problem by using more generic “brand” names, such as CBT (when cognitive restructuring is a core element of the therapy), BAT (when pleasant activity scheduling is one of the core elements of the therapy), or PST (when problem-solving techniques are the core element of the therapy). However, such generic brand names often include several different subtypes of therapy with different manuals or different approaches. For many of these subtypes, no meta-analyses have ever been done. This information is important, however, because when only some of these subtypes are effective, it indicates which manuals or approaches should be advised to clinicians and patients. Moreover, generic brand names do not indicate how these therapies should be conducted in routine practice, while manuals and specific approaches do.

In the current study, we build on the approach to categorize psychotherapies for depression that we used in an earlier meta-analysis (Cuijpers, van Straten, Warmerdam, & Andersson, 2008). In this study, we closely examined the therapies that were used in 91 comparative outcome studies on depression and categorized them into clusters of therapies of 5 or more studies. We formulated definitions of the major types of psychological treatment that were found, and checked whether the interventions from the studies met these descriptions (the definitions are given in Cuijpers, van Straten, Warmerdam, & Andersson, 2008). The exact formulation of the definitions was discussed in detail by the authors, who are experts in the research and practice of psychological treatments for depression.

In the current study, we started with the major categories and definitions that were developed in the earlier study (“generic” therapies), and tried to make subcategories and more specified definitions of the treatments (“specific”) therapies. We did this using the same approach as we used for the broad categories, by examining the interventions described in the studies, as well as literature on each of the treatments (the exact methods and references are given in the Methods section). We stopped this process until we had treatments that had a sufficient number of studies (at least 5) and was based on one specific manual or method that could clearly be distinguished from other psychological treatments of the same category.

Our goal was to examine the effects of each of these specific treatments compared to control conditions (waiting list, care as usual, placebo, other) and to explore in meta-regression analyses whether we could find differences between the specific psychological treatments.

We used an existing database of studies on the psychological treatment of depression. This database has been described in detail elsewhere (Cuijpers, Reijnders, & Karyotaki, 2018), and has been used in a series of earlier published meta-analyses (Cuijpers, 2017). For this database, we searched four major bibliographical databases (PubMed, PsycInfo, Embase and the Cochrane Library) by combining terms (both index terms and text words) indicative of depression and psychotherapies, with filters for randomized controlled trials. The full search string for one database (PubMed) is given in Appendix A. We also searched a number of bibliographical databases to identify trials in non-Western countries (Cuijpers, Karyotaki, Reijnders, Purgato, & Barbui, 2018), because the number of trials on psychological treatments in these countries is growing rapidly. Furthermore, we checked the references of earlier meta-analyses on psychological treatments of depression. The database is continuously updated and was developed through a comprehensive literature search (from 1966 to January, 1st 2018). All records were screened by two independent researchers and all papers that could possibly meet inclusion criteria according to one of the researchers were retrieved as full-text. The decision to include or exclude a study in the database was also done by the two independent researchers, and disagreements were solved through discussion.

We included studies that were: (i) a randomized trial (ii) in which a psychotherapy (iii) for adult depression was (iv) compared with a control group (waiting list, care-as-usual, placebo, other inactive treatment) or another treatment (psychological or pharmacological). Depression could be established with a diagnostic interview or with a score above a cut-off on a self-report measure. We defined psychotherapy according to Norcross (Campbell, Norcross, Vasquez, & Kaslow, 2013):

Psychotherapy is the informed and intentional application of clinical methods and interpersonal stances derived from established psychological principles for the purpose of assisting people to modify their behaviors, cognitions, emotions, and/or other personal characteristics in directions that the participants deem desirable.

Because we found in a previous meta-analyses (Cuijpers, Noma, Karyotaki, Cipriani, & Furukawa, 2019) that there are no significant differences between treatment formats (individual, group, guided self-help, internet-based therapy) as long as there is human involvement, we allowed any of these treatment formats.

Co-morbid mental or somatic disorders were not used as an exclusion criterion. Because of the differences in control conditions, studies on inpatients were excluded, in order to reduce heterogeneity (Cuijpers et al., 2011). We also excluded maintenance studies, aimed at people who had already recovered or partly recovered after an earlier treatment.

As indicated earlier, we used the broad categories and definitions of psychotherapies that were developed in an earlier study as a starting point (Cuijpers, van Straten, Warmerdam, et al., 2008). We removed one category (social skills training, because there were only few controlled trials on this treatment) and added two other categories that were examined in more than 10 trials (the cluster of third wave therapies and life review therapies). We critically read the controlled trials on psychotherapies for depression and tried to identify more specific treatments. Wherever possible, we built on the literature for each of these therapy types. We stopped with further specification when the number of studies was 5 or less. The resulting specified types of therapy and the literature on which these specified types were based, are given in .

Download CSVDisplay Table

As in our previous meta-analyses using our database of randomized trials, we assessed the validity of included studies using four criteria of the “Risk of bias” assessment tool, developed by the Cochrane Collaboration (Higgins et al., 2011). This tool assesses possible sources of bias in randomized trials, including the adequate generation of allocation sequence; the concealment of allocation to conditions; the prevention of knowledge of the allocated intervention (masking of assessors); and dealing with incomplete outcome data (this was assessed as positive when intention-to-treat analyses were conducted, meaning that all randomized patients were included in the analyses). Assessment of the validity of the included studies was conducted by two independent researchers, and disagreements were solved through discussion.

We also coded participant characteristics (depressive disorder of scoring high on a self-rating scale; recruitment method; target group); characteristics of the psychotherapies (treatment format; number of sessions); and general characteristics of the studies (type of control group; country where the study was conducted). Treatment format was coded as individual, group or guided-self help (including internet-based guided self-help).

For each comparison between a psychotherapy and a control condition, the effect size indicating the difference between the two groups at post-test was calculated (Hedges’ g) (Hedges & Olkin, 1985). Effect sizes of 0.8 can be assumed to be large, while effect sizes of 0.5 are moderate, and effect sizes of 0.2 are small (Cohen, 1988). Effect sizes were calculated by subtracting (at post-test) the average score of the psychotherapy group from the average score of the control group, and dividing the result by the pooled standard deviation. Because some studies had relatively small sample sizes we corrected the effect size for small sample bias (Hedges & Olkin, 1985). If means and standard deviations were not reported, we used the procedures of the Comprehensive Meta-Analysis software (see below) to calculate the effect size using dichotomous outcomes (Borenstein, Hedges, Higgins, & Rothstein, 2009); and if these were not available either, we used other statistics (such as t-value or p-value) to calculate the effect size.

In order to calculate effect sizes we used all measures examining depressive symptoms (such as the Beck Depression Inventory/BDI (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961); the BDI-II (Beck, Steer, & Brown, 1996); or the Hamilton Rating Scale for Depression/HAMD-17 (Hamilton, 1960)). If more than one depression measure was used in a study, the effect sizes of for these measures were pooled within the study, before the effect sizes were pooled across studies, so that each comparison had only one effect size.

To calculate pooled mean effect sizes, we used the computer program Comprehensive Meta-Analysis (version 3.3070; CMA). Because we expected considerable heterogeneity among the studies, we employed a random-effects pooling model in all analyses.

Numbers-needed-to-be-treated (NNT) were calculated using the formulae provided by Furukawa (1999), in which the control group’s event rate was set at a conservative 19% (based on the pooled response rate of 50% reduction of symptoms across trials in psychotherapy for depression) (Cuijpers, Turner, Koole, van Dijke, & Smit, 2014). As a test of homogeneity of effect sizes, we calculated the I2-statistic, which is an indicator of heterogeneity in percentages. A value of 0% indicates no observed heterogeneity, and larger values indicate increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity (Higgins, Thompson, Deeks, & Altman, 2003). We calculated 95% confidence intervals around I2 (Ioannidis, Patsopoulos, & Evangelou, 2007), using the non-central chi-squared-based approach within the heterogi module for Stata (Orsini, Bottai, Higgins, & Buchan, 2006). In addition, we calculated the prediction interval, which indicates the range in which the true effect size of 95% of all populations will fall.

We tested for publication bias by inspecting the funnel plot on primary outcome measures and by Duval and Tweedie’s trim and fill procedure (Duval & Tweedie, 2000) as implemented in CMA. This procedure yields an estimate of the effect size after the publication bias has been taken into account, through imputing negative studies that should have been available according to the asymmetry of the funnel plot, but were not found in the systematic searches. We also conducted Egger’s test of the intercept to quantify the bias captured by the funnel plot and to test whether it was significant.

In order to examine whether the effects of the different types of therapy differed significantly from each other, we conducted a meta-regression analysis with the effect size as the dependent variable. As predictors we entered the categories of therapies, as well as three variables that have been consistently found to be significant predictors of the effects of therapies: type of control condition (Mohr et al., 2014), risk of bias (Cuijpers, van Straten, Bohlmeijer, Hollon, & Andersson, 2010), and whether or not the study was conducted in a Western country (Cuijpers et al., 2018). No other characteristic of psychotherapies or studies has been found to consistently predict outcome (Cuijpers, 2017; Cuijpers, van Straten, Andersson, et al., 2008; Cuijpers, van Straten, et al., 2009).

A summary of key characteristics of the included studies are presented in Appendix B.

Of the 385 psychotherapy conditions, 167 (43%) used an individual format, 121 (31%) a group format, 77 (20%) guided self-help, 11 (3%) telephone, and 9 (2%) used a mixed treatment format. The number of sessions ranged from 1 to 60, with 113 conditions (29%) have 6 or less sessions, 202 (52%) with 7 to 12, 60 (16%) with 13 to 24, and 3 (1%) with more than 24 sessions (in 3 comparisons the number of sessions was not reported).

In 204 of the 385 comparisons between a treatment and a control condition, CBT was used as the intervention (53%), 21 used BAT (5%), 19 used third wave therapies (5%), 30 used PST (8%), 27 used IPT (7%), 12 used psychodynamic therapy (3%), 19 used non-directive supportive therapy (5%), and the remaining 53 used another type of treatment (14%).

In 127 studies (41%), participants were (partly) recruited through the community, 77 studies (25%) recruited only from clinical samples, and 105 (34%) used other recruitment strategies (such as screening general medical patients or pregnant women). In 161 studies (52%) participants had to meet criteria for a diagnosed mood disorder, while in 148 studies (48%) they had to score above a cut-off on a self-rating depression scale. Of the 309 studies, 174 (56%) were aimed at adults in general, 47 (15%) were aimed at older adults, 21 (7%) at student populations, 42 (14%) at women with postpartum depression, 64 (21%) at patients with comorbid general medical disorders and 37 (12%) were aimed at other specific target groups.

Of the 309 studies, 262 were conducted in a Western country (85%). A total of 27 studies (9%) were conducted before 1991, 105 (34%) between 1991 and 2010, and 177 (57%) after 2010. For the control group, 146 studies used care-as-usual (47%), nine studies used placebo (3%), 111 studies used a waiting list (36%), and the remaining 42 used another control group (14%).

The effect size was calculated based on the means, standard deviation and N in 345 comparisons (90%), in 28 comparisons it was based on dichotomous outcomes (7%), and 12 comparisons (3%) used other statistics (e.g., t or p-value) to calculate the effect size.

The risk of bias in most studies was considerable. A total of 169 of the 309 studies reported an adequate sequence generation (55%). 144 studies reported allocation to conditions by an independent (third) party (47%). 83 studies reported using blinded outcome assessors (27%), and 202 used only self-report outcomes (65%). In 184 studies intent-to-treat analyses were conducted (60%). Only 91 studies (29%) met all quality criteria (using a self-report measure was rated as positive for blinding of outcome assessors). 136 studies (44%) met two or three of the criteria and the 82 remaining studies met no or only one criterion (27%).

The overall pooled effect size for all psychotherapies was g = 0.72 (95% CI: 0.67 ∼ 0.78), which corresponds with an NNT of 4.04. Heterogeneity was high (I2 = 81; 95% CI: 79 ∼ 81), and the prediction interval ranged from −0.17 to 1.61 ().

Download CSVDisplay Table

There were several studies in which more than one psychotherapy was compared with the same control group. These effect sizes are not independent of each other and may affect the effect sizes and heterogeneity. We examined this by conducting two analyses in which only one effect size per study was included, one with only the highest effect size of the study and one with only the lowest effect of the study. As can be seen in , the overall pooled effect size and the level of heterogeneity was not affected considerably. We also conducted an analysis in which outliers, specifically studies with very large effect sizes (g > 2), were excluded. This resulted in a somewhat smaller effect size (g = 0.61), but heterogeneity remained moderate to high (I2 = 65).

Duvall and Tweedie’s trim and fill procedure pointed at a considerable risk for publication bias in all main analyses. After imputation of missing studies, the effect size dropped below g = 0.50 in most analyses ().

We first examined the effects of the generic types of psychotherapy, based on our previous definitions of psychotherapy (Cuijpers, van Straten, Warmerdam, et al., 2008). These definitions are given in . The effect sizes for each of these therapies ranged from g = 0.39 for psychodynamic therapy to g = 1.05 for BAT. Heterogeneity was high in all therapies, except in counseling where heterogeneity was moderate (I2 ranged from 45% for counseling to 87% for PST and IPT). The NNTs ranged from 3 to 8. Because the effects of the therapies may differ across type of control group, we have also given the effect sizes for each type of therapy for the three types of control group (waiting list, care-as-usual, and the other control groups) in .

Download CSVDisplay Table

Indications for publication bias were found for CBT, BAT, PST, psychodynamic therapy, and counseling. The adjusted effect sizes were considerably smaller than the unadjusted effect sizes for all these therapies. No indications for publication bias were found for 3rd wave therapies and IPT.

We examined whether the generic types of therapy differed from each other in a multivariate meta-regression analysis (), in which we adjusted for the three variables that have been consistently found to be associated with the effect size (type of control condition, risk of bias, and whether or not the study was conducted in a Western country or not). As can be seen in , a test for the whole set of generic therapies was not significant either (p = .73), although BAT was found to be more effective than CBT (the reference).

Download CSVDisplay Table

The effects of the fifteen more specific types of psychotherapy are given in Tables II and III. The effect sizes ranged from g = 0.38 for the “Coping with Depression” course to g = 1.10 for Life review therapy. All effect sizes were significantly different from zero. The NNTs ranged from 3 to 8. Heterogeneity was high (I2 > 75%) for contextual behavioral activation, extended and brief PST, full and brief IPT, and life review. Heterogeneity was low to moderate (I2 < 50%) for acceptance and commitment therapy, pleasant events behavioral activation, guided self-help with Burns’ book, the “Coping with Depression” course, and non-directive counseling.

Indications for publication bias was found for the majority of therapies, except for acceptance and commitment therapy, pleasant events behavioral activation, guided self-help with Burns’ book, and full and brief IPT. The effect size was reduced with more than 50% after adjustment for publication bias in contextual BAT, extended and brief PST, and psychodynamic therapy.

In the meta-regression analyses (), we found that overall the specific types of therapies did not differ significantly from each other (p = 0.16). However, when the “Coping with Depression” course was compared to the reference group (Beck’s CBT), it was significantly less effective (p = <0.001).

Because we found a highly significant association between the effect size and risk of bias, we conducted sensitivity analyses with the set of studies with low risk of bias. In 103 comparisons (27%) risk of bias was low. The pooled effect size of these comparisons was g = 0.48, which corresponds with an NNT of 6.44. Heterogeneity was high (I2 = 69) and the prediction interval ranged from −0.05 to 1.01.

For the generic therapies, non-directive counseling and IPT had less than 5 studies and were not included in these analyses. The effect sizes from the other generic therapies ranged from g = 0.27 (PST) to 0.78 (BAT). Heterogeneity was still considerable for all therapies, except for third wave therapies. There were indications of publication bias in all generic types of therapy. In a multivariate metaregression analysis in which we adjusted for control group, and whether or not the study was conducted in a Western country, we found no indication that one type of generic therapy was more or less effective than other therapies.

Only four specific therapies had five or more studies with low risk of bias (). Effect sizes ranged from g = 0.34 for the “Coping with Depression” course, to g = 0.87 for Beck’s CBT. Heterogeneity was considerable in all four therapies. When the effect sizes were adjusted for publication bias (using Duval and Tweedie’s trim and fill procedure ()), only two therapies remained effective: the “Coping with Depression” course and self-examination therapy.

Download CSVDisplay Table

We examined the effects of 7 generic types of psychotherapy for adult depression, as well as those of 15 more specific types of psychotherapy. We found that all therapies, generic and specific, had significant, moderate to large effects on adult depression. Psychodynamic therapy, the “Coping with Depression” course and a specific version of PST (self-examination therapy) had effect sizes smaller than g = 0.5. All other therapies had effect sizes ranging from g = 0.57 for full IPT to g = 1.07 for extended PST. This is good news in the sense that all therapies had significant effects on depression. However, these findings were much less positive after taking heterogeneity, publication bias and risk of bias into account.

One problem for many types of the examined psychotherapy is that the level of heterogeneity was high, and many of the prediction intervals were broad and included zero. This means that it is difficult to predict the effect size of the next study that is done with this therapy, and that study may just as well find negative effects. The resulting effect sizes differ so much for one type of therapy, that it cannot be reliably predicted what the true effect size is.

Publication bias is another problem for this body of research. Although we examined this with indirect evidence (the asymmetry of the funnel plot), we found strong indications that the effects of several of the therapies are overestimated because negative studies are not published. Although it may be argued that this is indirect evidence and there may be other causes for these findings, this is in line with other research in which we found that NIH-funded trials on psychotherapy for depression were indeed often not published and that this affects the overall effects found for psychotherapy that are comparable to what we found (Driessen, Hollon, Bockting, Cuijpers, & Turner, 2015). After adjustment for publication bias, four specific types were no longer significantly different from zero (contextual behavioral activation therapy, extended and brief PST, and psychodynamic therapy).

Risk of bias is another important problem in research on psychotherapies for depression. In 70% of the trials (92/309) there was at least some risk of bias. And the studies with low risk of bias, clearly indicated smaller effect sizes than the ones that had (at least some) risk of bias. Only four of the 15 specific types of therapy had 5 or more trials without risk of bias. And the effects found in these studies were more modest than what was found for all studies (including the ones with risk of bias). When the studies with low risk of bias were adjusted for publication bias, only two types of therapy remained significant (the “Coping with Depression” course, and self-examination therapy).

These findings suggest that the effects of the therapies are considerably overestimated when all studies are taken together and when heterogeneity, publication bias, and risk of bias are not taken into account. In an earlier study, we already found that the effects of CBT for depression and anxiety disorders have been overestimated considerably because of publication bias and risk of bias (Cuijpers, Cristea, Karyotaki, Reijnders, & Huibers, 2016). Now we have the same problem for other, more specific types of psychotherapy for depression.

The present study adds considerably to the existing evidence for a number of specific types of therapies for depression. For example, previous systematic reviews were not able to document the efficacy of specific types of psychotherapy for depression (i.e., ACT) for the treatment of depression, or the efficacy had, to the best of our knowledge, yet not been explored in recent previous meta-analyses (i.e., life review therapy). For several of the examined therapies, such as mindfulness-based CBT, acceptance and commitment therapy, and life-review it was not clear yet how these findings relate to other types of psychotherapy for depression. Our finding clearly indicates that these types of therapies may not only be effective in the treatment of depression, but also result in comparable effect sizes to more established treatments such as CBT.

We did not find any significant difference between the effect sizes of the generic therapies. This suggests that these therapies have comparable effects, although it should be noted that trials in which therapies are directly compared with each other give much stronger evidence on whether therapies are indeed equally effective (Barth et al., 2013; Cuijpers, van Straten, Andersson, et al., 2008). We also found no significant differences between the specific therapies, although there were some indications that the “Coping with Depression” course was less effective than other therapies. This may be a true differential effect (the “Coping with Depression” course differs from other therapies in that it is a psychoeducational treatment; Lewinsohn, Antonuccio, Steinmetz, & Teri, 1984), but it is also possible that the studies in which this course is used is conducted with more complicated target groups. This course is often used with difficult populations, such as alcoholics or juvenile delinquents with depression, because it can be easily adapted to different populations. In a separate meta-analysis of the “Coping with Depression” course that we conducted some time ago, we found that trials in which the course was directly compared with other interventions did not indicate significant differences with other therapies (Cuijpers, Munoz, Clarke, & Lewinsohn, 2009). This suggests that the course may not be less effective than other therapies. For now, however, we must assume that it does have smaller effects, based on the results of the current study. It was also, however, one of the two studies that remained significant after excluding studies with risk of bias and after adjustment for publication bias.

That most therapies seem to have comparable effects may be interpreted as indirect evidence that the therapies work through common, non-specific or universal mechanisms (Wampold & Imel, 2015). That is certainly a good possibility. However, it must be noted that there are other reasons that may explain these equal, comparable effects (Cuijpers, Reijnders, & Huibers, 2018). Comparable effects cannot be seen as evidence that the effects are realized by the same mechanisms. Depression is a complex disorder, with many different dimensions and characteristics. It is very well possible that a therapy changes one specific dimension or characteristic, which in turn changes and improves the whole system of depression-related characteristics of the patient. Furthermore, all characteristics of the patient, the therapist and the interaction between patient and therapist may be so complicated that therapies work through numerous different pathways, that all lead to improvement, but that may not be detected in meta-analyses because only averages across all studies and patients are examined.

One important issue is that we measured the effects of psychotherapies compared to control conditions, including waiting list, care-as-usual and other control conditions. These control conditions, however, differ considerably from each other (Gold et al., 2017; Mohr, Spring, Freedland, Beckner, Arean, Hollon et al., 2009). This introduces heterogeneity in our analyses, although after adjustment for control condition in the multivariate metaregression analyses, the results did not change. It should also be mentioned that comparing psychotherapies with control conditions can result in estimates of the effects of these therapies, but comparisons between therapies result in much stronger evidence for potential differences.

The results of this meta-analysis should be considered in the light of some important limitations. One important limitation is that the categories of the therapies we examined in this meta-analyses, are not always straightforward. Therapies may be based on specific manuals or methods, but are typically still adapted to the population or setting where it is used to a certain extent. “Pure” manuals are only seldom used. Making categories of therapies has therefore always some inherent uncertainties. This is made worse by the fact that authors often describe their treatments very briefly, often enclosed by word limits of the journals where the studies are published. Other limitations of this meta-analyses include the considerable risk of bias in the majority of studies, which we already discussed, and the problem of publication bias. We also only looked at short-term outcomes and not at longer-term outcomes, because they are usually not reported, or reported in widely differing follow-up periods, and because they are almost always naturalistic follow-up studies. Furthermore, the generalizability of our findings are limited to patients treated in an outpatient setting, because, due to considerable differences, we excluded studies focusing on inpatients. In our analyses we did not examine all potential moderators of outcome, although previous meta-analytic research has not indicated that for example treatment format (Cuijpers et al., 2019), number of sessions (Cuijpers, Huibers, Ebert, Koole, & Andersson, 2013) or characteristics of participants of psychotherapies (Cuijpers, Karyotaki, Reijnders, et al., 2018) are associated with outcome. Another limitation is that the number of studies for some categories was very small, and power may have been too low to find significant effects. We also did not register the agreements and disagreements between raters of the characteristics of the studies because this was done over a period of more than 10 years with every update of our dataset. Furthermore, we focused only on depressive symptoms as outcomes and did not focus on other outcomes, such as quality of life or functional limitations. We did not examine therapist effects either, which may have influenced the outcomes.

Despite these limitations, we can conclude that the 15 types of therapy that were examined in this meta-analysis may be effective in the treatment of depression, but the evidence is not conclusive because of high levels of heterogeneity, publication bias, and the risk of bias in the majority of studies.

Table I. Definitions of psychological treatments of depression.

Table II. Effects of 15 therapies compared with control groups: Hedges’ ga.

Table III. Effects of 15 therapies stratified across different types of control groups: Hedges’ ga.

Table IV. Standardized regression coefficients of generic and specific treatments of depression: multivariate metaregression analyses.

Table V. Effects of 15 therapies compared with control groups from randomized trials with low risk of bias: Hedges’ ga.

Download MS Word (152 KB)

Pim Cuijpers http://orcid.org/0000-0001-5497-2743

• Barth, J., Munder, T., Gerger, H., Nuesch, E., Trelle, S., Znoj, H., … Cuijpers, P. (2013). Comparative efficacy of seven psychotherapeutic interventions for patients with depression: A network meta-analysis. PLoS Medicine, 10(5), e1001454. doi: 10.1371/journal.pmed.1001454 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive therapy of depression (1st ed.). New York: The Guilford Press. [Google Scholar]
• Beck, A. T., Steer, R. A., & Brown, G. K. (1996). BDI-II. Beck depression inventory second edition: Manual. San Antonio: Psychological Corporation. [Google Scholar]
• Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4(6), 561–571. doi: 10.1001/archpsyc.1961.01710120031004 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Bohlmeijer, E., Smit, F., & Cuijpers, P. (2003). Effects of reminiscence and life review on late-life depression: A meta-analysis. International Journal of Geriatric Psychiatry, 18(12), 1088–1094. doi: 10.1002/gps.1018 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Borenstein, M., Hedges, L.V., Higgins, J.P.T., & Rothstein, H.R. (2009). Introduction to meta-analysis. Chichester: Wiley. [Crossref], [Google Scholar]
• Bowman, V., Ward, L. C., Bowman, D., & Scogin, F. (1996). Self-examination therapy as an adjunct treatment for depressive symptoms in substance abusing patients. Addictive Behaviors, 21(1), 129–133. doi: 10.1016/0306-4603(95)00027-5 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Burns, D. D. (1980). Feeling good: The new mood therapy (1st ed.). New York: Morrow. [Google Scholar]
• Butler, R. N. (1963). The life review: An interpretation of reminiscence in the aged. Psychiatry, 26(1), 65–76. [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
• Campbell LF, Norcross JC, Vasquez MJ, Kaslow NJ (2013). Recognition of psychotherapy effectiveness: The APA resolution. Psychotherapy, 50, 98–101. [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Churchill, R., Davies, P., Caldwell, D., Moore, T. H., Jones, H., Lewis, G., & Hunot, V. (2010). Interpersonal, cognitive analytic and other integrative therapies versus treatment as usual for depression. The Cochrane Library, 9, CD008703. doi: 10.1002/14651858.CD008703 [Crossref], [Google Scholar]
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. [Google Scholar]
• Cuijpers, P. (2017). Four decades of outcome research on psychotherapies for adult depression: An overview of a series of meta-analyses. Canadian Psychology, 58(1), 7–19. doi: 10.1037/cap0000096 [Crossref], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Clignet, F., van Meijel, B., van Straten, A., Li, J., & Andersson, G. (2011). Psychological treatment of depression in inpatients: A systematic review and meta-analysis. Clinical Psychology Review, 31, 353–360 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Cristea, I. A., Karyotaki, E., Reijnders, M., & Huibers, M. J. (2016). How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence. World Psychiatry, 15(3), 245–258. doi: 10.1002/wps.20346 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., de Wit, L., Kleiboer, A., Karyotaki, E., & Ebert, D. D. (2017). Problem-solving therapy for adult depression: An updated meta-analysis. European Psychiatry, published online. doi: 10.1016/j.eurpsy.2017.11.006 [Crossref], [PubMed], [Google Scholar]
• Cuijpers, P., Donker, T., Weissman, M. M., Ravitz, P., & Cristea, I. A. (2016). Interpersonal psychotherapy for mental health problems: A comprehensive meta-analysis. American Journal of Psychiatry, 173(7), 680–687. doi: 10.1176/appi.ajp.2015.15091141 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Driessen, E., Hollon, S. D., van Oppen, P., Barth, J., & Andersson, G. (2012). The efficacy of non-directive supportive therapy for adult depression: A meta-analysis. Clinical Psychology Review, 32(4), 280–291. doi: 10.1016/j.cpr.2012.01.003 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Huibers, M., Ebert, D.D., Koole, S.L., & Andersson, G. (2013). How much psychotherapy is needed to treat depression? A metaregression analysis. Journal of Affective Disorders, 149, 1–13. [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Karyotaki, E., Reijnders, M., & Huibers, M.J.H. (2018). Who benefits from psychotherapies for adult depression? A meta-analytic update of the evidence. Cognitive Behaviour Therapy, 47, 91–106. [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
• Cuijpers, P, Karyotaki, E, Reijnders, M, Purgato, M, & Barbui, C. (2018). Psychotherapies for depression in low-and middle-income countries: A meta-analysis. World Psychiatry, 17, 90–101. [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Munoz, R. F., Clarke, G. N., & Lewinsohn, P. M. (2009). Psychoeducational treatment and prevention of depression: The ‘Coping with Depression’ course thirty years later. Clinical Psychology Review, 29(5), 449–458. doi: 10.1016/j.cpr.2009.04.005 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Noma, H., Karyotaki, E., Cipriani, A., & Furukawa, T.A. (2019). Individual, group, telephone, self-help and internet-based cognitive behavior therapy for adult depression; A network meta-analysis of delivery methods. JAMA Psychiatry, in press. [Crossref], [Google Scholar]
• Cuijpers, P., Reijnders, M., & Huibers, M. (2018). The role of common factors in psychotherapy outcome. Annual Review of Clinical Psychology, in press. [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Reijnders, M., & Karyotaki, E. (2018). Psychological treatment of depression: An updated meta-analytic database of randomized trials. Submitted for publication. [Google Scholar]
• Cuijpers, P., Sijbrandij, M., Koole, S. L., Andersson, G., Beekman, A. T., & Reynolds, C. F. (2013). The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: A meta-analysis of direct comparisons. World Psychiatry, 12(2), 137–148. doi: 10.1002/wps.20038 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., Turner, E. H., Koole, S. L., van Dijke, A., & Smit, F. (2014). What is the threshold for a clinically relevant effect? The case of major depressive disorders. Depression and Anxiety, 31(5), 374–378. doi: 10.1002/da.22249 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2008). Psychotherapy for depression in adults: A meta-analysis of comparative outcome studies. Journal of Consulting and Clinical Psychology, 76(6), 909–922. doi: 10.1037/a0013075 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). The effects of psychotherapy for adult depression are overestimated: A meta-analysis of study quality and effect size. Psychological Medicine, 40(2), 211–223. doi: 10.1017/S0033291709006114 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., van Straten, A., Smit, F., & Andersson, G. (2009). Is psychotherapy for depression equally effective in younger and older adults? A meta-regression analysis. International Psychogeriatrics, 21(1), 16–24. doi: 10.1017/S1041610208008089 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Cuijpers, P., van Straten, A., Warmerdam, L., & Andersson, G. (2008). Psychological treatment of depression: A meta-analytic database of randomized studies. BMC Psychiatry, 8(1), 36. [Crossref], [PubMed], [Google Scholar]
• Dimidjian, S., Barrera Jr, M., Martell, C., Muñoz, R. F., & Lewinsohn, P. M. (2011). The origins and current status of behavioral activation treatments for depression. Annual Review of Clinical Psychology, 7(1), 1–38. doi: 10.1146/annurev-clinpsy-032210-104535 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Driessen, E., Hollon, S. D., Bockting, C. L. H., Cuijpers, P., & Turner, E. H. (2015). Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of us national institutes of health-funded trials. PloS One, 10(9), e0137864. doi: 10.1371/journal.pone.0137864 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Driessen, E., Van, H. L., Don, F. J., Peen, J., Kool, S., Westra, D., … Dekker, J. J. M. (2013). The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatment of major depression: A randomized clinical trial. The American Journal of Psychiatry, 170(9), 1041–1050. doi: 10.1176/appi.ajp.2013.12070899 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. doi: 10.1111/j.0006-341X.2000.00455.x [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Ekers, D., Richards, D., & Gilbody, S. (2008). A meta-analysis of randomized trials of behavioural treatment of depression. Psychological Medicine, 38(05), 611–623. doi: 10.1017/S0033291707001614 [Crossref], [PubMed], [Google Scholar]
• Furukawa, T. A. (1999). From effect size into number needed to treat. The Lancet, 353(9165), 1680. doi: 10.1016/S0140-6736(99)01163-0 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Furukawa, T. A., Noma, H., Caldwell, D. M., Honyashiki, M., Shinohara, K., Imai, H., … Churchill, R. (2014). Waiting list may be a nocebo condition in psychotherapy trials: A contribution from network meta-analysis. Acta Psychiatrica Scandinavica, 130(3), 181–192. doi: 10.1111/acps.12275 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Gold, S.M., Enck, P., Hasselmann, H., Friede, T., Hegerl, U., Mohr, D.C. & Otte, C. (2017). Control conditions for randomised trials of behavioural interventions in psychiatry: A decision framework. The Lancet Psychiatry, 4, 725–732. [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Hamilton, M. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery, and Psychiatry, 23(1), 56–62. [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (1999). Acceptance and commitment therapy: An experiential approach to behavior change. New York: Guilford Press. [Google Scholar]
• Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press. [Crossref], [Google Scholar]
• Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., … Sterne, J. A. C. (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ, 343, d5928. doi: 10.1136/bmj.d5928 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ (Clinical Research Ed.), 327(7414), 557–560. doi: 10.1136/bmj.327.7414.557 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007). Uncertainty in heterogeneity estimates in meta-analyses. BMJ, 335(7626), 914–916. doi: 10.1136/bmj.39343.408449.80 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Jacobson, N. S., Martell, C. R., & Dimidjian, S. (2001). Behavioral activation treatment for depression: Returning to contextual roots. Clinical Psychology: Science and Practice, 8(3), 255–270. doi: 10.1093/clipsy.8.3.255 [Crossref], [Web of Science ®], [Google Scholar]
• Kahl, K. G., Winter, L., & Schweiger, U. (2012). The third wave of cognitive behavioural therapies: What is new and what is effective? Current Opinion in Psychiatry, 25(6), 522–528. doi: 10.1097/YCO.0b013e328358e531 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Klerman, G. L., Weissman, M. M., Rounsaville, B. J., & Chevron, E. S. (1984). Interpersonal psychotherapy of depression: A brief, focused, specific strategy (1st ed.). Northvale, N.J: Jason Aronson, Incorporated. [Google Scholar]
• Lewinsohn, P. M. (1974). A behavioral approach to depression. In J. C. Coyle (Ed.) Essential papers on depression (pp.150–172). New York: New York University Press. [Google Scholar]
• Lewinsohn, P. M., Antonuccio, D. O., Steinmetz, J., & Teri, L. (1984). The coping with depression course: A psycho-educational intervention for unipolar depression. Eugene: Castalia Publishing Company. [Google Scholar]
• Malouff, J. M., Thorsteinsson, E. B., & Schutte, N. S. (2007). The efficacy of problem solving therapy in reducing mental and physical health problems: A meta-analysis. Clinical Psychology Review, 27(1), 46–57. doi: 10.1016/j.cpr.2005.12.005 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Martell, C. R., Addis, M. E., & Jacobson, N. S. (2001). Depression in context: Strategies for guided action. New York: WW Norton & Co. [Google Scholar]
• Mazzucchelli, T., Kane, R., & Rees, C. (2009). Behavioral activation treatments for depression in adults: A meta-analysis and review. Clinical Psychology: Science and Practice, 16(4), 383–411. doi: 10.1111/j.1468-2850.2009.01178.x [Crossref], [Web of Science ®], [Google Scholar]
• Mohr, D. C., Ho, J., Hart, T. L., Baron, K. G., Berendsen, M., Beckner, V., … Duffecy, J. (2014). Control condition design and implementation features in controlled trials: A meta-analysis of trials evaluating psychotherapy for depression. Translational Behavioral Medicine, 4(4), 407–423. doi: 10.1007/s13142-014-0262-3 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Mohr, DC, Spring, B, Freedland, KE, Beckner, V, Arean, P, Hollon, SD, … Ockene, J. (2009). The Selection and Design of Control Conditions for Randomized Controlled Trials of Psychological Interventions. Psychotherapy and Psychosomatics, 78, 275–284. [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Mynors-Wallis, L. M., Gath, D. H., Day, A., & Baker, F. (2000). Randomised controlled trial of problem solving treatment, antidepressant medication, and combined treatment for major depression in primary care. BMJ, 320, 26–30. doi: 10.1136/bmj.320.7226.26 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Mynors-Wallis, L. M., Gath, D. H., Lloyd-Thomas, A. R., & Tomlinson, D. (1995). Randomised controlled trial comparing problem solving treatment with amitriptyline and placebo for major depression in primary care. BMJ, 310(6977), 441–445. doi: 10.1136/bmj.310.6977.441 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Nezu, A. M. (1986). Efficacy of a social problem-solving therapy approach for unipolar depression. Journal of Consulting and Clinical Psychology, 54(2), 196–202. doi: 10.1037/0022-006X.54.2.196 [Crossref], [PubMed], [Web of Science ®], [Google Scholar]
• Nezu, A. M., & D’Zurilla, T. J. (1979). An experimental evaluation of the decision-making process in social problem solving. Cognitive Therapy and Research, 3(3), 269–277. doi: 10.1007/BF01185967 [Crossref], [Web of Science ®], [Google Scholar]
• Orsini, N., Bottai, M., Higgins, J., & Buchan, I. (2006). Heterogi: Stata module to quantify heterogeneity in a meta-analysis. Statistical Software Components. Retrieved from https://ideas.repec.org/c/boc/bocode/s449201.html [Google Scholar]
• Segal, Z. V., Williams, J. M. G., & Teasdale, J. D. (2002). Mindfulness-based cognitive therapy for depression: A new approach to relapse prevention. New York: Guilford Press. [Google Scholar]
• Shinohara, K., Honyashiki, M., Imai, H., Hunot, V., Caldwell, D. M., Davies, P., … Churchill, R. (2013). Behavioural therapies versus other psychological therapies for depression. The Cochrane Library, 16(10), CD008696. doi: 10.1002/14651858.CD008696 [Crossref], [Google Scholar]
• Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). New York: Routledge. [Crossref], [Google Scholar]
• Weissman, M. M., Hankerson, S. H., Scorza, P., Olfson, M., Verdeli, H., Shea, S., … Wainberg, M. (2014). Interpersonal counseling (IPC) for depression in primary care. American Journal of Psychotherapy, 68(4), 359–383. [Crossref], [PubMed], [Google Scholar]