A systematic review and meta-analysis of trans-diagnostic third-wave online psychological interventions

Revisión sistemática y metaanálisis de intervenciones transdiagnósticas en línea de tercera generación

Esta obra está bajo una Licencia Creative Commons Atribución-NoComercial-CompartirIgual 4.0 Internacional.

Evidence-based therapies propose the diversification and flexibilization of delivery modes for psychological interventions (Kazdin, 2014). Tele-health strategies consist of using information and communication technologies (ICT) to evaluate, diagnose, treat, and follow-up on health conditions (Barak et al., 2008). Said strategies have been gradually gaining legitimacy and support from users and organizations as alternatives to traditional face to face intervention (Kruse et al., 2017; Shigekawa et al., 2018). In the wake of the global COVID-19 pandemic, and with most countries over the world under preemptive self-quarantine and social-distancing measures, tele-health strategies have become a powerful tool for responding to the patient’s healthcare needs in most high and middle-income counties becoming a part of the new normality (Hollander & Carr, 2020; Monaghesh & Hajizadeh, 2020; Ohannessian et al., 2020; Smith et al., 2020).

“Tele-psychology” (APA, 2013; Bryant et al., 2020; Perle et al., 2011) is an umbrella term that includes: self-help web-based interventions, synchronic communication involving virtual therapy and counseling through video-call or instant messages, and automatic online softwares (Barak et al., 2009; Wangelin et al., 2016). Online psychological interventions or OPIs (Peñate et al., 2014; Perle et al., 2011; Sierra et al., 2018), on the other hand, refer specifically to asynchronic self-help programs or services often inspired by existing evidence-based interventions, which represent a promising and flexible alternative to not only traditional psychotherapy but also synchronic tele-psychology approaches. Most OPIs feature psychoeducation regarding the target diagnosis, some sort of evaluation process, and explicit intervention components (Andersson & Cuijpers, 2009; Nelson et al., 2011; Peñate et al., 2014; Titov, 2007).

Regarding mental health treatment and prevention under the global COVID-19 pandemic, most countries have found information and communication technologies quite useful to grant the people access to the demanded services (Reglitz & Rudnick, 2020). However, even high-income countries have struggled with the creation and implementation of alternatives to synchronic virtual visits (Duan & Zhu, 2020; Monaghesh & Hajizadeh, 2020). Considering that mental health is often seen as a human right in the 21^st century (Barrera-Rojas & Baeza-Ruiz, 2021; the development of OPI strategies is a fundamental step towards mitigating the symptoms aroused or worsened by the pandemic. These focus on promoting resilience and well-being (Reyes-Rojas et al., 2021) and including everyone in the community (Duan & Zhu, 2020; Zhou, 2020): COVID-19 positive patients (Kimhi et al., 2020), but also families (Cusinato et al., 2020; Davidson et al., 2020), healthcare personnel (Coulombe et al., 2020; Huffman et al., 2021) and people among the general population (Gloster et al., 2020;Ivbijaro et al., 2020; Paredes et al., 2021) coping with their particular aversive experiences related to the pandemic.

The third wave of cognitive-behavioral therapies are a diverse group of evidence-based psychological interventions that tend to focus on the behavioral functions of private experiences rather than their form, frequency, or intensity. Third-wave includes: Acceptance and Commitment Therapy (ACT;Hayes et al., 1999), Functional Analytic Psychotherapy (FAP; Kohlenberg & Tsai, 1991), Behavioral Activation (BA; Martell et al., 2001), Dialectical Behavioral Therapy (DBT; Linehan, 1993), Metacognitive Therapy (MCT; Wells, 2009), Mindfulness-Based Cognitive Therapy (MBCT; Segal et al., 2002), and Rumination-Focused Cognitive-Behavioral Therapy (RF-CBT; Watkins, 2016). In spite of their conceptual and methodological differences, third-wave therapies share three common goals: promoting the client’s openness towards their private experiences, improving their awareness of their own behavior, and reaching valued outcomes in the presence of unpleasant private experiences (Hayes et al., 2011).

General reviews on the efficacy of OPIs typically feature randomized controlled trials (RCTs) based on cognitive behavioral therapy (CBT). Results suggest that these interventions are effective for the treatment of depressive and anxiety disorders with effect sizes being moderated by variables such as duration, support, and baseline symptom levels (Andersson & Cuijpers, 2009; Andrews et al., 2010; Newman et al., 2011; Richards & Richardson, 2012; Spek et al., 2007; Wangelin et al., 2016). On the other hand, the existing meta-analyses assessing the effect of third-wave OPIs are symptom based (i. e. focused on the treatment of a specific set of disorders or symptoms); according to their results, medium effect sizes are expected for the treatment of depression (O’Connor et al., 2018; Sierra et al., 2018) and overall small to medium effect sizes for the treatment of anxiety (Kelson et al., 2019;O’Connor et al, 2018).

There is a very limited number of explicitly trans-diagnostic third-wave OPIs; therefore, reviews are very scarce. The reported variability among the kinds of interventions and disorders included in the existing reviews (Linardon, 2019;Sevilla-Llewellyn-Jones et al., 2018, Thompson et al., 2021) complicates the assessment of their current empirical evidence promoting mental health. In response to the latter, the present review/meta-analysis aims to analyze the current empirical evidence of explicitly trans-diagnostic third-wave OPIs that aim to decrease psychological distress or symptomatology and promote psychological flexibility within an RCT logic among non-specific samples. Said analyses might help towards the development and implementation of cost-efficient evidence-based programs that allow at least a slight mitigation of the existing lack of mental health coverage worldwide.

The database search yielded 1 408 results, with no duplicate studies found as only Scopus was used for the search; all the results were written in English. The titles and abstracts of studies found were screened for full-text review, which excluded 1 387 (Figure 1). Among the 21 studies kept for full-text review, 15 were excluded for diverse reasons: five did not feature a third-wave OPI (Anstiss & Davies, 2015; Farrer et al., 2019; Simpson et al., 2015; Weisel et al., 2018; Weisel et al., 2019), three were actually study protocols and did not present outcome data (González-Robles et al., 2015; Rahmadiana et al., 2019; Witlox et al., 2018), seven of them did not feature a valid control comparison as their experimental designs were pre-post or open trial (Economides et al., 2019; Finlay-Jones et al., 2016; Firestone et al., 2019; Kladnitski et al., 2018; Krüsche et al., 2013; Levin et al., 2017; Silva-Almodovar et al., 2018). Also, two studies came to the author’s attention after the search, but neither was included due to exclusion criteria (Haeger et al., 2020; Riva et al., 2020). Finally, six studies were included for data analysis (Flett et al., 2020; Kladnitski et al., 2020; Levin et al., 2013; Levin et al., 2016; Levin et al., 2020;Viskovich & Pakenham, 2019) their main characteristics are summarized in Table 1. Nine separate third-wave OPI vs. control comparisons were identified within the included studies and included as individual interventions for data analysis.

Table 1
Main characteristics of the reviewed studies

ACT = Acceptance and Commitment TherapyDASS-21= Depression Anxiety and Stress Scale

Risk of bias

In order to assess risk of bias, all relevant methodological aspects that could represent a threat to the validity of the included studies were evaluated, and a statistically significant agreement between the studies assessments ranging from V = 1 (95 % CI [.80, 1.00]) to V = .83 (95 % CI [.55, .95]) was found. However, the study with the most threats identified was Levin et al. (2020) (V = .50 (95 % CI [.25, .74]) which had a relatively low sample size (< 234), did not conduct ITT analysis and did not explicitly report that there were no statistically significant differences between the groups at pretest. These limitations are expectable and justified because the study focuses on the analysis of the change processes involved rather than the existence of the effect. Results suggest that all the studies had a low risk of compromising the results validity. Sensitivity analyses were attempted but, due to the latter, categorization of the studies according to a cutoff point set according to the total quality score (range: 1-7; median: 4) assigned all of the studies under the low-risk category and thus no further sensitivity analysis (e. g. meta-regression) was either pertinent or needed.

According to heterogeneity analysis for general distress at post-treatment (Q = 217.40, df = 8, p = .00; t² = .89; I² = 96.32) and last follow-up (Q = 203.12, df = 4, p = .00; I² = 98.03), the observed variability was significantly due to between-study variance and a large proportion of that variability can be attributed to true variations on effect sizes (I2 = 98.03). For psychological flexibility at post-treatment (Q = 167.70, df = 5, p = .00; t² = 1.07; I² = 97.01) and last follow-up (Q = 92.68, df = 3, p = .00; t² = 2.07; I² = 96.07) the observed variability was also significantly produced by between-study variance (Q = 167.70, df = 5, p = .00; t² = 1.07) and a large proportion of it was still attributable to true variations among effect sizes (I² = 96.07).

The latter is expected and explained by the nature of the studies and the relative variability within the included OPIs; homogeneity across them would actually be very hard to achieve unless the study used far more restrictive inclusion and exclusion criteria to narrow the analyzable studies rendering meta-analysis even more difficult to perform. On the other hand, the quality assessment conducted to identify threats to validity among the included studies suggest a low risk of bias related to the inclusion of the studies. Finally, the qualitative similarities among the included studies are robust and justified enough to consider that there are no methodological issues in comparing and synthesizing their evidence.

Meta-analysis for General Distress

General effect sizes and meta-analytic data for general distress at post-treatment are presented in table 2, significant and very large effect sizes were found for the random effects model (g = 1.30, p = .00; CI 95 % = .66, 1.94). Duration and mean age were not significant as mediators of the posttreatment effect according to the conducted meta-regressions. However, the mixed effects meta-regression model for attrition rate was significant (Q_model= 12.12, df = 1, p = .00, Q_residual = 205.27, df = 7, p = .00; t. = .73), implying that higher levels of attrition predict smaller effect sizes for general distress at posttreatment and significantly explaining heterogeneity among the sample.

Table 2
Meta-analysis of general distress at post-treatment and LFUP

On the other hand, effect sizes and meta-analytic data for general distress at the last follow-up are also presented in Table 2 showing significantly huge effect sizes for the random effects model (g = 2.4, p = .00; CI 95% = .66, 4.13). Duration, and mean age were not significant as mediators of the effect for the last follow-up. However, the mixed effects meta-regression model for attrition rate was significant (Q_model= 20.79, df = 1, p = .00, Q_residual = 2.26, df = 3, p = .51; t² = .73), implying that higher levels of attrition predict smaller effect sizes for general distress at the last follow-up with non-significant unexplained heterogeneity.

Meta-analysis for Psychological Flexibility

For psychological flexibility, general effect sizes and meta-analytic data at posttreatment are featured in Table 3; significant and very large effect sizes were found for the random effects model (g = 1.23, p = .004; CI 95% = .38, 2.84). According to the method of moments meta-regression models: duration (Q_model= 87.01, df = 1, p = .00, Q_residual = 80.68, df = 4, p = .00; t² = .58), mean age (Q_model = 27.12, df = 1, p = .00, Q_residual = 140.68, df = 4, p = .00; t² = 1.74) and attrition rate (Q_model = 41.12, df = 1, p = .00, Q_residual = 126.57, df = 4, p = .00; t² = 1.46). This implies that longer interventions predict larger effect sizes for psychological flexibility at posttreatment and especially for younger patients. As seen with general distress, large attrition rates predict smaller effect sizes and significantly explain heterogeneity between studies.

Table 3
Meta-analysis of psychological flexibility at post-treatment and LFUP

ACT = Acceptance and Commitment Therapy; ACT-E = ACT-Engaged; ACT- F= ACT-Full; ACT-O = ACT-Open; LFUP = Last Follow-up; MHW = Mental Health Website; WLC = Waitlist Control.

Finally, effect sizes and meta-analytic data for psychological flexibility at the last follow-up are also presented in Table 3, with significant and very large effect sizes for the random effects model (g = 1.81, p = .01; CI 95 % = .37, 3.26). Duration, attrition rate and mean age were not significant as mediators of the psychological flexibility effect for the last follow-up.

Overall, the results suggest that third-wave trans-diagnostic OPIs are effective to decrease general distress and promote psychological flexibility. The quality assessment and heterogeneity analyses conducted suggest there is a low risk of bias among the included studies. As expected, and according to the random effects model assumptions, high levels of variability were observed among the OPIs variance and effect sizes for both variables and large proportions of said variability were attributable to the real and addressed differences between them. The latter would mean that the identified factors significantly explain the existing heterogeneity providing a statistical fundament that allows to regard the calculated summary effects as valid.

Meta-regression analyses are helpful for understanding the results. The fact that higher attrition rates significantly predicted smaller effect sizes on general distress at the last follow-up explains the high variability in effect sizes among the OPIs as the inclusion of studies with such attrition rates and no significant standalone effect (Flett et al., 2020; Levin et al., 2016) is necessarily going to affect the summary effect sizes obtained in meta-analysis, suggesting that further efforts are needed to guarantee adherence. On the other hand, the fact that the OPIs duration in weeks significantly predicted larger effect sizes for psychological flexibility at posttreatment is very interesting from a theoretical standpoint, as ACT understands psychological flexibility as a skill that needs to be promoted continuously by repeated practice (Hayes et al., 1999; Hayes et al., 2011). Finally, finding that mean age significantly and inversely predicts larger effect sizes suggests that promoting the mentioned ability is easier in younger participants (Greco et al., 2008; Hayes et al., 1999).

The latter supports some of the findings on the general OPI reviews such as the effect moderations by duration (Andersson & Cuijpers, 2009; Andrews et al., 2010; Newman et al., 2011; Richards & Richardson, 2012; Spek et al., 2007; Wangelin et al., 2016). The current results are also coherent with the findings of existing literature that reported small to medium effect sizes for the treatment of depression and anxiety using third-wave OPIs (Kelson et al., 2019; O’Connor et al., 2018; Sierra et al., 2018). However, simple comparisons of meta-analytic data among the reviews suggest that trans-diagnostic OPIs are overall more effective than diagnosis specific OPIs. Further research regarding this matter is highly encouraged, not only for the case of OPIs but also for clinical practice and research among evidence-based psychotherapies.

The present study has some methodological limitations related to several of the decisions taken: (1) although searching only on a single database (Scopus) eliminates the appearance of duplicate studies and ensures some degree of methodological quality among the included studies, it narrows the pool of possible includable studies; (2) the lack of a thesaurus or mesh terms equivalent to the ones the current study used on the search string certainly could limit the emergence of relevant studies; (3) although only studies following the logical guidelines of RCT were included, an RCT design was not considered mandatory inclusion criteria, which might impact the validity of the conclusions; (4) pretest vs. posttest designs without control groups were excluded as their data is not necessarily meta-analyzable in the same way and its inclusion could represent further deviations from the existing the meta-analytic guidelines; and (5) the inability to perform publication bias and certainty analyses and the subsequent effect size corrections due to the number of reviewed studies could also compromise the validity of the findings.

However, considering that tele-psychology is still an emerging field, the current study aimed to maximize analyzable data in order to perform the featured analysis and their subsequent interpretations. Further research is needed and, when more studies exist, future reviews are encouraged to address the previous limitations. New meta-analyses to assess the effect of OPIs explicitly designed for the treatment of anxiety, depression, stress, or chronic pain are strongly suggested as most of the existing reviews on third-wave OPIs include too many variables that make the comprehension of meta-analytic data from a theoretical standpoint slightly challenging.

In conclusion, OPIs continue to provide evidence of their utility as legitimate self-help alternatives to traditional psychotherapy; their expectable effect sizes are overall large for decreasing psychological distress or symptoms and promoting psychological flexibility. However, considering that the law of attrition (Eysenbach, 2005) applies to most research on the tele-psychology field, it’s necessary to make better efforts to mitigate drop-out rates, such as offering customized feedback, the possibility to prolong their participation until completion or any other kind of benefit that motivates the participant to finish, and, when this not possible, embracing the intention-to treat (ITT) approach to data-analysis as most of the reviewed studies. As the evidence-based therapies suggest, the only way to make a real difference on the existing global lack of mental health coverage and access is to be open and embrace flexible alternatives that help mitigate this problem (Kazdin, 2014). The latter is critical in order to respond at least partially to the current mental health situations that the global pandemic has arisen (Gloster et al., 2020; Ivbijaro et al., 2020; Monaghesh & Hajizadeh, 2020; Paredes et al., 2021). Therefore, this is an open invitation to embrace OPIs and the world of possibilities they offer in service of human wellbeing.