Four-Year Cost-effectiveness of Cognitive Behavior Therapy for Preventing First-episode Psychosis: The Dutch Early Detection Intervention Evaluation (EDIE-NL) Trial

Design and Participants

The participants were assessed 4 years after inclusion, comparing CBTuhr with RC. The study was approved by the Medical Ethics Committee and registered at Current Controlled Trials (ISRCTN21353122). Inclusion criteria for EDIE-NL were: (1) age 14–35 years, (2) a family history of psychosis or Comprehensive Assessment of At-Risk Mental States (CAARMS)² scores in the range of the at-risk mental state (ARMS), and (3) impaired social functioning (a score on the Social and Occupational Functioning Assessment Scale [SOFAS]¹² of ≤54 or a reduction of 30% on the SOFAS for the duration of at least 1 month in the past year). Exclusion criteria for EDIE-NL were: (1) presence of a current or past psychotic disorder, (2) insufficient command of the Dutch language, (3) severe learning impairment, and (4) current or previous use of antipsychotic medication with a cumulative dose of ≥15mg haloperidol equivalents.

Participants were contacted 4 years after baseline assessment (between June 2012 and January 2014). The mean follow-up time was a little longer than 4 years (mean: 1520 days; SD: 170.87). The interview consisted of assessments of psychopathology and took place at local outpatient services. If participants did not consent to a face-to-face interview, they were asked whether they would consent to a brief telephone assessment, such that a minimal set of clinical and functional outcome data could be collected.

Figure 1 presents the flow of the participants through the trial. Of the 201 patients included in the original trial, 5 participants were included incorrectly at baseline and were excluded from all further analyses. The final sample consisted of 196 participants: 101 in the RC condition and 95 in the CBTuhr condition. At the 18-month follow-up measurement, 140 participants (71.4%) participated, and at 4 years, 113 patients (57.7%) had a complete follow-up assessment. Of the 83 dropouts, 9 (10.8%) had already transitioned to psychosis during the first 18 months of the study, and for 16 participants (19.3%), it could be verified that they were not diagnosed with a Diagnostic and Statistical Manual of Mental Disorders (DSM) psychotic disorder 4 years post-baseline. Verification was obtained by comparing the electronic patient files with their medical history.

Interventions

Participants in both treatment arms were treated with RC as provided for the nonpsychotic DSM-IV Axis 1 or Axis 2 disorders for which they were seeking treatment. RC was given according to the evidence-based clinical Dutch¹³ and the National Institute for Health and Care Excellence¹⁴ guidelines. The experimental group received RC plus CBTuhr,¹⁵ with a maximum of 26 sessions in the first 6 months after inclusion,⁹ with the aim to prevent first-episode psychosis. The mean (and median) number of sessions was 10 (95% CI 8–12), partly caused by early completers, dropouts, or early transitions. The therapists were all experienced in CBT for psychosis and were trained in the use of the protocol during a day and supervised once a month in a group. For further information, see the Results section of the article.⁹

Primary Outcome

The primary clinical outcome was the number of transitions to psychosis averted. This was calculated as the proportion of participants who did not develop a first-episode psychosis within 4 years post-baseline as assessed with the CAARMS.^2,3 The CAARMS is a semi-structured interview used to rate the intensity and frequency of subclinical psychotic symptoms and discriminates between psychosis, UHR, or neither. The CAARMS was repeatedly administered at baseline and at 2, 4, 6, 9, 12, 15, 18, and 48 months to detect transition to psychosis. In the case of a transition to psychosis, according to the CAARMS, the Dutch version of the Schedules for Clinical Assessment in Neuropsychiatry (SCAN 2.1)¹⁶ was used to diagnose participants. Details are reported elsewhere.¹¹

Secondary Outcome

The secondary outcome was health-related quality of life, operationalized by quality-adjusted life years (QALYs), based on the EQ-5D 3-level version of the ‘EuroQoL’ group (EQ-5D-3L).¹⁷ The EQ-5D-3L consists of 5 health state dimensions (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression) on which the respondent has to rate his/her own health. Each dimension has 3 levels: no problems, some problems, and extreme problems; therefore, the EQ-5D-3L can describe as many as 3⁵ = 243 distinct health states. We used the UK tariff to value health states.¹⁸ QALYs were computed as the amount of time spent in a health state weighted by the corresponding utility U. There are 4 time intervals (0–6 months, 6–12 months, 12–18 months, and 18–48 months) between the measurements, and we applied the mean area under the curve (AUC) method¹⁹ as: QALY = (t2−t1)×(U1=U2)/2+(t3−t2)× (U2+U3)/2+(t4−t3)×(U3+U4)/2+(t5−t4)×(U4+U5)/2, where the first 3 time intervals equal 6 months (ie, 0.5 of a year) and the last interval equals 30 months (5×6 months). An advantage of the QALY is that it facilitates comparison with other interventions in different diseases.²⁰

Other Measures

At 4-year follow-up, the SOFAS¹² was used to assess overall functioning. Symptoms were assessed with the Beck Depression Inventory II (BDI-II),²¹ the Calgary Depression Scale (CDS),²² and The Social Interaction Anxiety Scale (SIAS).²³ The Personal Beliefs About Illness Questionnaire (PBIQ-R)²⁴ assessed the appraisal of their illness, while the Manchester Short Assessment of Quality of Life (MANSA)²⁵ measured perceived quality of life.

Measures of Service Use and Costing

In the base case, costs are defined from the mental health care system perspective and encompass (1) intervention costs, (2) costs related to psychiatric health care uptake (Supplementary Table S1) including the costs of medication, and (3) participants’ travel costs for obtaining health services. Costs were originally expressed in euro for the reference year 2014 on a per-participant basis over the full period of 4 years. The costs were then converted to US dollars using the purchasing power parities from the Organization for Economic Co-operation and Development, taking into account both the exchange rate and the differential buying power across countries. For the reference year 2014, US$1.00 equated NL€0.825046. The AUC method was applied for computing cumulative QALY gains and costs.¹⁹

The cost data were collected using the Trimbos Institute and Institute of Medical Technology Assessment Questionnaire for Costs associated with Psychiatric Illness (TiC-P).²⁶ Because the sample included adolescents and young adults who made their first appearance in the health care system (ie, without a significant history of health care use), no baseline cost data were collected.

Health service use was valued by multiplying the units of health service (eg, visits, sessions, and hospital days) by their standard full economic cost price (Supplementary Table S1). To these we added the costs of antipsychotic medication according to the Dutch Health Care Insurance Board,²⁷ calculated as the cost price per standard daily dose as reported in the Pharmaceutical Compass,²⁷ multiplied by the number of prescription days, plus the pharmacist’s dispensing costs of US$8.59 per monthly prescription.²⁸ The costs of the intervention were calculated by multiplying the number of sessions by the standard full economic cost prices for a session with a psychologist or psychiatrist (Supplementary Table S1). Direct nonmedical costs (ie, the patient’s out-of-pocket costs) consisted of costs for traveling to the health services. Travel costs were computed as the average distance to a mental health service (7 km) multiplied by the costs per kilometre (US$0.28), as most participants used public transport.²⁸

Statistical Analysis

The data were analyzed in agreement with the intention-to-treat principle. For the main analysis, missing data in cost and outcomes (transitions and QALY gains) at the 4-year follow-up were imputed using Little & Rubin’s expectation-maximization (EM) algorithm²⁹ (as implemented in SPSS 23.0.0) with predictors both of the outcome and the missingness pattern. The first set of predictors are used to increase the precision of the imputed variable; the second to account for selective dropout and to meet the missing at random assumption underlying the EM algorithm. Condition (experimental vs. control) was also included as one of the predictors. Missing cost data (n = 83) were imputed with condition, age, years of education, psychopathology scores, transition psychosis (y/n), and health care costs as observed in the first 18 months. In the main analysis, missing clinical outcome on the CAARMS at 4-year follow-up (n = 58) was imputed with baseline age, gender, ethnicity (Dutch/non-Dutch origin), baseline CAARMS distress positive symptoms, and BDI scores as predictors. Missing QALYs (n = 83) were estimated with condition, gender, QALYs, psychopathology total scores of the different time points as predictor variables. All predictor variables were identified as significant predictors of the imputed variable by (logistic) regression analyses.

Testing Differential Effectiveness

Differences between the treatment arms in 4-year transition rates to psychosis were estimated under a linear probability model. Incremental effects with regard to QALY health gains were tested in a regression model. These analyses were conducted with STATA 13.1 using robust standard errors that were based on the first-order Taylor-series linearization method to account for clustering in the data. The study was conducted as a multisite trial.

Economic Evaluation

The economic evaluation consisted of a CEA and a cost-utility analysis (CUA). For both analyses, the incremental cost-effectiveness ratio (ICER) was calculated as (C₁ − C₀)/(E₁ − E₀), where C is the average per-participant cost, E is the effect, and subscripts 1 and 0 refer to the CBTuhr plus RC and the RC alone condition, respectively. The ICER represents the extra costs needed (or saved) per additionally averted transition to psychosis. The cost-utility ratio does the same per additional QALY gained. To handle uncertainty in the cost and effect data, nonparametric bootstrapping was conducted in Microsoft Excel to simulate 2500 ICERs.^20,30,31 The simulated ICERs can be presented as a scatter over a cost-effectiveness plane, with differences in costs on the vertical axis and differences in effects on the horizontal axis. If the ICER appears in the northwest (NW) quadrant of the plane, less effect is obtained for additional costs. If the ICER appears in the southeast (SE) quadrant, more health gains are obtained for fewer costs; the intervention is then preferred over standard care. The distribution of the simulated ICERs over the different quadrants can be used to determine the probability that the intervention is cost-effective or the probability that the intervention is both cost saving and more effective. The probability that the intervention is cost-effective for different willingness-to-pay threshold values is visualized using the cost-effectiveness acceptability curve.³²

Of note, this study was not powered to test differences between the treatment arms with regard to QALYs but was designed and powered to evaluate transitions to psychosis as the primary outcome. For this reason, the CEA is the main analysis, and the CUA is only added as a secondary analysis. The CUA is therefore restricted to reporting probabilistic outcomes within the medical decision-making framework, as is common when conducting health-economic evaluations. This is done to inform decisions under uncertainty.³³

Sensitivity Analyses

In the main analysis, EM imputation was used to handle missing data. Due to the high attrition rate, we repeated all analyses using last observation carried forward (LOCF) imputation, as this method is likely to result in wider CIs and is thus informative with regard to how probabilistic results are affected by such choices. In addition to the health care perspective adopted in the main analysis, outcomes were assessed using a societal perspective, thereby including the economic benefits (and losses) of increased (decreased) productivity. Productivity losses and gains in paid work were calculated according to the human capital approach³⁴ and reflected the changes in the number of hours worked per week. These were valued with the average age- and gender-specific hourly productivity costs (Supplementary Table S2).²⁸ The AUC method¹⁹ was used to compute the economic losses (or gains) owing to changes in working hours over the full 4-year follow-up period. Furthermore, costs and effects were discounted at different rates (for the Dutch setting costs were discounted with 4.0% and effects with 1.5% for the UK setting costs and effects were discounted both with 3.5% and 3.0% for the US setting).

Sample Characteristics

There were no significant differences between the two treatment arms in sociodemographic and clinical characteristics (Table 1).

Outcomes

In the CBTuhr condition, 12.6% of the participants converted to psychosis over the 4-year follow-up period. In the RC condition, 24.8% made a transition over the 4-year follow-up. The risk difference was 0.248 − 0.126 = 0.122 (95%CI = 0.00–0.25), favoring CBTuhr over RC; this difference was significant (b = 1.324, SE_b = 0.017, z = 7.99, P < 0.001). The average QALY health gains over the 4-year period were 2.628 (95% CI = 2.457–2.799) for CBTuhr compared with 2.464 (95% CI = 2.264–2.665) for the RC condition. The difference in QALY gains was 2.628 − 2.464 = 0.164, favoring CBTuhr. As expected, this difference was not significant (b = −9.318, robust SE_b = 7.703, t = −1.21, P = 0.28).

Costs

Table 2 presents the mean costs in both treatment arms over the 4-year period. The larger share of the total costs was attributable to direct medical costs. Overall, the CBTuhr condition generated lower costs (US$19 121) than the RC condition (US$24 898), representing a (nonsignificant) cost reduction of US$5777 (savings: 95% CI = −16 952–4190) per participant.

The difference between CBTuhr and RC in intervention services received is partly explained by the lower psychosis conversion rate in the CBTuhr group and partly by a generally higher service use in RC. Patients with conversion: US$35 300 in RC vs US$23 600 in CBTuhr. Patients without conversion: US$21 700 in RC vs US$18 300 in CBTuhr. Moreover, the additional costs incurred after conversion in RC (+US$13 600) is higher than the additional costs incurred after conversion in CBTuhr (+US$5300).

Economic Evaluation

From a health care perspective, the incremental costs were −US$5 777 (savings; 95% CI = −16 952 to 4190) and the incremental effect was 0.12 (a larger fraction of averted transitions to psychosis in the CBTuhr condition). This represents a situation that health economists refer to as ‘dominant’, ie, the new intervention dominates the RC condition from a cost-effectiveness perspective. The ICER is subject to stochastic uncertainty. Including all psychiatric health care costs, the intervention is associated with 82.9% of the 2500 simulated ICERs appearing in the dominant SE quadrant (Table 3), indicating a likelihood of about 83% that more transitions to psychosis are averted for fewer costs by the CBTuhr intervention relative to RC alone. The northeast (NE) quadrant contained 13.6% of the simulated ICERs, the NW quadrant 0.7%, and the final 2.8% was located in the southwest (SW) quadrant. Figure 2a depicts a scatterplot of 2500 simulated ICERs on the ICER plane. The ICER acceptability curve (figure 2c) for CBTuhr vs RC showed that with no willingness to pay (WTP) for one averted transition to psychosis, there is an 86% probability that CBTuhr results in positive net benefits compared with RC. The probability of positive net benefits increases slightly when the society is willing to pay more for an additional averted psychosis. With a WTP of US$10 000 and US$20 000, CBTuhr has a probability of 90% and 92%, respectively, of resulting in positive net benefits compared with RC alone.

Furthermore, the QALY difference was 0.16 (95% CI = 0.14–0.46). Of the 2500 simulated ICERs (Figure 2b), 74.8% fell into the SE quadrant, indicating the likelihood that more QALYs were generated for fewer costs by the intervention relative to RC alone. The NE quadrant contained 10.8% of the simulated ICERs, the inferior NW quadrant 2.9%, with the final 11.5% located in the SW quadrant. The ICER acceptability curve (figure 2d) showed that even without any WTP for one extra QALY, there is an 86% (74.8% + 11.5%) probability that CBTuhr results in positive net benefits compared with RC. The probability increases slightly when society is willing to pay more for a QALY gained. At the Dutch WTP threshold value (US$ 24 560 or €20 000 per QALY) the intervention has a 92% probability of being cost-effective compared with RC.

Sensitivity Analysis

Sensitivity analyses were conducted by repeating the main analysis with LOCF imputation. This was repeated for both the CEA and the CUA and from both a health care system perspective and a societal perspective (Table 3). As expected, imputation with LOCF generally resulted in wider CIs, but this led to only modest changes in the probability of the intervention being both more effective and cost saving. The costs from a societal perspective were associated with wider CIs, as the amount of productivity varied strongly between participants. Alternative discount rates led to minor changes in costs (the change in incremental costs was less than 2% for both the health care and societal perspective) and effects (the change in incremental effects was less than 4% for both the health care and societal perspective), thereby not altering our conclusions.

Main Findings

The cost-effectiveness analysis showed that CBTuhr had an 83% probability of being more effective and less costly than RC. Similarly, the CTU shows that CBTuhr had a 75% likelihood of being more effective and less costly than RC. Various sensitivity analyses attested to the robustness of these findings. Using the Dutch WTP threshold of US$24 560 (€20 000) per QALY, the probability is 92% that the intervention is cost-effective. Also from the societal perspective, there is a substantial likelihood (73% in the primary and 78% in the secondary analysis) that CBTuhr results in more effects at lower costs. The conclusions indicate that the results of the 18 months analysis³⁵ do not deteriorate when considering a period of 4 years and guided some patients through a critical period with declining transition rates.³⁶

Costs of Identifying UHR Patients

As UHR patients were already involved with mental health care services, additional costs are mainly screening costs. The Prodromal Questionnaire 16³⁷ is an online screener used to preselect patients for the CAARMS interview in the mental health care and takes 2min time. The costs of identifying the UHR patients and training staff (US$724) were not factored into the cost of intervention, because they are the same for both groups. For further information, see Ising et al.³⁵ Actual implementation of the intervention would, however, require an additional US$724 per identified patient, thereby lowering the observed cost reduction of US$5777 with US$724 per patient. This would decrease the probability of more effects at lower costs from 83% to 79% in the primary analysis and from 75% to 72% in the secondary analysis.

Clinical and Public Health Relevance

A recent meta-analysis⁴ demonstrated that CBT could prevent transition to psychosis. Moreover, it is a bonus that the intervention is cost-effective and perhaps even cost saving,³⁴ considering limited resources for health care. Until now, health economic evaluations in this field were either based on decision modeling^38–40 or were based on a small sample with a short-term follow-up.⁴¹ The present study supports the notion that CBTuhr is effective and probably cost saving in the long term.

In the present study, the highest risk of transition was present in the first 12 months after study inclusion; this temporal pattern of transition is comparable to the findings of a recent meta-analysis.³⁶ At 4-year follow-up, only two more patients had transitioned to psychosis, suggesting that there is a critical period for transition, rather than a lifelong threat.

The main drivers of differences in costs between RC and CBTuhr were the costs arising from hospital admissions for psychosis (a difference of US$2000) and day care (a difference of USS$1000). During the 4-year period, participants in RC were more often admitted to a psychiatric hospital (RC: 7 participants with a total of 15 admissions vs CBTuhr: 4 participants who had 5 admissions). The duration of the admission was 2.2 in RC and 2.3 in CBTuhr.

Besides the proportion of averted transitions to psychosis,^9,35 the difference in QALYs showed a clinically relevant but statistically nonsignificant effect of the intervention on the lives of at-risk individuals and at least no deterioration in their health-related quality of life.

Limitations

Some limitations of the present study should be addressed. First, our study suffered from a high attrition rate. Comparing the participants who were retained with those who were lost to follow-up and using the 18-month follow-up data to describe both groups showed that those who were later lost to follow-up had less dysfunctional beliefs about their illness, fewer emotional disturbances, better quality of life, and lower symptom levels of general psychopathology,⁴² ie, those who were lost appeared to have a better prognosis. So costs might be slightly overestimated and savings slightly underestimated. The use of EM to handle missing data might have introduced some bias. In the sensitivity analyses, outcomes and costs were imputed using LOCF to replace missing observations. The results were comparable to the results based on EM imputation. Both imputation methods showed a firm probability that the intervention generates better effects at lower costs (80.5%–82.9% for the primary analysis and 74.8%–78.4% for the secondary analysis).

Second, the study was not powered to detect statistical differences in QALY gains between CBTuhr and RC, and the difference in QALY health gains was not significant across the treatment arms. We therefore relied on a probabilistic decision-making approach. QALYs are a generic and standardized metric that capture improvements in quality of life and play an important role in health economic evaluations, because QALYs can be compared across various disorders; this adds to the generalizability of the CUA. Moreover, QALY health gains were measured with the EuroQoL, which is known to be insensitive to change in mental health.⁴³

Third, all measures were based on self-report, which can be vulnerable to recall bias. However, Bouwmans et al found that the agreement between reported data on contacts with psychotherapists and long-term absence from work and registration data was around 70%–75%.⁴² Furthermore, we cross-validated 90% of the self-reports with the electronic patient files and found an overestimation of service use in 5% of the cases, which was then corrected. Because 10% of the electronic patient files were not available for inspection, we had to rely on the self-reported health care uptake. A minor over-report of 0.5% (0.10×5%) is expected.

Fourth, costs for other medication than antipsychotic medication, eg antidepressants, were not measured.

Fifth, no pre-baseline cost data were collected. Therefore, it was not possible to investigate whether conditions differed in terms of health care uptake at the start of the trial or whether pre-baseline costs were related to attrition.

Lastly, because the study was conducted in the Netherlands, the results may not be generalizable to countries that have a different health care system.