Reliability and validity of assessing user satisfaction with web-based health interventions.

BACKGROUND: The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users' satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. OBJECTIVE: The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). METHODS: The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. RESULTS: Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=-.35, P<.001) and perceived stress (r=-.48, P<.001) demonstrated the construct validity of the scale. CONCLUSIONS: The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user's general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user's perspective in the overall evaluation of Web-based health interventions

Reducing problematic alcohol use in employees: economic evaluation of guided and unguided web-based interventions alongside a three-arm randomized controlled trial

© 2021 The Authors. Addiction published by John Wiley & Sons Ltd on behalf of Society for the Study of Addiction.Aims: To perform an economic evaluation of guided and unguided internet-based interventions to reduce problematic alcohol consumption in employees compared with a waiting-list control condition (WLC) with unrestricted access to treatment-as-usual. Design: A cost-effectiveness analysis (CEA) and cost–utility analysis (CUA) from a societal and a cost–benefit analysis from the employer's perspective with a 6-month time horizon. Setting: Open recruitment in the German working population. Participants: Employees (178 males, 256 females, mean age 47 years) consuming at least 14 (women) or 21 (men) standard units of alcohol (SUAs) per week and scoring ≥ 8 (men) or 6 (women) on the Alcohol Use Disorders Identification Test. Measurements: On-line questionnaires administered to assess SUAs and assess quality of life (AQoL-8D) and resource use. Outcome measure was responder (≤ 14/≤ 21 SUAs) for the CEA and quality-adjusted life years (QALYs) for the CUA. Net benefit regression was used to estimate cost-effectiveness for each study arm. Bootstrapping and sensitivity analyses were performed to account for uncertainty. Interventions: Five weekly modules including personalized normative feedback, motivational interviewing, goal setting, problem-solving and emotion regulation, provided with adherence-focused guidance [n = 142; responders: n = 73 (51.4%); QALYs = 0.364, standard error (SE) = 0.006] or without guidance [n = 146; n = 66 (45.2%); 0.359, 0.007]. Controls were on a waiting-list [n = 144; n = 38 (26.4%); 0.342, 0.007]. Findings: From a societal perspective, the guided intervention had a probability of 55% (54%) of being the most efficient strategy at a willingness-to-pay (WTP) of €0 per responder (QALY) gained, compared with the unguided intervention and the control condition. At a WTP of €20 000 per QALY gained, the probability was 78%. From an employer's perspective, the guided intervention had a higher probability of a positive return on investment (81%) compared with the unguided intervention (58%). Conclusion: A guided internet-based intervention to reduce problematic alcohol consumption in employees appears to be both cost-beneficial and cost-effective

Do guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis

Little is known about clinically relevant changes in guided Internet-based interventions for depression. Moreover, methodological and power limitations preclude the identification of patients’ groups that may benefit more from these interventions. This study aimed to investigate response rates, remission rates, and their moderators in randomized controlled trials (RCTs) comparing the effect of guided Internet-based interventions for adult depression to control groups using an individual patient data meta-analysis approach. Literature searches in PubMed, Embase, PsycINFO and Cochrane Library resulted in 13,384 abstracts from database inception to January 1, 2016. Twenty-four RCTs (4889 participants) comparing a guided Internet-based intervention with a control group contributed data to the analysis. Missing data were multiply imputed. To examine treatment outcome on response and remission, mixed-effects models with participants nested within studies were used. Response and remission rates were calculated using the Reliable Change Index. The intervention group obtained significantly higher response rates (OR = 2.49, 95% CI 2.17–2.85) and remission rates compared to controls (OR = 2.41, 95% CI 2.07–2.79). The moderator analysis indicated that older participants (OR = 1.01) and native-born participants (1.66) were more likely to respond to treatment compared to younger participants and ethnic minorities respectively. Age (OR = 1.01) and ethnicity (1.73) also moderated the effects of treatment on remission.Moreover, adults with more severe depressive symptoms at baseline were more likely to remit after receiving internet-based treatment (OR = 1.19). Guided Internet-based interventions lead to substantial positive treatment effects on treatment response and remission at post-treatment. Thus, such interventions may complement existing services for depression and potentially reduce the gap between the need and provision of evidence-based treatments