Response to Comment on “Estimating the reproducibility of psychological science”

Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.status: publishe

Challenges in Recruiting University Students for Web-Based Indicated Prevention of Depression and Anxiety: Results From a Randomized Controlled Trial (ICare Prevent)

©Felix Bolinski, Annet Kleiboer, Koen Neijenhuijs, Eirini Karyotaki, Reinout Wiers, Lisa de Koning, Corinna Jacobi, Anna-Carlotta Zarski, Kiona K Weisel, Pim Cuijpers, Heleen Riper.Background: Depression and anxiety are common mental health conditions in college and university student populations. Offering transdiagnostic, web-based prevention programs such as ICare Prevent to those with subclinical complaints has the potential to reduce some barriers to receiving help (eg, availability of services, privacy considerations, and students’ desire for autonomy). However, uptake of these interventions is often low, and accounts of recruitment challenges are needed to complement available effectiveness research in student populations. Objective: The aims of this study were to describe recruitment challenges together with effective recruitment strategies for ICare Prevent and provide basic information on the intervention’s effectiveness. Methods: A 3-arm randomized controlled trial was conducted in a student sample with subclinical symptoms of depression and anxiety on the effectiveness of an individually guided (human support and feedback on exercises provided after each session, tailored to each participant) and automatically guided (computer-generated messages provided after each session, geared toward motivation) version of ICare Prevent, a web-based intervention with transdiagnostic components for the indicated prevention of depression and anxiety. The intervention was compared with care as usual. Descriptive statistics were used to outline recruitment challenges and effective web-based and offline strategies as well as students’ use of the intervention. A basic analysis of intervention effects was conducted using a Bayesian linear mixed model, with Bayes factors reported as the effect size. Results: Direct recruitment through students’ email addresses via the central student administration was the most effective strategy. Data from 35 participants were analyzed (individually guided: n=14, 40%; automatically guided: n=8, 23%; care as usual: n=13, 37%). Use of the intervention was low, with an average of 3 out of 7 sessions (SD 2.9) completed. The analyses did not suggest any intervention effects other than anecdotal evidence (all Bayes factors10≤2.7). Conclusions: This report adds to the existing literature on recruitment challenges specific to the student population. Testing the feasibility of recruitment measures and the greater involvement of the target population in their design, as well as shifting from direct to indirect prevention, can potentially help future studies in the field. In addition, this report demonstrates an alternative basic analytical strategy for underpowered randomized controlled trials. Trial Registration: International Clinical Trials Registry Platform NTR6562; https://tinyurl.com/4rbexzrk International Registered Report Identifier (IRRID): RR2-10.1186/s13063-018-2477-

Translation of the eHealth Impact Questionnaire for a Population of Dutch Electronic Health Users: Validation Study

BACKGROUND: The eHealth Impact Questionnaire (eHIQ) provides a standardized method to measure attitudes of electronic health (eHealth) users toward eHealth. It has previously been validated in a population of eHealth users in the United Kingdom and consists of 2 parts and 5 subscales. Part 1 measures attitudes toward eHealth in general and consists of the subscales attitudes towards online health information (5 items) and attitudes towards sharing health experiences online (6 items). Part 2 measures the attitude toward a particular eHealth application and consists of the subscales confidence and identification (9 items), information and presentation (8 items), and understand and motivation (9 items). OBJECTIVE: This study aimed to translate and validate the eHIQ in a Dutch population of eHealth users. METHODS: The eHIQ was translated and validated in accordance with the COnsensus-based Standards for the selection of health status Measurement INstruments criteria. The validation comprised 3 study samples, with a total of 1287 participants. Structural validity was assessed using confirmatory factor analyses and exploratory factor analyses (EFAs; all 3 samples). Internal consistency was assessed using hierarchical omega (all 3 samples). Test-retest reliability was assessed after 2 weeks, using 2-way intraclass correlation coefficients (sample 1). Measurement error was assessed by calculating the smallest detectable change (sample 1). Convergent and divergent validity were assessed using correlations with the remaining measures (all 3 samples). A graded response model was fit, and item information curves were plotted to describe the information provided by items across item trait levels (all 3 samples). RESULTS: The original factor structure showed a bad fit in all 3 study samples. EFAs showed a good fit for a modified factor structure in the first study sample. This factor structure was subsequently tested in samples 2 and 3 and showed acceptable to good fits. Internal consistency, test-retest reliability, convergent validity, and divergent validity were acceptable to good for both the original as the modified factor structure, except for test-retest reliability of one of the original subscales and the 2 derivative subscales in the modified factor structure. The graded response model showed that some items underperformed in both the original and modified factor structure. CONCLUSIONS: The Dutch version of the eHIQ (eHIQ-NL) shows a different factor structure compared with the original English version. Part 1 of the eHIQ-NL consists of 3 subscales: attitudes towards online health information (5 items), comfort with sharing health experiences online (3 items), and usefulness of sharing health experiences online (3 items). Part 2 of the eHIQ-NL consists of 3 subscales: motivation and confidence to act (10 items), information and presentation (13 items), and identification (3 items)

Challenges in Recruiting University Students for Web-Based Indicated Prevention of Depression and Anxiety: Results From a Randomized Controlled Trial (ICare Prevent)

Background: Depression and anxiety are common mental health conditions in college and university student populations. Offering transdiagnostic, web-based prevention programs such as ICare Prevent to those with subclinical complaints has the potential to reduce some barriers to receiving help (eg, availability of services, privacy considerations, and students’ desire for autonomy). However, uptake of these interventions is often low, and accounts of recruitment challenges are needed to complement available effectiveness research in student populations.  Objective: The aims of this study were to describe recruitment challenges together with effective recruitment strategies for ICare Prevent and provide basic information on the intervention’s effectiveness.  Methods: A 3-arm randomized controlled trial was conducted in a student sample with subclinical symptoms of depression and anxiety on the effectiveness of an individually guided (human support and feedback on exercises provided after each session, tailored to each participant) and automatically guided (computer-generated messages provided after each session, geared toward motivation) version of ICare Prevent, a web-based intervention with transdiagnostic components for the indicated prevention of depression and anxiety. The intervention was compared with care as usual. Descriptive statistics were used to outline recruitment challenges and effective web-based and offline strategies as well as students’ use of the intervention. A basic analysis of intervention effects was conducted using a Bayesian linear mixed model, with Bayes factors reported as the effect size. , Results : Direct recruitment through students’ email addresses via the central student administration was the most effective strategy. Data from 35 participants were analyzed (individually guided: n=14, 40%; automatically guided: n=8, 23%; care as usual: n=13, 37%). Use of the intervention was low, with an average of 3 out of 7 sessions (SD 2.9) completed. The analyses did not suggest any intervention effects other than anecdotal evidence (all Bayes factors10≤2.7).   Conclusions : This report adds to the existing literature on recruitment challenges specific to the student population. Testing the feasibility of recruitment measures and the greater involvement of the target population in their design, as well as shifting from direct to indirect prevention, can potentially help future studies in the field. In addition, this report demonstrates an alternative basic analytical strategy for underpowered randomized controlled trial

A systematic review of the measurement properties of the Body Image Scale (BIS) in cancer patients

Introduction: Body image is acknowledged as an important aspect of health-related quality of life in cancer patients. The Body Image Scale (BIS) is a patient-reported outcome measure (PROM) to evaluate body image in cancer patients. The aim of this study was to systematically review measurement properties of the BIS among cancer patients. Methods: A search in Embase, MEDLINE, PsycINFO, and Web of Science was performed to identify studies that investigated measurement properties of the BIS (Prospero ID 42017057237). Study quality was assessed (excellent, good, fair, poor), and data were extracted and analyzed according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology on structural validity, internal consistency, reliability, measurement error, hypothesis testing for construct validity, and responsiveness. Evidence was categorized into sufficient, insufficient, inconsistent, or indeterminate. Results: Nine studies were included. Evidence was sufficient for structural validity (one factor solution), internal consistency (α = 0.86–0.96), and reliability (r > 0.70); indeterminate for measurement error (information on minimal important change lacked) and responsiveness (increasing body image disturbance in only one study); and inconsistent for hypothesis testing (conflicting results). Quality of the evidence was moderate to low. No studies reported on cross-cultural validity. Conclusion: The BIS is a PROM with good structural validity, internal consistency, and test-retest reliability, but good quality studies on the other measurement properties are needed to optimize evidence. It is recommended to include a wider variety of cancer diagnoses and treatment modalities in these future studies