Individual participant data meta analysis to compare EPDS accuracy to detect major depression with and without the self-harm item

Item 10 of the Edinburgh Postnatal Depression Scale (EPDS) is intended to assess thoughts of intentional self-harm but may also elicit concerns about accidental self-harm. It does not specifically address suicide ideation but, nonetheless, is sometimes used as an indicator of suicidality. The 9-item version of the EPDS (EPDS-9), which omits item 10, is sometimes used in research due to concern about positive endorsements of item 10 and necessary follow-up. We assessed the equivalence of total score correlations and screening accuracy to detect major depression using the EPDS-9 versus full EPDS among pregnant and postpartum women. We searched Medline, Medline In-Process and Other Non-Indexed Citations, PsycINFO, and Web of Science from database inception to October 3, 2018 for studies that administered the EPDS and conducted diagnostic classification for major depression based on a validated semi-structured or fully structured interview among women aged 18 or older during pregnancy or within 12 months of giving birth. We conducted an individual participant data meta-analysis. We calculated Pearson correlations with 95% prediction interval (PI) between EPDS-9 and full EPDS total scores using a random effects model. Bivariate random-effects models were fitted to assess screening accuracy. Equivalence tests were done by comparing the confidence intervals (CIs) around the pooled sensitivity and specificity differences to the equivalence margin of δ = 0.05. Individual participant data were obtained from 41 eligible studies (10,906 participants, 1407 major depression cases). The correlation between EPDS-9 and full EPDS scores was 0.998 (95% PI 0.991, 0.999). For sensitivity, the EPDS-9 and full EPDS were equivalent for cut-offs 7-12 (difference range - 0.02, 0.01) and the equivalence was indeterminate for cut-offs 13-15 (all differences - 0.04). For specificity, the EPDS-9 and full EPDS were equivalent for all cut-offs (difference range 0.00, 0.01). The EPDS-9 performs similarly to the full EPDS and can be used when there are concerns about the implications of administering EPDS item 10.This study was funded by the Canadian Institutes of Health Research (CIHR, KRS-140994). Dr. Qiu was supported by a scholarship from the China Scholarship Council. Drs. Wu and Levis were supported by Fonds de recherche du Québec—Santé (FRQ-S) Postdoctoral Training Fellowships. Dr. Benedetti was supported by a Fonds de recherche du Québec – Santé (FRQS) researcher salary award. Dr. Thombs was supported by a Tier 1 Canada Research Chair. Ms. Rice was supported by a Vanier Canada Graduate Scholarship. The primary study by Alvarado et al. was supported by the Ministry of Health of Chile. The primary study by Barnes et al. was supported by a grant from the Health Foundation (1665/608). The primary study by Beck et al. was supported by the Patrick and Catherine Weldon Donaghue Medical Research Foundation and the University of Connecticut Research Foundation. The primary study by Helle et al. was supported by the Werner Otto Foundation, the Kroschke Foundation, and the Feindt Foundation. The primary study by Figueira et al. was supported by the Brazilian Ministry of Health and by the National Counsel of Technological and Scientific Development (CNPq) (Grant no.403433/2004-5). The primary study by Couto et al. was supported by the National Counsel of Technological and Scientific Development (CNPq) (Grant no. 444254/2014-5) and the Minas Gerais State Research Foundation (FAPEMIG) (Grant no. APQ-01954-14). The primary study by Chorwe-Sungani et al. was supported by the University of Malawi through grant QZA-0484 NORHED 2013. The primary study by de Figueiredo et al. was supported by Fundação de Amparo à Pesquisa do Estado de São Paulo. The primary study by Tissot et al. was supported by the Swiss National Science Foundation (grant 32003B 125493). The primary study by Fernandes et al. was supported by grants from the Child: Care Health and Development Trust and the Department of Psychiatry, University of Oxford, Oxford, UK, and by the Ashok Ranganathan Bursary from Exeter College, University of Oxford. Dr. Fernandes is supported by a University of Southampton National Institute for Health Research (NIHR) academic clinical fellowship in Paediatrics. The primary study by van Heyningen et al. was supported by the Medical Research Council of South Africa (fund no. 415865), Cordaid Netherlands (Project 103/10002 G Sub 7) and the Truworths Community Foundation Trust, South Africa. Dr. van Heyningen was supported by the National Research Foundation of South Africa and the Harry Crossley Foundation. VHYTHE001/1232209. The primary study by Tendais et al. was supported under the project POCI/SAU-ESP/56397/2004 by the Operational Program Science and Innovation 2010 (POCI 2010) of the Community Support Board III and by the European Community Fund FEDER. The primary study by Fisher et al. was supported by a grant under the Invest to Grow Scheme from the Australian Government Department of Families, Housing, Community Services and Indigenous Affairs. The primary study by Green et al. was supported by a grant from the Duke Global Health Institute (453-0751). The primary study by Howard et al. was supported by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (Grant Reference Numbers RP-PG-1210-12002 and RP-DG-1108-10012) and by the South London Clinical Research Network. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. The primary study by Kettunen et al. was supported with an Annual EVO Financing (Special government subsidies from the Ministry of Health and Welfare, Finland) by North Karelia Central Hospital and Päijät-Häme Central Hospital. The primary study by Phillips et al. was supported by a scholarship from the National Health and Medical and Research Council (NHMRC). The primary study by Roomruangwong et al. was supported by the Ratchadaphiseksomphot Endowment Fund 2013 of Chulalongkorn University (CU-56-457-HR). The primary study by Martínez et al. was supported by Iniciativa Científica Milenio, Chile, process # IS130005 and by Fondo Nacional de Desarrollo Científico y Tecnológico, Chile, process # 1130230. The primary study by Nakić Radoš et al. was supported by the Croatian Ministry of Science, Education, and Sports (134-0000000-2421). The primary study by Usuda et al. was supported by Grant-in-Aid for Young Scientists (A) from the Japan Society for the Promotion of Science (primary investigator: Daisuke Nishi, MD, PhD), and by an Intramural Research Grant for Neurological and Psychiatric Disorders from the National Center of Neurology and Psychiatry, Japan. The primary study by Pawlby et al. was supported by a Medical Research Council UK Project Grant (number G89292999N). The primary study by Rochat et al. was supported by grants from the University of Oxford (HQ5035), the Tuixen Foundation (9940), the Wellcome Trust (082384/Z/07/Z and 071571), and the American Psychological Association. Dr. Rochat receives salary support from a Wellcome Trust Intermediate Fellowship (211374/Z/18/Z). The primary study by Rowe et al. was supported by the diamond Consortium, beyondblue Victorian Centre of Excellence in Depression and Related Disorders. The primary study by Comasco et al. was supported by funds from the Swedish Research Council (VR: 521-2013-2339, VR:523-2014-2342), the Swedish Council for Working Life and Social Research (FAS: 2011-0627), the Marta Lundqvist Foundation (2013, 2014), and the Swedish Society of Medicine (SLS-331991). The primary study by Smith-Nielsen et al. was supported by a grant from the charitable foundation Tryg Foundation (Grant ID no 107616). The primary study by Prenoveau et al. was supported by The Wellcome Trust (grant number 071571). The primary study by Stewart et al. was supported by Professor Francis Creed’s Journal of Psychosomatic Research Editorship fund (BA00457) administered through University of Manchester. The primary study by Su et al. was supported by grants from the Department of Health (DOH94F044 and DOH95F022) and the China Medical University and Hospital (CMU94-105, DMR-92-92 and DMR94-46). The primary study by Tandon et al. was funded by the Thomas Wilson Sanitarium. The primary study by Tran et al. was supported by the Myer Foundation who funded the study under its Beyond Australia scheme. Dr. Tran was supported by an early career fellowship from the Australian National Health and Medical Research Council. The primary study by Vega-Dienstmaier et al. was supported by Tejada Family Foundation, Inc, and Peruvian-American Endowment, Inc. The primary study by Yonkers et al. was supported by a National Institute of Child Health and Human Development grant (5 R01HD045735). No other authors reported funding for primary studies or for their work on this study

Accuracy of the Edinburgh Postnatal Depression Scale (EPDS) for screening to detect major depression among pregnant and postpartum women: systematic review and meta-analysis of individual participant data

Objective: To evaluate the Edinburgh Postnatal Depression Scale (EPDS) for screening to detect major depression in pregnancy and postpartum. Design: Individual participant data meta-analysis. Data Sources: Medline, Medline In-Process & Other Non-Indexed Citations, PsycINFO, and Web of Science were searched (inception – October 3, 2018). Eligibility criteria for selecting studies: Eligible datasets included EPDS scores and major depression classification via validated diagnostic interviews. Bivariate random-effects meta-analysis was used to estimate EPDS sensitivity and specificity compared to semi-structured, fully structured (Mini International Neuropsychiatric Interview [MINI] excluded), and MINI diagnostic interviews, separately, using individual participant data. One-stage meta-regression was used to examine accuracy by reference standard categories and participant characteristics. Results: Individual participant data were obtained from 58 of 83 eligible studies (70%; 15,557 of 22,788 eligible participants [68%], 2,069 cases). Combined sensitivity and specificity was maximized at cutoff 11 across reference standards. Among studies with a semi-structured interview (36 studies, 9,066 participants, 1,330 cases), sensitivity and specificity (95% CI) were 0.85 (0.79 to 0.90) and 0.84 (0.79 to 0.88) for cutoff 10, 0.81 (0.75 to 0.87) and 0.88 (0.85 to 0.91) for cutoff 11, and 0.66 (0.58 to 0.74) and 0.95 (0.92 to 0.96) for cutoff 13. Accuracy was similar across reference standards and subgroups, including for women in pregnancy and postpartum. Conclusions: An EPDS cutoff of 11 maximized combined sensitivity and specificity; a cutoff of ≥ 13 was less sensitive but more specific. To identify women in pregnancy and postpartum with higher symptom levels, a cutoff of 13 or greater could be used. Lower cutoffs could be used if the intention is to avoid false negatives and identify most patients who meet diagnostic criteria.</p

Probability of major depression classification based on the SCID, CIDI, and MINI diagnostic interviews: a synthesis of three individual participant data meta-analyses

Introduction: Three previous individual participant data meta-analyses (IPDMAs) reported that, compared to the Structured Clinical Interview for the DSM (SCID), alternative reference standards, primarily the Composite International Diagnostic Interview (CIDI) and the Mini International Neuropsychiatric Interview (MINI), tended to misclassify major depression status, when controlling for depression symptom severity. However, there was an important lack of precision in the results. Objective: To compare the odds of the major depression classification based on the SCID, CIDI, and MINI. Methods: We included and standardized data from 3 IPDMA databases. For each IPDMA, separately, we fitted binomial generalized linear mixed models to compare the adjusted odds ratios (aORs) of major depression classification, controlling for symptom severity and characteristics of participants, and the interaction between interview and symptom severity. Next, we synthesized results using a DerSimonian-Laird random-effects meta-analysis. Results: In total, 69,405 participants (7,574 [11%] with major depression) from 212 studies were included. Controlling for symptom severity and participant characteristics, the MINI (74 studies; 25,749 participants) classified major depression more often than the SCID (108 studies; 21,953 participants; aOR 1.46; 95% confidence interval [CI] 1.11–1.92]). Classification odds for the CIDI (30 studies; 21,703 participants) and the SCID did not differ overall (aOR 1.19; 95% CI 0.79–1.75); however, as screening scores increased, the aOR increased less for the CIDI than the SCID (interaction aOR 0.64; 95% CI 0.52–0.80). Conclusions: Compared to the SCID, the MINI classified major depression more often. The odds of the depression classification with the CIDI increased less as symptom levels increased. Interpretation of research that uses diagnostic interviews to classify depression should consider the interview characteristics

An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data

Abstract Background Selective reporting of results from only well-performing cut-offs leads to biased estimates of accuracy in primary studies of questionnaire-based screening tools and in meta-analyses that synthesize results. Individual participant data meta-analysis (IPDMA) of sensitivity and specificity at each cut-off via bivariate random-effects models (BREMs) can overcome this problem. However, IPDMA is laborious and depends on the ability to successfully obtain primary datasets, and BREMs ignore the correlation between cut-offs within primary studies. Methods We compared the performance of three recent multiple cut-off models developed by Steinhauser et al., Jones et al., and Hoyer and Kuss, that account for missing cut-offs when meta-analyzing diagnostic accuracy studies with multiple cut-offs, to BREMs fitted at each cut-off. We used data from 22 studies of the accuracy of the Edinburgh Postnatal Depression Scale (EPDS; 4475 participants, 758 major depression cases). We fitted each of the three multiple cut-off models and BREMs to a dataset with results from only published cut-offs from each study (published data) and an IPD dataset with results for all cut-offs (full IPD data). We estimated pooled sensitivity and specificity with 95% confidence intervals (CIs) for each cut-off and the area under the curve. Results Compared to the BREMs fitted to the full IPD data, the Steinhauser et al., Jones et al., and Hoyer and Kuss models fitted to the published data produced similar receiver operating characteristic curves; though, the Hoyer and Kuss model had lower area under the curve, mainly due to estimating slightly lower sensitivity at lower cut-offs. When fitting the three multiple cut-off models to the full IPD data, a similar pattern of results was observed. Importantly, all models had similar 95% CIs for sensitivity and specificity, and the CI width increased with cut-off levels for sensitivity and decreased with an increasing cut-off for specificity, even the BREMs which treat each cut-off separately. Conclusions Multiple cut-off models appear to be the favorable methods when only published data are available. While collecting IPD is expensive and time consuming, IPD can facilitate subgroup analyses that cannot be conducted with published data only

Individual participant data meta-analysis to compare EPDS accuracy to detect major depression with and without the self-harm item

Item 10 of the Edinburgh Postnatal Depression Scale (EPDS) is intended to assess thoughts of intentional self-harm but may also elicit concerns about accidental self-harm. It does not specifically address suicide ideation but, nonetheless, is sometimes used as an indicator of suicidality. The 9-item version of the EPDS (EPDS-9), which omits item 10, is sometimes used in research due to concern about positive endorsements of item 10 and necessary follow-up. We assessed the equivalence of total score correlations and screening accuracy to detect major depression using the EPDS-9 versus full EPDS among pregnant and postpartum women. We searched Medline, Medline In-Process and Other Non-Indexed Citations, PsycINFO, and Web of Science from database inception to October 3, 2018 for studies that administered the EPDS and conducted diagnostic classification for major depression based on a validated semi-structured or fully structured interview among women aged 18 or older during pregnancy or within 12 months of giving birth. We conducted an individual participant data meta-analysis. We calculated Pearson correlations with 95% prediction interval (PI) between EPDS-9 and full EPDS total scores using a random effects model. Bivariate random-effects models were fitted to assess screening accuracy. Equivalence tests were done by comparing the confidence intervals (CIs) around the pooled sensitivity and specificity differences to the equivalence margin of δ = 0.05. Individual participant data were obtained from 41 eligible studies (10,906 participants, 1407 major depression cases). The correlation between EPDS-9 and full EPDS scores was 0.998 (95% PI 0.991, 0.999). For sensitivity, the EPDS-9 and full EPDS were equivalent for cut-offs 7-12 (difference range - 0.02, 0.01) and the equivalence was indeterminate for cut-offs 13-15 (all differences - 0.04). For specificity, the EPDS-9 and full EPDS were equivalent for all cut-offs (difference range 0.00, 0.01). The EPDS-9 performs similarly to the full EPDS and can be used when there are concerns about the implications of administering EPDS item 10.Trial registration: The original IPDMA was registered in PROSPERO (CRD42015024785).</p

Data-driven methods distort optimal cutoffs and accuracy estimates of depression screening tools: a simulation study using individual participant data

Objective: To evaluate, across multiple sample sizes, the degree that data-driven methods result in (1) optimal cutoffs different from population optimal cutoff and (2) bias in accuracy estimates. Study design and setting: A total of 1,000 samples of sample size 100, 200, 500 and 1,000 each were randomly drawn to simulate studies of different sample sizes from a database (n = 13,255) synthesized to assess Edinburgh Postnatal Depression Scale (EPDS) screening accuracy. Optimal cutoffs were selected by maximizing Youden's J (sensitivity+specificity–1). Optimal cutoffs and accuracy estimates in simulated samples were compared to population values. Results: Optimal cutoffs in simulated samples ranged from ≥ 5 to ≥ 17 for n = 100, ≥ 6 to ≥ 16 for n = 200, ≥ 6 to ≥ 14 for n = 500, and ≥ 8 to ≥ 13 for n = 1,000. Percentage of simulated samples identifying the population optimal cutoff (≥ 11) was 30% for n = 100, 35% for n = 200, 53% for n = 500, and 71% for n = 1,000. Mean overestimation of sensitivity and underestimation of specificity were 6.5 percentage point (pp) and -1.3 pp for n = 100, 4.2 pp and -1.1 pp for n = 200, 1.8 pp and -1.0 pp for n = 500, and 1.4 pp and -1.0 pp for n = 1,000. Conclusions: Small accuracy studies may identify inaccurate optimal cutoff and overstate accuracy estimates with data-driven methods.</p