Performance of methods for meta-analysis of diagnostic test accuracy with few studies or sparse data

Hierarchical models such as the bivariate and hierarchical summary receiver operating characteristic (HSROC) models are recommended for meta-analysis of test accuracy studies. These models are challenging to fit when there are few studies and/or sparse data (for example zero cells in contingency tables due to studies reporting 100% sensitivity or specificity); the models may not converge, or give unreliable parameter estimates. Using simulation, we investigated the performance of seven hierarchical models incorporating increasing simplifications in scenarios designed to replicate realistic situations for meta-analysis of test accuracy studies. Performance of the models was assessed in terms of estimability (percentage of meta-analyses that successfully converged and percentage where the between study correlation was estimable), bias, mean square error and coverage of the 95% confidence intervals. Our results indicate that simpler hierarchical models are valid in situations with few studies or sparse data. For synthesis of sensitivity and specificity, univariate random effects logistic regression models are appropriate when a bivariate model cannot be fitted. Alternatively, an HSROC model that assumes a symmetric SROC curve (by excluding the shape parameter) can be used if the HSROC model is the chosen meta-analytic approach. In the absence of heterogeneity, fixed effect equivalent of the models can be applied

Long-stay in forensic-psychiatric care in the UK

Purpose Forensic services provide care for mentally disordered offenders. In England this is provided at three levels of security—low, medium and high. Significant number of patients within these settings remain detained for protracted periods of time. This is both very costly and restrictive for individuals. No national studies have been conducted on this subject in England. Methods We employed a cross-sectional design using anonymised data from medical records departments in English secure forensic units. Data were collected from a large sample of medium secure patients (n = 1572) as well as the total high secure patient population (n = 715) resident on the census date (01-04-2013). We defined long-stay as a stay of more than 10 years in high, 5 years in medium or 15 years in a mix of high and medium secure settings. Long-stay status was assessed against patient demographic and admission information. Results We identified a significant proportion of long-stayers: 23.5% in high secure and 18.1% in medium secure care. Amongst medium secure units a large variation in long-stay prevalence was observed from 0 to 50%. Results indicated that MHA section, admission source and current ward type were independent factors associated with long-stay status. Conclusion This study identified a significant proportion of long-stayers in forensic settings in England. Sociodemographic factors identified in studies in individual settings may be less important than previously thought. The large variation in prevalence of long-stayers observed in the medium secure sample warrants further investigation

Second trimester serum tests for Down's Syndrome screening

Background Down's syndrome occurs when a person has three copies of chromosome 21 - or the specific area of chromosome 21 implicated in causing Down's syndrome - rather than two. It is the commonest congenital cause of mental retardation. Noninvasive screening based on biochemical analysis of maternal serum or urine, or fetal ultrasound measurements, allows estimates of the risk of a pregnancy being affected and provides information to guide decisions about definitive testing. Objectives To estimate and compare the accuracy of second trimester serum markers for the detection of Down’s syndrome. Search methods We carried out a sensitive and comprehensive literature search of MEDLINE (1980 to May 2007), EMBASE (1980 to 18 May 2007), BIOSIS via EDINA (1985 to 18 May 2007), CINAHL via OVID (1982 to 18 May 2007), The Database of Abstracts of Reviews of Effectiveness (The Cochrane Library 2007, Issue 1), MEDION (May 2007), The Database of Systematic Reviews and Meta-Analyses in Laboratory Medicine (May 2007), The National Research Register (May 2007), Health Services Research Projects in Progress database (May 2007). We studied reference lists and published review articles. Selection criteria Studies evaluating tests of maternal serum in women at 14-24 weeks of gestation for Down's syndrome, compared with a reference standard, either chromosomal verification or macroscopic postnatal inspection. Data collection and analysis Data were extracted as test positive/test negative results for Down's and non-Down's pregnancies allowing estimation of detection rates (sensitivity) and false positive rates (1-specificity). We performed quality assessment according to QUADAS criteria. We used hierarchical summary ROC meta-analytical methods to analyse test performance and compare test accuracy. Analysis of studies allowing direct comparison between tests was undertaken. We investigated the impact of maternal age on test performance in subgroup analyses. Main results Fifty-nine studies involving 341,261 pregnancies (including 1,994 with Down's syndrome) were included. Studies were generally high quality, although differential verification was common with invasive testing of only high-risk pregnancies. Seventeen studies made direct comparisons between tests. Fifty-four test combinations were evaluated formed from combinations of 12 different tests and maternal age; alpha-fetoprotein (AFP), unconjugated oestriol (uE3), total human chorionic gonadotrophin (hCG), free beta human chorionic gonadotrophin (βhCG), free alpha human chorionic gonadotrophin (αhCG), Inhibin A, SP2, CA125, troponin, pregnancy-associated plasma protein A (PAPP-A), placental growth factor (PGF) and proform of eosinophil major basic protein (ProMBP). Meta-analysis of 12 best performing or frequently evaluated test combinations showed double and triple tests (involving AFP, uE3, total hCG, free βhCG) significantly outperform individual markers, detecting six to seven out of every 10 Down's syndrome pregnancies at a 5% false positive rate. Tests additionally involving inhibin performed best (eight out of every 10 Down's syndrome pregnancies) but were not shown to be significantly better than standard triple tests in direct comparisons. Significantly lower sensitivity occurred in women over the age of 35 years. Women who miscarried in the over 35 group were more likely to have been offered an invasive test to verify a negative screening results, whereas those under 35 were usually not offered invasive testing for a negative screening result. Pregnancy loss in women under 35 therefore leads to under ascertainment of screening results, potentially missing a proportion of affected pregnancies and affecting the accuracy of the sensitivity. Authors' conclusions Tests involving two or more markers in combination with maternal age are significantly more sensitive than those involving one marker. The value of combining four or more tests or including inhibin have not been proven to show statistically significant improvement. Further study is required to investigate reduced test performance in women aged over 35 and the impact of differential pregnancy loss on study findings

The bi-factor structure of the 17-item Hamilton Depression Rating Scale in persistent major depression; dimensional measurement of outcome

Background:The 17-item Hamilton Depression Rating Scale (HDRS17) is used world-wide as an observer-rated measure of depression in randomised controlled trials (RCTs) despite continued uncertainty regarding its factor structure. This study investigated the dimensionality of HDRS17 for patients undergoing treatment in UK mental health settings with moderate to severe persistent major depressive disorder (PMDD).Methods:Exploratory Structural Equational Modelling (ESEM) was performed to examine the HDRS17 factor structure for adult PMDD patients with HDRS17 score ≥16. Participants (n = 187) were drawn from a multicentre RCT conducted in UK community mental health settings evaluating the outcomes of a depression service comprising CBT and psychopharmacology within a collaborative care model, against treatment as usual (TAU). The construct stability across a 12-month follow-up was examined through a measurement equivalence/invariance (ME/I) procedure via ESEM.Results:ESEM showed HDRS17 had a bi-factor structure for PMDD patients (baseline mean (sd) HDRS17 22.6 (5.2); 87% PMDD >1 year) with an overall depression factor and two group factors: vegetative-worry and retardation-agitation, further complicated by negative item loading. This bi-factor structure was stable over 12 months follow up. Analysis of the HDRS6 showed it had a unidimensional structure, with positive item loading also stable over 12 months.Conclusions:In this cohort of moderate-severe PMDD the HDRS17 had a bi-factor structure stable across 12 months with negative item loading on domain specific factors, indicating that it may be more appropriate to multidimensional assessment of settled clinical states, with shorter unidimensional subscales such as the HDRS6 used as measures of change

The factor structure of the Zanarini Rating Scale for Borderline Personality Disorder: Exploratory Structural Equation Modelling and measurement invariance over time

ObjectivesThere is a lack of independent longitudinal evidence on the factor structure and validity of the Zanarini Rating Scale for Borderline Personality Disorder (ZAN‐BPD). This study aimed to investigate the dimensionality of ZAN‐BPD and its conceptual consistency over time.MethodsAdult BPD participants (n = 276) were recruited for a multicentre, two‐arm randomised clinical trial with ZAN‐BPD measured at baseline and follow up at 12, 24 and 52 weeks. The construct and stability of the ZAN‐BPD across 52 weeks was examined through a measurement equivalence/invariance procedure via Exploratory Structural Equation Modelling.ResultsFactor analysis results showed that the ZAN‐BPD had a bi‐2 factor structure that was stable over 52 weeks with a general factor and two specific factors. Factor loadings for eight of the nine items were greater for the general factor than the two specific factors. Factor 1 contrasts externalising distress with internalising distress. Factor 2 contrasts depression and self‐destruction with interpersonal anxiety and conflict.ConclusionZAN‐BPD is a conceptually and empirically valid measure of total BPD symptom severity in BPD patients over time suitable for use in clinical trials. Two factors related to the expression of distress and self‐harm may be utilised as possible predictors of outcome

Feasibility of an online intervention (STAK-D) to promote physical activity in children with type 1 diabetes: protocol for a randomised controlled trial

Background: Regular physical activity has important health benefits for children with type 1 diabetes mellitus (T1DM), yet children and their parents face barriers to participation such as lack of self-efficacy or concerns around hypoglycaemia. Multimedia interventions are useful for educating children about their health and demonstrate potential to improve children’s health-related self-efficacy, but few paediatric clinics offer web-based resources as part of routine care. The Steps to Active Kids with Diabetes (STAK-D) programme is an online intervention grounded in psychological theory (social cognitive theory) and informed by extensive preliminary research. The aim of the programme is to encourage and support safe engagement with physical activity for children with T1DM. The aim of this research is to explore the feasibility of delivering the STAK-D programme to children aged 9–12 years with T1DM, and to assess the feasibility of further research to demonstrate its clinical and cost-effectiveness. Methods: Up to 50 children aged 9–12 years with T1DM and their parents will be recruited from two paediatric diabetes clinics in the UK. Child-parent dyads randomised to the intervention group will have access to the intervention website (STAK-D) and a wrist-worn activity monitor for 6 months. The feasibility of intervention and further research will be assessed by rate of recruitment, adherence, retention, data completion and adverse events. Qualitative interviews will be undertaken with a subsample of children and parents (up to 25 dyads) and health care professionals (up to 10). Health outcomes and the feasibility of outcome measurement tools will be assessed. These include self-efficacy (CSAPPA), objective physical activity, self-reported physical activity (PAQ), fear of hypoglycaemia (CHFS; PHFS), glycaemic control (HbA1c), insulin dose, Body Mass Index (BMI), health related quality of life (CHU9D; CHQ-PF28), health service use and patient-clinician communication. Assessments will be taken at baseline (T0), 8 weeks (T1) and at 6-month follow-up (T2). Discussion: The goal of this feasibility trial is to assess the delivery of STAK-D to promote physical activity among children with T1DM, and to assess the potential for further, definitive research to demonstrate its effectiveness. Results will provide the information necessary to design a larger randomised controlled trial and maximise the recruitment rate, intervention delivery and trial retention

The effectiveness of exercise-based interventions for preventing or treating postpartum depression: a systematic review and meta-analysis

© 2018, Springer-Verlag GmbH Austria, part of Springer Nature. Postpartum depression can have detrimental effects on both a mother’s physical and mental health and on her child’s growth and emotional development. The aim of this study is to assess the effectiveness of exercise/physical activity-based interventions in preventing and treating postpartum depressive symptoms in primiparous and multiparous women to the end of the postnatal period at 52weeks postpartum. Electronic databases were searched for published and unpublished randomised controlled trials of exercise/physical activity-based interventions in preventing and treating depressive symptoms and increasing health-related quality of life in women from 4 to 52weeks postpartum. The results of the studies were meta-analysed and effect sizes with confidence intervals were calculated. The Grading of Recommendations Assessment and Development and Evaluation (GRADE) system was used to determine the confidence in the effect estimates. Eighteen trials conducted across a range of countries met the inclusion criteria. Most of the exercise interventions were aerobic and coaching compared to usual care, non-intervention and active controls. Small effect sizes of exercise-based interventions in reducing depressive symptoms were observed collectively and the quality of evidence was low across the individual studies. Although exercise-based interventions could create an alternative therapeutic approach for preventing major depression in postpartum women who experience subthreshold elevated depressive symptoms, the clinical effectiveness and the cost-effectiveness of exercise-based and physical activity interventions need to be better established. There is a need for further more rigorous testing of such interventions in high-quality randomised controlled trials against active control conditions before large-scale roll-out of these interventions in clinical practice is proposed

The validity of the Strengths and Difficulties Questionnaire (SDQ) for children with ADHD symptoms

Background: The Strengths and Difficulties Questionnaire (SDQ) is widely used to assess child and adolescent mental health problems. However, the factor structure of the SDQ is subject to debate and there is limited evidence investigating measurement equivalence invariance (ME/I) between treatment groups, informants, and across time.Method: A randomised controlled trial (RCT) recruited 250 participants (6-17 years) who had been referred for an attention deficit hyperactivity disorder (ADHD) assessment. Participants and their clinician either received or did not receive a QbTest report (computer task measuring attention, impulsivity and activity). Parents and teachers completed the SDQ at baseline and 6-months later. This study aimed to understand the factor structure of the SDQ in a clinic referred ADHD sample, and validate the scale as a screening/diagnostic aide and as a measure of treatment outcome both in clinical and research settings. Exploratory Structural Equation Modelling (ESEM) was performed to examine the factor structure, and ME/I was assessed between treatment groups, informants, and time points. The criterion validity of the SDQ predictive algorithm for ADHD was compared with clinician and research diagnoses using logistic regression and tests of diagnostic accuracy. Results: A 5-factor structure provided the best fit with strong factorial invariance between treatment groups and across time points, but not across informants (parent and teacher ratings). SDQ ratings of ‘probable’ hyperactivity disorder were good predictors of clinical (OR=10.20, 95%CI 2.18-48.71,p=0.003) and research diagnoses of ADHD (OR=6.82, 95%CI 1.95-23.84,p=0.003), and research diagnoses of Hyperkinetic disorder (OR=4.02, 95%CI 1.13-14.25,p=0.031). Further examination of the SDQ hyperactivity ‘probable’ rating showed good specificity (84.5%-74.5%) but poor sensitivity (45.0-42.5%) for ADHD.Conclusion: The findings indicate the SDQ is a valid outcome measure for use in RCTs and clinical settings. However, care should be taken when using the SDQ predictive algorithm to screen for ADHD in clinically referred samples

Sequential Effects in Essay Ratings: Evidence of Assimilation Effects Using Cross-Classified Models

Writing assessments are an indispensable part of most language competency tests. In our research, we used cross-classified models to study rater effects in the real essay rating process of a large-scale, high-stakes educational examination administered in China in 2011. Generally, four cross-classified models are suggested for investigation of rater effects: (1) the existence of sequential effects, (2) the direction of the sequential effects, and (3) differences in raters by their individual characteristics. We applied these models to the data to account for possible cluster effects caused by the application of multiple rating strategies. The results of our research showed that raters demonstrated sequential effects during the rating process. In contrast to many other studies on rater effects, our study found that raters exhibited assimilation effects. The more experienced, lenient, and qualified raters were less susceptible to assimilation effects. In addition, our research demonstrated the feasibility and appropriateness of using cross-classified models in assessing rater effects for such data structures. This paper also discusses the implications for educators and practitioners who are interested in reducing sequential effects in the rating process, and suggests directions for future research