Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews

BACKGROUND: Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus. METHODS: A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity. RESULTS: The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity. CONCLUSION: A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use

Assessing the format and content of journal published and non-journal published rapid review reports : A comparative study

BACKGROUND: As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured. OBJECTIVES: To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs. METHODS: JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented. RESULTS: We identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups. CONCLUSIONS: This comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice. STUDY REGISTRATION: The full protocol is available at: https://osf.io/29xvk/

Male responsibility and maternal morbidity: a cross-sectional study in two Nigerian states

<p>Abstract</p> <p>Background</p> <p>Nigeria continues to have high rates of maternal morbidity and mortality. This is partly associated with lack of adequate obstetric care, partly with high risks in pregnancy, including heavy work. We examined actionable risk factors and underlying determinants at community level in Bauchi and Cross River States of Nigeria, including several related to male responsibility in pregnancy.</p> <p>Method</p> <p>In 2009, field teams visited a stratified (urban/rural) last stage random sample of 180 enumeration areas drawn from the most recent censuses in each of Bauchi and Cross River states. A structured questionnaire administered in face-to-face interviews with women aged 15-49 years documented education, income, recent birth history, knowledge and attitudes related to safe birth, and deliveries in the last three years. Closed questions covered female genital mutilation, intimate partner violence (IPV) in the last year, IPV during the last pregnancy, work during the last pregnancy, and support during pregnancy. The outcome was complications in pregnancy and delivery (eclampsia, sepsis, bleeding) among survivors of childbirth in the last three years. We adjusted bivariate and multivariate analysis for clustering.</p> <p>Findings</p> <p>The most consistent and prominent of 28 candidate risk factors and underlying determinants for non-fatal maternal morbidity was intimate partner violence (IPV) during pregnancy (ORa 2.15, 95%CIca 1.43-3.24 in Bauchi and ORa 1.5, 95%CI 1.20-2.03 in Cross River). Other spouse-related factors in the multivariate model included not discussing pregnancy with the spouse and, independently, IPV in the last year. Shortage of food in the last week was a factor in both Bauchi (ORa 1.66, 95%CIca 1.22-2.26) and Cross River (ORa 1.32, 95%CIca 1.15-1.53). Female genital mutilation was a factor among less well to do Bauchi women (ORa 2.1, 95%CIca 1.39-3.17) and all Cross River women (ORa 1.23, 95%CIca 1.1-1.5).</p> <p>Interpretation</p> <p>Enhancing clinical protocols and skills can only benefit women in Nigeria and elsewhere. But the violence women experience throughout their lives – genital mutilation, domestic violence, and steep power gradients – is accentuated through pregnancy and childbirth, when women are most vulnerable. IPV especially in pregnancy, women's fear of husbands or partners and not discussing pregnancy are all within men's capacity to change.</p

Do sugar-sweetened beverages cause adverse health outcomes in adults? A systematic review protocol

Background: Chronic diseases, such as cardiovascular disease and type 2 diabetes, impose significant burden to public health. Most chronic diseases are associated with underlying preventable risk factors, such as elevated blood pressure, blood glucose, and lipids, physical inactivity, excessive sedentary behaviours, overweight and obesity, and tobacco usage. Sugar-sweetened beverages are known to be significant sources of additional caloric intake, and given recent attention to their contribution in the development of chronic diseases, a systematic review is warranted. We will assess whether the consumption of sugar-sweetened beverages in adults is associated with adverse health outcomes and what the potential moderating factors are. Methods/Design: Of interest are studies addressing sugar-sweetened beverage consumption, taking a broad perspective. Both direct consumption studies as well as those evaluating interventions that influence consumption (e.g. school policy, educational) will be relevant. Non-specific or multi-faceted behavioural, educational, or policy interventions may also be included subject to the level of evidence that exists for the other interventions/exposures. Comparisons of interest and endpoints of interest are pre-specified. We will include randomized controlled trials, controlled clinical trials, interrupted time series studies, controlled before-after studies, prospective and retrospective comparative cohort studies, case-control studies, and nested case-control designs. The MEDLINE®, Embase, The Cochrane Library, CINAHL, ERIC, and PsycINFO® databases

Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses

Abstract Background Systematic reviews are the cornerstone of evidence-based medicine. However, systematic reviews are time consuming and there is growing demand to produce evidence more quickly, while maintaining robust methods. In recent years, artificial intelligence and active-machine learning (AML) have been implemented into several SR software applications. As some of the barriers to adoption of new technologies are the challenges in set-up and how best to use these technologies, we have provided different situations and considerations for knowledge synthesis teams to consider when using artificial intelligence and AML for title and abstract screening. Methods We retrospectively evaluated the implementation and performance of AML across a set of ten historically completed systematic reviews. Based upon the findings from this work and in consideration of the barriers we have encountered and navigated during the past 24 months in using these tools prospectively in our research, we discussed and developed a series of practical recommendations for research teams to consider in seeking to implement AML tools for citation screening into their workflow. Results We developed a seven-step framework and provide guidance for when and how to integrate artificial intelligence and AML into the title and abstract screening process. Steps include: (1) Consulting with Knowledge user/Expert Panel; (2) Developing the search strategy; (3) Preparing your review team; (4) Preparing your database; (5) Building the initial training set; (6) Ongoing screening; and (7) Truncating screening. During Step 6 and/or 7, you may also choose to optimize your team, by shifting some members to other review stages (e.g., full-text screening, data extraction). Conclusion Artificial intelligence and, more specifically, AML are well-developed tools for title and abstract screening and can be integrated into the screening process in several ways. Regardless of the method chosen, transparent reporting of these methods is critical for future studies evaluating artificial intelligence and AML

Do sugar-sweetened beverages cause adverse health outcomes in adults? A systematic review protocol

Abstract Background Chronic diseases, such as cardiovascular disease and type 2 diabetes, impose significant burden to public health. Most chronic diseases are associated with underlying preventable risk factors, such as elevated blood pressure, blood glucose, and lipids, physical inactivity, excessive sedentary behaviours, overweight and obesity, and tobacco usage. Sugar-sweetened beverages are known to be significant sources of additional caloric intake, and given recent attention to their contribution in the development of chronic diseases, a systematic review is warranted. We will assess whether the consumption of sugar-sweetened beverages in adults is associated with adverse health outcomes and what the potential moderating factors are. Methods/Design Of interest are studies addressing sugar-sweetened beverage consumption, taking a broad perspective. Both direct consumption studies as well as those evaluating interventions that influence consumption (e.g. school policy, educational) will be relevant. Non-specific or multi-faceted behavioural, educational, or policy interventions may also be included subject to the level of evidence that exists for the other interventions/exposures. Comparisons of interest and endpoints of interest are pre-specified. We will include randomized controlled trials, controlled clinical trials, interrupted time series studies, controlled before-after studies, prospective and retrospective comparative cohort studies, case-control studies, and nested case-control designs. The MEDLINE®, Embase, The Cochrane Library, CINAHL, ERIC, and PsycINFO® databases and grey literature sources will be searched. The processes for selecting studies, abstracting data, and resolving conflicts are described. We will assess risk of bias using design-specific tools. To determine sets of confounding variables that should be adjusted for, we have developed causal directed acyclic graphs and will use those to inform our risk of bias assessments. Meta-analysis will be conducted where appropriate; parameters for exploring statistical heterogeneity and effect modifiers are pre-specified. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used for determining the quality of evidence for outcomes. Systematic review registration PROSPERO CRD4201400963

Safety of influenza vaccination during pregnancy: a systematic review

Objective We conducted a systematic review to evaluate associations between influenza vaccination during pregnancy and adverse birth outcomes and maternal non-obstetric serious adverse events (SAEs), taking into consideration confounding and temporal biases.Methods Electronic databases (Ovid MEDLINE ALL, Embase Classic+Embase and the Cochrane Central Register of Controlled Trials) were searched to June 2021 for observational studies assessing associations between influenza vaccination during pregnancy and maternal non-obstetric SAEs and adverse birth outcomes, including preterm birth, spontaneous abortion, stillbirth, small-for-gestational-age birth and congenital anomalies. Studies of live attenuated vaccines, single-arm cohort studies and abstract-only publications were excluded. Records were screened using a liberal accelerated approach initially, followed by a dual independent approach for full-text screening, data extraction and risk of bias assessment. Pairwise meta-analyses were conducted, where two or more studies met methodological criteria for inclusion. The Grading of Recommendations, Assessment, Development and Evaluation approach was used to assess evidence certainty.Results Of 9443 records screened, 63 studies were included. Twenty-nine studies (24 cohort and 5 case–control) evaluated seasonal influenza vaccination (trivalent and/or quadrivalent) versus no vaccination and were the focus of our prioritised syntheses; 34 studies of pandemic vaccines (2009 A/H1N1 and others), combinations of pandemic and seasonal vaccines, and seasonal versus seasonal vaccines were also reviewed. Control for confounding and temporal biases was inconsistent across studies, limiting pooling of data. Meta-analyses for preterm birth, spontaneous abortion and small-for-gestational-age birth demonstrated no significant associations with seasonal influenza vaccination. Immortal time bias was observed in a sensitivity analysis of meta-analysing risk-based preterm birth data. In descriptive summaries for stillbirth, congenital anomalies and maternal non-obstetric SAEs, no significant association with increased risk was found in any studies. All evidence was of very low certainty.Conclusions Evidence of very low certainty suggests that seasonal influenza vaccination during pregnancy is not associated with adverse birth outcomes or maternal non-obstetric SAEs. Appropriate control of confounding and temporal biases in future studies would improve the evidence base

Screening for depression in children and adolescents: a protocol for a systematic review update

Abstract Background Major depressive disorder is common, debilitating, and affects feelings, thoughts, mood, and behaviors. Childhood and adolescence are critical periods for the development of depression and adolescence is marked by an increased incidence of mental health disorders. This protocol outlines the planned scope and methods for a systematic review update that will evaluate the benefits and harms of screening for depression in children and adolescents. Methods This review will update a previously published systematic review by Roseman and colleagues. Eligible studies are randomized controlled trials (RCTs) assessing formal screening in primary care to identify children or adolescents not already self-reporting symptoms of, diagnosed with, or treated for depression. If no or only a single RCT is available, we will consider controlled studies without random assignment. Studies of participants with characteristics associated with an elevated risk of depression will be analyzed separately. Outcomes of interest are symptoms of depression, classification of major depressive disorder based on a validated diagnostic interview, suicidality, health-related quality of life, social function, impact on lifestyle behavior (e.g., substance use, school performance, lost time at work, or school), false-positive results, overdiagnosis, overtreatment, labeling, and other harms such as those arising from treatment. We will search MEDLINE, Embase, PsycINFO, CINAHL, the Cochrane Library, and grey literature sources. Two reviewers will independently screen the titles and abstracts using the liberal accelerated method. Full-text screening will be performed independently by two reviewers using pre-specified eligibility criteria. Data extraction and risk of bias assessments will be performed independently by two reviewers. Pre-planned analyses, including subgroup and sensitivity analyses, are detailed within this protocol. Two independent reviewers will assess and finalize through consensus the certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, and prepare GRADE evidence profiles and summary of findings tables for each outcome of interest. Discussion The systematic review will provide a current state of the evidence of benefits and harms of depression screening in children and adolescents. These findings will be used by the Canadian Task Force on Preventive Health Care to inform the development of recommendations on depression screening. Systematic review registration PROSPERO CRD4202015037