Living systematic reviews:2. Combining human and machine effort

New approaches to evidence synthesis, which utilise human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities ('crowds') as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential - and limitations - of new ways of undertaking specific tasks in living systematic reviews, identifying areas where these human / machine 'technologies' are already in use, and where further research and development is needed. While the context is living systematic reviews, many of these enabling technologies apply equally to standard approaches to systematic reviewing

Selecting, refining and identifying priority Cochrane Reviews in health communication and participation in partnership with consumers and other stakeholders

Abstract : Background: Priority-setting partnerships between researchers and stakeholders (meaning consumers, health professionals and health decision-makers) may improve research relevance and value. The Cochrane Consumers and Communication Group (CCCG) publishes systematic reviews in 'health communication and participation', which includes concepts such as shared decision-making, patient-centred care and health literacy. We aimed to select and refine priority topics for systematic reviews in health communication and participation, and use these to identify five priority CCCG Cochrane Reviews. Methods: Twenty-eight participants (14 consumers, 14 health professionals/decision-makers) attended a 1-day workshop in Australia. Using large-group activities and voting, participants discussed, revised and then selected 12 priority topics from a list of 21 previously identified topics. In mixed small groups, participants refined these topics, exploring underlying problems, who they affect and potential solutions. Thematic analysis identified cross-cutting themes, in addition to key populations and potential interventions for future Cochrane Reviews. We mapped these against CCCG's existing review portfolio to identify five priority reviews. Results: Priority topics included poor understanding and implementation of patient-centred care by health services, the fact that health information can be a low priority for health professionals, communication and coordination breakdowns in health services, and inadequate consumer involvement in health service design. The four themes underpinning the topics were culture and organisational structures, health professional attitudes and assumptions, inconsistent experiences of care, and lack of shared understanding in the sector. Key populations for future reviews were described in terms of social health characteristics (e.g. people from indigenous or culturally and linguistically diverse backgrounds, elderly people, and people experiencing socioeconomic disadvantage) more than individual health characteristics. Potential interventions included health professional education, interventions to change health service/health professional culture and attitudes, and health service policies and standards. The resulting five priority Cochrane Reviews identified were improving end-of-life care communication, patient/family involvement in patient safety, improving future doctors' communication skills, consumer engagement strategies, and promoting patient-centred care. Conclusions: Stakeholders identified priority topics for systematic reviews associated with structural and cultural challenges underlying health communication and participation, and were concerned that issues of equity be addressed. Priority-setting with stakeholders presents opportunities and challenges for review producers

Genetic Influences on Patient-Oriented Outcomes in Traumatic Brain Injury : A Living Systematic Review of Non-Apolipoprotein E Single-Nucleotide Polymorphisms

There is a growing literature on the impact of genetic variation on outcome in traumatic brain injury (TBI). Whereas a substantial proportion of these publications have focused on the apolipoprotein E (APOE) gene, several have explored the influence of other polymorphisms. We undertook a systematic review of the impact of single-nucleotide polymorphisms (SNPs) in non-apolipoprotein E (non-APOE) genes associated with patient outcomes in adult TBI). We searched EMBASE, MEDLINE, CINAHL, and gray literature from inception to the beginning of August 2017 for studies of genetic variance in relation to patient outcomes in adult TBI. Sixty-eight articles were deemed eligible for inclusion into the systematic review. The SNPs described were in the following categories: neurotransmitter (NT) in 23, cytokine in nine, brain-derived neurotrophic factor (BDNF) in 12, mitochondrial genes in three, and miscellaneous SNPs in 21. All studies were based on small patient cohorts and suffered from potential bias. A range of SNPs associated with genes coding for monoamine NTs, BDNF, cytokines, and mitochondrial proteins have been reported to be associated with variation in global, neuropsychiatric, and behavioral outcomes. An analysis of the tissue, cellular, and subcellular location of the genes that harbored the SNPs studied showed that they could be clustered into blood-brain barrier associated, neuroprotective/regulatory, and neuropsychiatric/degenerative groups. Several small studies report that various NT, cytokine, and BDNF-related SNPs are associated with variations in global outcome at 6-12 months post-TBI. The association of these SNPs with neuropsychiatric and behavioral outcomes is less clear. A definitive assessment of role and effect size of genetic variation in these genes on outcome remains uncertain, but could be clarified by an adequately powered genome-wide association study with appropriate recording of outcomes.Peer reviewe

Producing Cochrane systematic reviews—a qualitative study of current approaches and opportunities for innovation and improvement

Background: Producing high-quality, relevant systematic reviews and keeping them up to date is challenging. Cochrane is a leading provider of systematic reviews in health. For Cochrane to continue to contribute to improvements in heath, Cochrane Reviews must be rigorous, reliable and up to date. We aimed to explore existing models of Cochrane Review production and emerging opportunities to improve the efficiency and sustainability of these processes. Methods: To inform discussions about how to best achieve this, we conducted 26 interviews and an online survey with 106 respondents. Results: Respondents highlighted the importance and challenge of creating reliable, timely systematic reviews. They described the challenges and opportunities presented by current production models, and they shared what they are doing to improve review production. They particularly highlighted significant challenges with increasing complexity of review methods; difficulty keeping authors on board and on track; and the length of time required to complete the process. Strong themes emerged about the roles of authors and Review Groups, the central actors in the review production process. The results suggest that improvements to Cochrane's systematic review production models could come from improving clarity of roles and expectations, ensuring continuity and consistency of input, enabling active management of the review process, centralising some review production steps; breaking reviews into smaller "chunks", and improving approaches to building capacity of and sharing information between authors and Review Groups. Respondents noted the important role new technologies have to play in enabling these improvements. Conclusions: The findings of this study will inform the development of new Cochrane Review production models and may provide valuable data for other systematic review producers as they consider how best to produce rigorous, reliable, up-to-date reviews

Producing an evidence‐based treatment information website in partnership with people affected by multiple sclerosis

Background and Aims: In earlier work, we identified that people affected by multiple sclerosis (MS) can have difficulty finding online treatment information that is up to date, trustworthy, understandable, and applicable to personal circumstances, but does not provoke confusion or negative emotional consequences. The objective was to develop online consumer summaries of MS treatment evidence (derived from Cochrane Reviews) that respond to identified treatment information needs of people affected by MS. Methods: A 2-phase mixed-methods project, conducted in partnership with consumers and an MS organisation. Phase 1 included review panels with consumers (Australians affected by MS) and health professionals to test paper-based treatment summaries before development, and pilot testing of the website. Phase 2 involved an online survey after website launch. Results: Eighty-three participants (85% affected by MS) took part. Phase 1 participants strongly endorsed key review summary components, including layering information, and additional sections to aid personal applicability. Participants additionally suggested questions for health professionals. Participants across both phases were receptive to the idea of being provided with Cochrane Review summaries online but were seeking other types of evidence and information, such as personal experiences and the latest experimental treatments, which could not be provided. While the small survey sample size (n = 58) limits application of the results to a broader population, the website was viewed favourably, as a useful, understandable, and trustworthy information source. Conclusion: We describe a partnership approach to developing online evidence-based treatment information, underpinned by an in-depth understanding of consumers' information needs

Genetic Influences on Patient-Oriented Outcomes in Traumatic Brain Injury: A Living Systematic Review of Non-Apolipoprotein E Single-Nucleotide Polymorphisms

There is a growing literature on the impact of genetic variation on outcome in traumatic brain injury (TBI). Whereas a substantial proportion of these publications have focused on the apolipoprotein E (APOE) gene, several have explored the influence of other polymorphisms.We undertook a systematic review of the impact of single-nucleotide polymorphisms (SNPs) in non–apolipoprotein E (non-APOE) genes associated with patient outcomes in adult TBI). We searched EMBASE, MEDLINE, CINAHL, and gray literature from inception to the beginning of August 2017 for studies of genetic variance in relation to patient outcomes in adult TBI. Sixty-eight articles were deemed eligible for inclusion into the systematic review. The SNPs described were in the following categories: neurotransmitter (NT) in 23, cytokine in nine, brain-derived neurotrophic factor (BDNF) in 12, mitochondrial genes in three, and miscellaneous SNPs in 21. All studies were based on small patient cohorts and suffered from potential bias. A range of SNPs associated with genes coding for monoamine NTs, BDNF, cytokines, and mitochondrial proteins have been reported to be associated with variation in global, neuropsychiatric, and behavioral outcomes. An analysis of the tissue, cellular, and subcellular location of the genes that harbored the SNPs studied showed that they could be clustered into blood–brain barrier associated, neuroprotective/regulatory, and neuropsychiatric/degenerative groups. Several small studies report that various NT, cytokine, and BDNF-related SNPs are associated with variations in global outcome at 6–12 months post-TBI. The association of these SNPs with neuropsychiatric and behavioral outcomes is less clear. A definitiv

Blood-Based Protein Biomarkers for the Management of Traumatic Brain Injuries in Adults Presenting to Emergency Departments with Mild Brain Injury: A Living Systematic Review and Meta-Analysis

Accurate diagnosis of traumatic brain injury (TBI) is critical to effective management and intervention, but can be challenging in patients with mild TBI. A substantial number of studies have reported the use of circulating biomarkers as signatures for TBI, capable of improving diagnostic accuracy and clinical decision making beyond current practice standards. We performed a systematic review and meta-analysis to comprehensively and critically evaluate the existing body of evidence for the use of blood protein biomarkers (S100 calcium binding protein B [S100B], glial fibrillary acidic protein [GFAP], neuron specific enolase [NSE], ubiquitin C-terminal hydrolase-L1 [UCH-L1]. tau, and neurofilament proteins) for diagnosis of intracranial lesions on CT following mild TBI. Effects of potential confounding factors and differential diagnostic performance of the included markers were explored. Further, appropriateness of study design, analysis, quality, and demonstration of clinical utility were assessed. Studies published up to October 2016 were identified through a MEDLINE®, Embase, and Cumulative Index to Nursing and Allied Health Literature (CINAHL) search. Following screening of the identified articles, 26 were selected as relevant. We found that measurement of S100B can help informed decision making in the emergency department, possibly reducing resource use; however, there is insufficient evidence that any of the other markers is ready for clinical application. Our work pointed out serious problems in the design, analysis, and reporting of many of the studies, and identified substantial heterogeneity and research gaps. These findings emphasize the importance of methodologically rigorous studies focused on a biomarker's intended use, and defining standardized, validated, and reproducible approaches. The living nature of this systematic review, which will summarize key updated infor

Living systematic reviews: 2. Combining human and machine effort

Published online 11 September 2017New approaches to evidence synthesis, which use human effort and machine automation in mutually reinforcing ways, can enhance the feasibility and sustainability of living systematic reviews. Human effort is a scarce and valuable resource, required when automation is impossible or undesirable, and includes contributions from online communities ("crowds") as well as more conventional contributions from review authors and information specialists. Automation can assist with some systematic review tasks, including searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment. Workflows can be developed in which human effort and machine automation can each enable the other to operate in more effective and efficient ways, offering substantial enhancement to the productivity of systematic reviews. This paper describes and discusses the potential-and limitations-of new ways of undertaking specific tasks in living systematic reviews, identifying areas where these human/machine "technologies" are already in use, and where further research and development is needed. While the context is living systematic reviews, many of these enabling technologies apply equally to standard approaches to systematic reviewing.James Thomas, Anna Noel-Storr, Iain Marshall, Byron Wallace, Steven McDonald, Chris Mavergames, Paul Glasziou, Ian Shemilt, Anneliese Synnot, Tari Turner, Julian Elliott, on behalf of the Living Systematic Review Network (Zachary Munn