Technology Assisted Reviews: Finding the Last Few Relevant Documents by Asking Yes/No Questions to Reviewers

The goal of a technology-assisted review is to achieve high recall with low human effort. Continuous active learning algorithms have demonstrated good performance in locating the majority of relevant documents in a collection, however their performance is reaching a plateau when 80\%-90\% of them has been found. Finding the last few relevant documents typically requires exhaustively reviewing the collection. In this paper, we propose a novel method to identify these last few, but significant, documents efficiently. Our method makes the hypothesis that entities carry vital information in documents, and that reviewers can answer questions about the presence or absence of an entity in the missing relevance documents. Based on this we devise a sequential Bayesian search method that selects the optimal sequence of questions to ask. The experimental results show that our proposed method can greatly improve performance requiring less reviewing effort.Comment: This paper is accepted by SIGIR 201

Reducing systematic review workload using text mining: opportunities and pitfalls

This EAHIL workshop focussed on three applications of text mining to assist with screening citations for systematic reviews, and encouraged participants to discuss issues affecting their adoption. This paper outlines these applications and summarises the factors raised by participants in relation to their uptake. Key aspects to uptake include having an accepted advantage over existing approaches, coupled with training and user support

The 'active ingredients' for successful community engagement with disadvantaged expectant and new mothers: a qualitative comparative analysis

AIMS: To explore which conditions of community engagement are implicated in effective interventions targeting disadvantaged pregnant women and new mothers. BACKGROUND: Adaptive experiences during pregnancy and the early years are key to reducing health inequalities in women and children worldwide. Public health nurses, health visitors and community midwives are well placed to address such disadvantage, often using community engagement strategies. Such interventions are complex; however, and we need to better understand which aspects of community engagement are aligned with effectiveness. DESIGN: Qualitative comparative analysis conducted in 2013, of trials data included in a recently published systematic review. METHODS: Two reviewers agreed on relevant conditions from 24 maternity or early years intervention studies examining four models of community engagement. Effect size estimates were converted into 'fuzzy' effectiveness categories and truth tables were constructed. Using fsQCA software, Boolean minimization identified solution sets. Random effects multiple regression and fsQCA were conducted to rule out risk of methodological bias. RESULTS/FINDINGS: Studies focused on antenatal, immunization, breastfeeding and early professional intervention outcomes. Peer delivery (consistency 0·83; unique coverage 0·63); and mother-professional collaboration (consistency 0·833; unique coverage 0·21) were moderately aligned with effective interventions. Community-identified health need plus consultation/collaboration in intervention design and leading on delivery were weakly aligned with 'not effective' interventions (consistency 0·78; unique coverage 0·29). CONCLUSIONS: For disadvantaged new and expectant mothers, peer or collaborative delivery models could be used in interventions. A need exists to design and test community engagement interventions in other areas of maternity and early years care and to further evaluate models of empowerment

Context and Implications Document for: Ongoing developments in meta-analytic and quantitative synthesis methods: Broadening the types of research questions that can be addressed

This guide accompanies the following article: Alison O’Mara-Eves and James Thomas, Ongoingdevelopments in meta-analytic and quantitative synthesis methods: Broadening the types ofresearch questions that can be addressed, Review of Education, Volume 4, Issue 1, pages 5–27, February 2016; DOI:10.1002/rev3.306

Ongoing developments in meta-analytic and quantitative synthesis methods: Broadening the types of research questions that can be addressed

The purpose of this paper is to outline ongoing developments in meta-analytic methods and quantitative approaches to synthesising evidence. We discuss the increased awareness by meta-analysts of the need for methods which better grapple with complex social contexts, and meta-analysts’ responses to the increasing informational needs of review audiences by developing methods are that are more fit for purpose and fit for use than their predecessors. Models of meta-analysis that we cover include both aggregative (e.g., classical meta-analysis) and configurative approaches (including subgroup analyses, meta-regression and multilevel analyses, multiple outcome analyses, and network meta-analysis). We then consider the role of additional data sources and multi-method approaches to synthesis by focusing on mixed methods synthesis, the use of largescale datasets and individual participant data, and qualitative comparative analysis. We highlight key issues for meta-analysis in educational research (publication bias and interpreting meta-analytic results). We end with reflections on the relation between meta-analytic methods and theory, and a discussion of how meta-analysis in education can move forward. Throughout, we place a particular emphasis on the importance of using a method that is appropriate for the research question, and how emerging methods allow us to address a broader range of research questions

School closure in response to epidemic outbreaks: Systems-based logic model of downstream impacts [version 1; peer review: 2 approved]

Background: School closures have been a recommended non-pharmaceutical intervention in pandemic response owing to the potential to reduce transmission of infection between children, school staff and those that they contact. However, given the many roles that schools play in society, closure for any extended period is likely to have additional impacts. Literature reviews of research exploring school closure to date have focused upon epidemiological effects; there is an unmet need for research that considers the multiplicity of potential impacts of school closures. Methods: We used systematic searching, coding and synthesis techniques to develop a systems-based logic model. We included literature related to school closure planned in response to epidemics large and small, spanning the 1918-19 ‘flu pandemic through to the emerging literature on the 2019 novel coronavirus. We used over 170 research studies and a number of policy documents to inform our model. Results: The model organises the concepts used by authors into seven higher level domains: children’s health and wellbeing, children’s education, impacts on teachers and other school staff, the school organisation, considerations for parents and families, public health considerations, and broader economic impacts. The model also collates ideas about potential moderating factors and ethical considerations. While dependent upon the nature of epidemics experienced to date, we aim for the model to provide a starting point for theorising about school closures in general, and as part of a wider system that is influenced by contextual and population factors. Conclusions: The model highlights that the impacts of school closures are much broader than those related solely to health, and demonstrates that there is a need for further concerted work in this area. The publication of this logic model should help to frame future research in this area and aid decision-makers when considering future school closure policy and possible mitigation strategies

Interpretive analysis of 85 systematic reviews suggests narrative syntheses and meta-analyses are incommensurate in argumentation

Introduction. Using Toulmin’s argumentation theory, we analysed the texts of systematic reviews in the area of workplace health promotion to explore differences in the modes of reasoning embedded in reports of narrative synthesis as compared to reports of meta-analysis. Methods. We used framework synthesis, grounded theory and cross-case analysis methods to analyse 85 systematic reviews addressing intervention effectiveness in workplace health promotion. Results. Two core categories, or ‘modes of reasoning’, emerged to frame the contrast between narrative synthesis and meta-analysis: practical-configurational reasoning in narrative synthesis (‘what is going on here? what picture emerges?’) and inferential-predictive reasoning in meta-analysis (‘does it work, and how well? will it work again?’). Modes of reasoning examined quality and consistency of the included evidence differently. Meta-analyses clearly distinguished between warrant and claim, whereas narrative syntheses often presented joint warrant-claims. Conclusion. Narrative syntheses and meta-analyses represent different modes of reasoning. Systematic reviewers are likely to be addressing research questions in different ways with each method. It is important to consider narrative synthesis in its own right as a method and to develop specific quality criteria and understandings of how it is done, not merely as a complement to, or second-best option for, meta-analysis

The Value of Co-Production Research Project: A Rapid Critical Review of the Evidence

What are the different types of co-production within research and what are their value(s)

The effectiveness of community engagement in public health interventions for disadvantaged groups: a meta-analysis

BACKGROUND: Inequalities in health are acknowledged in many developed countries, whereby disadvantaged groups systematically suffer from worse health outcomes such as lower life expectancy than non-disadvantaged groups. Engaging members of disadvantaged communities in public health initiatives has been suggested as a way to reduce health inequities. This systematic review was conducted to evaluate the effectiveness of public health interventions that engage the community on a range of health outcomes across diverse health issues. METHODS: We searched the following sources for systematic reviews of public health interventions: Cochrane CDSR and CENTRAL, Campbell Library, DARE, NIHR HTA programme website, HTA database, and DoPHER. Through the identified reviews, we collated a database of primary studies that appeared to be relevant, and screened the full-text documents of those primary studies against our inclusion criteria. In parallel, we searched the NHS EED and TRoPHI databases for additional primary studies. For the purposes of these analyses, study design was limited to randomised and non-randomised controlled trials. Only interventions conducted in OECD countries and published since 1990 were included. We conducted a random effects meta-analysis of health behaviour, health consequences, self-efficacy, and social support outcomes, and a narrative summary of community outcomes. We tested a range of moderator variables, with a particular emphasis on the model of community engagement used as a potential moderator of intervention effectiveness. RESULTS: Of the 9,467 primary studies scanned, we identified 131 for inclusion in the meta-analysis. The overall effect size for health behaviour outcomes is d = .33 (95% CI .26, .40). The interventions were also effective in increasing health consequences (d = .16, 95% CI .06, .27); health behaviour self-efficacy (d = .41, 95% CI .16, .65) and perceived social support (d = .41, 95% CI .23, .65). Although the type of community engagement was not a significant moderator of effect, we identified some trends across studies. CONCLUSIONS: There is solid evidence that community engagement interventions have a positive impact on a range of health outcomes across various conditions. There is insufficient evidence to determine whether one particular model of community engagement is more effective than any other

Plain Tobacco Packaging: A Systematic Review

(From the Executive Summary): This systematic review outlines findings from 37 studies that provide evidence of the impacts of plain tobacco packaging. The review was conducted following the publication of the March 2011 White Paper Healthy Lives: Healthy People which set out a renewed Tobacco Control Plan for England. One of the key actions identified in the plan was to consult on possible options to reduce the promotional impact of tobacco packaging, including plain packaging. This systematic review was commissioned to provide a comprehensive overview of evidence on the impact of plain packaging in order to inform a public consultation on the issue