Unmasking the true effects of self-concept interventions and suggested guidelines for rectification

Over recent decades, traditional literature reviews have intimated that self-concept interventions have produced inconsistent or disappointing results (e.g., Marsh & Craven, 1997). It is put forth here that existing research practices, ranging from insufficient theoretical grounding to the use of inappropriate evaluation measures, have generally undermined the effectiveness of self-concept interventions to date. A brief rationale for the necessity of self-concept interventions for children and adolescents will be provided, followed by a review of self-concept enhancement research. This review will focus on the theoretical and methodological weaknesses that have typically resulted in an underestimation of the true effects of such interventions, as revealed by sophisticated meta-analytic techniques. A particular emphasis will be placed on the need for multidimensional approaches to intervention design and evaluation. Following from this, suggestions for improving self-concept intervention research will be posited, with the aim of increasing the consistency and effectiveness of such interventions, and thus unmasking the true effects of such interventions

A comprehensive multilevel model meta-analysis of self-concept interventions

The efficacy of self-concept interventions has previously been examined through traditional meta-analytic methods, and a host of moderators of intervention outcomes have been identified (O’Mara, Marsh, & Craven, 2004; Haney & Durlak, 1998; Hattie, 1992). However, traditional meta-analytic models have increasingly been criticized because they fail to account for the nested structure of effect sizes within studies, thereby violating statistical assumptions of independence. The multilevel model approach to meta-analysis takes into account the hierarchical structure of meta-analytic data, thus providing findings that are more statistically sound. Consequently, the present study applies the multilevel model technique to the analysis of the self-concept intervention literature. The overall mean effect size of .47 suggests a moderate impact of interventions on self-concept at post-test, and analyses show that intervention effects are maintained at follow-up. Other moderators examined include the construct validity approach to the multidimensionality of self-concept; the focus of the intervention on self-concept; the use of random assignment to treatment and control groups; the control group type; treatment type; and the treatment administrator. Intraclass correlations and the variance explained by each moderator model are presented to emphasise the importance of using a multilevel model approach to meta-analytic research. It is concluded that multilevel models provide a more accurate understanding of the self-concept intervention literature than traditional meta-analytic models. Suggestions for future self-concept intervention design and evaluation are provided

Interpretive analysis of 85 systematic reviews suggests narrative syntheses and meta-analyses are incommensurate in argumentation

Introduction. Using Toulmin’s argumentation theory, we analysed the texts of systematic reviews in the area of workplace health promotion to explore differences in the modes of reasoning embedded in reports of narrative synthesis as compared to reports of meta-analysis. Methods. We used framework synthesis, grounded theory and cross-case analysis methods to analyse 85 systematic reviews addressing intervention effectiveness in workplace health promotion. Results. Two core categories, or ‘modes of reasoning’, emerged to frame the contrast between narrative synthesis and meta-analysis: practical-configurational reasoning in narrative synthesis (‘what is going on here? what picture emerges?’) and inferential-predictive reasoning in meta-analysis (‘does it work, and how well? will it work again?’). Modes of reasoning examined quality and consistency of the included evidence differently. Meta-analyses clearly distinguished between warrant and claim, whereas narrative syntheses often presented joint warrant-claims. Conclusion. Narrative syntheses and meta-analyses represent different modes of reasoning. Systematic reviewers are likely to be addressing research questions in different ways with each method. It is important to consider narrative synthesis in its own right as a method and to develop specific quality criteria and understandings of how it is done, not merely as a complement to, or second-best option for, meta-analysis

Conceptual and statistical analysis of complex interventions in the presence of confounding variables: An example from public health

Background: Meta-analyses of complex interventions are challenging because causality operates through multiple paths and confounding variables can be difficult to distinguish. Objectives: To meta-analyse public health interventions that engage members of the community in their conception, design, or delivery. To disentangle intervention complexity by analysing according to their theories of change. Study selection criteria: Published after 1990; outcome or process evaluation; community engagement intervention; written in English; reported health or community outcomes; study populations or differential impacts reported according to social determinants of health. Analysis: Intervention complexity was examined by conceptualising, operationalising, and mapping their theories of change; and through random effects subgroup analyses. Main results: 131 studies were included in the synthesis. Three main theories of change were identified, which were useful in describing trends in intervention effectiveness. Statistically significant between-group differences were not detected, since there were likely to have been too many confounding variables. Conclusions: Intervention complexity in systematic reviews can be addressed through examining theories of change and trends in effect size estimates. Such complexity appears to defy current meta-analytical methods when confounding variables undermine analysis of variance

Prioritising references for systematic reviews with RobotAnalyst: A user study

Screening references is a time-consuming step necessary for systematic reviews and guideline development. Previous studies have shown that human effort can be reduced by using machine learning software to prioritise large reference collections such that most of the relevant references are identified before screening is completed. We describe and evaluate RobotAnalyst, a Web-based software system that combines text-mining and machine learning algorithms for organising references by their content and actively prioritising them based on a relevancy classification model trained and updated throughout the process. We report an evaluation over 22 reference collections (most are related to public health topics) screened using RobotAnalyst with a total of 43 610 abstract-level decisions. The number of references that needed to be screened to identify 95% of the abstract-level inclusions for the evidence review was reduced on 19 of the 22 collections. Significant gains over random sampling were achieved for all reviews conducted with active prioritisation, as compared with only two of five when prioritisation was not used. RobotAnalyst's descriptive clustering and topic modelling functionalities were also evaluated by public health analysts. Descriptive clustering provided more coherent organisation than topic modelling, and the content of the clusters was apparent to the users across a varying number of clusters. This is the first large-scale study using technology-assisted screening to perform new reviews, and the positive results provide empirical evidence that RobotAnalyst can accelerate the identification of relevant studies. The results also highlight the issue of user complacency and the need for a stopping criterion to realise the work savings

Are medical procedures that induce coughing or involve respiratory suctioning associated with increased generation of aerosols and risk of SARS-CoV-2 infection? A rapid systematic review.

BACKGROUND: The risk of transmission of SARS-CoV-2 from aerosols generated by medical procedures is a cause for concern. AIM: To evaluate the evidence for aerosol production and transmission of respiratory infection associated with procedures that involve airway suctioning or induce coughing/sneezing. METHODS: The review was informed by PRISMA guidelines. Searches were conducted in PubMed for studies published between January 1st, 2003 and October 6th, 2020. Included studies examined whether nasogastric tube insertion, lung function tests, nasendoscopy, dysphagia assessment, or suctioning for airway clearance result in aerosol generation or transmission of SARS-CoV-2, SARS-CoV, MERS, or influenza. Risk of bias assessment focused on robustness of measurement, control for confounding, and applicability to clinical practice. FINDINGS: Eighteen primary studies and two systematic reviews were included. Three epidemiological studies found no association between nasogastric tube insertion and acquisition of respiratory infections. One simulation study found low/very low production of aerosols associated with pulmonary lung function tests. Seven simulation studies of endoscopic sinus surgery suggested significant increases in aerosols but findings were inconsistent; two clinical studies found airborne particles associated with the use of microdebriders/drills. Some simulation studies did not use robust measures to detect particles and are difficult to equate to clinical conditions. CONCLUSION: There was an absence of evidence to suggest that the procedures included in the review were associated with an increased risk of transmission of respiratory infection. In order to better target precautions to mitigate risk, more research is required to determine the characteristics of medical procedures and patients that increase the risk of transmission of SARS-CoV-2

Pancreatic cancer in type 1 and young-onset diabetes: systematic review and meta-analysis

We conducted a systematic review of the risk of pancreatic cancer in people with type I and young-onset diabetes. In three cohort and six case–control studies, the relative risk for pancreatic cancer in people with (vs without) diabetes was 2.00 (95% confidence interval 1.37–3.01) based on 39 cases with diabetes

Cancer incidence and mortality in patients with insulin-treated diabetes: a UK cohort study

Raised risks of several cancers have been found in patients with type II diabetes, but there are few data on cancer risk in type I diabetes. We conducted a cohort study of 28 900 UK patients with insulin-treated diabetes followed for 520 517 person-years, and compared their cancer incidence and mortality with national expectations. To analyse by diabetes type, we examined risks separately in 23 834 patients diagnosed with diabetes under the age of 30 years, who will almost all have had type I diabetes, and 5066 patients diagnosed at ages 30–49 years, who probably mainly had type II. Relative risks of cancer overall were close to unity, but ovarian cancer risk was highly significantly raised in patients with diabetes diagnosed under age 30 years (standardised incidence ratio (SIR)=2.14; 95% confidence interval (CI) 1.22–3.48; standardised mortality ratio (SMR)=2.90; 95% CI 1.45–5.19), with greatest risks for those with diabetes diagnosed at ages 10–19 years. Risks of cancer at other major sites were not substantially raised for type I patients. The excesses of obesity- and alcohol-related cancers in type II diabetes may be due to confounding rather than diabetes per se

A Unified Model of the GABA(A) Receptor Comprising Agonist and Benzodiazepine Binding Sites

We present a full-length α(1)β(2)γ(2) GABA receptor model optimized for agonists and benzodiazepine (BZD) allosteric modulators. We propose binding hypotheses for the agonists GABA, muscimol and THIP and for the allosteric modulator diazepam (DZP). The receptor model is primarily based on the glutamate-gated chloride channel (GluCl) from C. elegans and includes additional structural information from the prokaryotic ligand-gated ion channel ELIC in a few regions. Available mutational data of the binding sites are well explained by the model and the proposed ligand binding poses. We suggest a GABA binding mode similar to the binding mode of glutamate in the GluCl X-ray structure. Key interactions are predicted with residues α(1)R66, β(2)T202, α(1)T129, β(2)E155, β(2)Y205 and the backbone of β(2)S156. Muscimol is predicted to bind similarly, however, with minor differences rationalized with quantum mechanical energy calculations. Muscimol key interactions are predicted to be α(1)R66, β(2)T202, α(1)T129, β(2)E155, β(2)Y205 and β(2)F200. Furthermore, we argue that a water molecule could mediate further interactions between muscimol and the backbone of β(2)S156 and β(2)Y157. DZP is predicted to bind with interactions comparable to those of the agonists in the orthosteric site. The carbonyl group of DZP is predicted to interact with two threonines α(1)T206 and γ(2)T142, similar to the acidic moiety of GABA. The chlorine atom of DZP is placed near the important α(1)H101 and the N-methyl group near α(1)Y159, α(1)T206, and α(1)Y209. We present a binding mode of DZP in which the pending phenyl moiety of DZP is buried in the binding pocket and thus shielded from solvent exposure. Our full length GABA(A) receptor is made available as Model S1