The challenges of analyzing behavioral response study data : an overview of the MOCHA (Multi-study OCean acoustics Human effects Analysis) project

Date of Acceptance:This paper describes the MOCHA project which aims to develop novel approaches for the analysis of data collected during Behavioral Response Studies (BRSs). BRSs are experiments aimed at directly quantifying the effects of controlled dosages of natural or anthropogenic stimuli (typically sound) on marine mammal behavior. These experiments typically result in low sample size, relative to variability, and so we are looking at a number of studies in combination to maximize the gain from each one. We describe a suite of analytical tools applied to BRS data on beaked whales, including a simulation study aimed at informing future experimental design.Postprin

Dealing with missing standard deviation and mean values in meta-analysis of continuous outcomes: a systematic review

Background: Rigorous, informative meta-analyses rely on availability of appropriate summary statistics or individual participant data. For continuous outcomes, especially those with naturally skewed distributions, summary information on the mean or variability often goes unreported. While full reporting of original trial data is the ideal, we sought to identify methods for handling unreported mean or variability summary statistics in meta-analysis. Methods: We undertook two systematic literature reviews to identify methodological approaches used to deal with missing mean or variability summary statistics. Five electronic databases were searched, in addition to the Cochrane Colloquium abstract books and the Cochrane Statistics Methods Group mailing list archive. We also conducted cited reference searching and emailed topic experts to identify recent methodological developments. Details recorded included the description of the method, the information required to implement the method, any underlying assumptions and whether the method could be readily applied in standard statistical software. We provided a summary description of the methods identified, illustrating selected methods in example meta-analysis scenarios. Results: For missing standard deviations (SDs), following screening of 503 articles, fifteen methods were identified in addition to those reported in a previous review. These included Bayesian hierarchical modelling at the meta-analysis level; summary statistic level imputation based on observed SD values from other trials in the meta-analysis; a practical approximation based on the range; and algebraic estimation of the SD based on other summary statistics. Following screening of 1124 articles for methods estimating the mean, one approximate Bayesian computation approach and three papers based on alternative summary statistics were identified. Illustrative meta-analyses showed that when replacing a missing SD the approximation using the range minimised loss of precision and generally performed better than omitting trials. When estimating missing means, a formula using the median, lower quartile and upper quartile performed best in preserving the precision of the meta-analysis findings, although in some scenarios, omitting trials gave superior results. Conclusions: Methods based on summary statistics (minimum, maximum, lower quartile, upper quartile, median) reported in the literature facilitate more comprehensive inclusion of randomised controlled trials with missing mean or variability summary statistics within meta-analyses

Extent, Awareness and Perception of Dissemination Bias in Qualitative Research: An Explorative Survey

BACKGROUND: Qualitative research findings are increasingly used to inform decision-making. Research has indicated that not all quantitative research on the effects of interventions is disseminated or published. The extent to which qualitative researchers also systematically underreport or fail to publish certain types of research findings, and the impact this may have, has received little attention. METHODS: A survey was delivered online to gather data regarding non-dissemination and dissemination bias in qualitative research. We invited relevant stakeholders through our professional networks, authors of qualitative research identified through a systematic literature search, and further via snowball sampling. RESULTS: 1032 people took part in the survey of whom 859 participants identified as researchers, 133 as editors and 682 as peer reviewers. 68.1% of the researchers said that they had conducted at least one qualitative study that they had not published in a peer-reviewed journal. The main reasons for non-dissemination were that a publication was still intended (35.7%), resource constraints (35.4%), and that the authors gave up after the paper was rejected by one or more journals (32.5%). A majority of the editors and peer reviewers "(strongly) agreed" that the main reasons for rejecting a manuscript of a qualitative study were inadequate study quality (59.5%; 68.5%) and inadequate reporting quality (59.1%; 57.5%). Of 800 respondents, 83.1% "(strongly) agreed" that non-dissemination and possible resulting dissemination bias might undermine the willingness of funders to support qualitative research. 72.6% and 71.2%, respectively, "(strongly) agreed" that non-dissemination might lead to inappropriate health policy and health care. CONCLUSIONS: The proportion of non-dissemination in qualitative research is substantial. Researchers, editors and peer reviewers play an important role in this. Non-dissemination and resulting dissemination bias may impact on health care research, practice and policy. More detailed investigations on patterns and causes of the non-dissemination of qualitative research are needed

Reporting of clinical trials: a review of research funders' guidelines

BACKGROUND: Randomised controlled trials (RCTs) represent the gold standard methodological design to evaluate the effectiveness of an intervention in humans but they are subject to bias, including study publication bias and outcome reporting bias. National and international organisations and charities give recommendations for good research practice in relation to RCTs but to date no review of these guidelines has been undertaken with respect to reporting bias. METHODS: National and international organisations and UK based charities listed on the Association for Medical Research Charities website were contacted in 2007; they were considered eligible for this review if they funded RCTs. Guidelines were obtained and assessed in relation to what was written about trial registration, protocol adherence and trial publication. It was also noted whether any monitoring against these guidelines was undertaken. This information was necessary to discover how much guidance researchers are given on the publication of results, in order to prevent study publication bias and outcome reporting bias. RESULTS: Seventeen organisations and 56 charities were eligible of 140 surveyed for this review, although there was no response from 12. Trial registration, protocol adherence, trial publication and monitoring against the guidelines were often explicitly discussed or implicitly referred too. However, only eleven of these organisations or charities mentioned the publication of negative as well as positive outcomes and just three of the organisations specifically stated that the statistical analysis plan should be strictly adhered to and all changes should be reported. CONCLUSION: Our review indicates that there is a need to provide more detailed guidance for those conducting and reporting clinical trials to help prevent the selective reporting of results. Statements found in the guidelines generally refer to publication bias rather than outcome reporting bias. Current guidelines need to be updated and include the statement that all primary and secondary outcomes prespecified in the protocol should be fully reported and should not be selected for inclusion in the final report based on their results

How do we create, and improve, the evidence base?

Providing best clinical care involves using the best available evidence of effectiveness to inform treatment decisions. Producing this evidence begins with trials and continues through synthesis of their findings towards evidence incorporation within comprehensible, usable guidelines, for clinicians and patients at the point of care. However, there is enormous wastage in this evidence production process, with less than 50% of the published biomedical literature considered sufficient in conduct and reporting to be fit for purpose. Over the last 30 years, independent collaborative initiatives have evolved to optimise the evidence to improve patient care. These collaborations each recommend how to improve research quality in a small way at many different stages of the evidence production and distillation process. When we consider these minimal improvements at each stage from an 'aggregation of marginal gains' perspective, the accumulation of small enhancements aggregates, thereby greatly improving the final product of 'best available evidence'. The myriad of tools to reduce research quality leakage and evidence loss should be routinely used by all those with responsibility for ensuring that research benefits patients, that is, those who pay for research (funders), produce it (researchers), take part in it (patients/participants) and use it (clinicians, policy makers and service commissioners)

Moderators of the effect of psychological interventions on depression and anxiety in cardiac surgery patients: A systematic review and meta-analysis

Cardiac surgery patients may be provided with psychological interventions to counteract depression and anxiety associated with surgical procedures. This systematic review and meta-analysis investigated whether intervention efficacy was impacted by type of cardiac procedure/cardiac event; control condition content; intervention duration; intervention timing; facilitator type; and risk of bias. MEDLINE, EMBASE, and PsycINFO were searched for randomized controlled trials comparing anxiety and depression outcomes, pre and post psychological and cardiac interventions. Twenty-four studies met the inclusion criteria for the systematic review (N = 2718) and 16 of those were meta-analysed (N = 1928). Depression and anxiety outcomes were reduced more in interventions that lasted longer, were delivered post-surgery, and by trained health professionals. Depression (but not anxiety) was reduced more when the experimental intervention was compared to an ‘alternative’ intervention, and when the intervention was delivered to coronary artery bypass graft patients. Anxiety (but not depression) was decreased more when interventions were delivered to implantable cardioverter defibrillator patients, and were of ‘high’ or ‘unclear’ risk of bias. In addition to estimating efficacy, future work in this domain needs to take into account the moderating effects of intervention, sample, and study characteristics

Premature Discontinuation of Prospective Clinical Studies Approved by a Research Ethics Committee - A Comparison of Randomised and Non-Randomised Studies.

Premature discontinuation of clinical studies affects about 25% of randomised controlled trials (RCTs) which raises concerns about waste of scarce resources for research. The risk of discontinuation of non-randomised prospective studies (NPSs) is yet unclear. To compare the proportion of discontinued studies between NPSs and RCTs that received ethical approval. We systematically surveyed prospective longitudinal clinical studies that were approved by a single REC in Freiburg, Germany between 2000 and 2002. We collected study characteristics, identified subsequent publications, and surveyed investigators to elucidate whether a study was discontinued and, if so, why. Of 917 approved studies, 547 were prospective longitudinal studies (306 RCTs and 241 NPSs). NPSs were on average smaller than RCTs, more frequently single centre and pilot studies, and less frequently funded by industry. NPSs were less frequently discontinued than RCTs: 32/221 (14%) versus 78/288 (27%, p<0.001, missing data excluded). Poor recruitment was the most frequent reason for discontinuation in both NPSs (36%) and RCTs (37%). Compared to RCTs, NPSs were at lower risk for discontinuation. Measures to reliably predict, sustain, and stimulate recruitment could prevent discontinuation of many RCTs but also of some NPSs

Systematic review of outcome domains and instruments used in clinical trials of tinnitus treatments in adults

BACKGROUND: There is no evidence-based guidance to facilitate design decisions for confirmatory trials or systematic reviews investigating treatment efficacy for adults with tinnitus. This systematic review therefore seeks to ascertain the current status of trial designs by identifying and evaluating the reporting of outcome domains and instruments in the treatment of adults with tinnitus. METHODS: Records were identified by searching PubMed, EMBASE CINAHL, EBSCO, and CENTRAL clinical trial registries (ClinicalTrials.gov, ISRCTN, ICTRP) and the Cochrane Database of Systematic Reviews. Eligible records were those published from 1 July 2006 to 12 March 2015. Included studies were those reporting adults aged 18 years or older who reported tinnitus as a primary complaint, and who were enrolled into a randomised controlled trial, a before and after study, a non-randomised controlled trial, a case-controlled study or a cohort study, and written in English. Studies with fewer than 20 participants were excluded. RESULTS: Two hundred and twenty-eight studies were included. Thirty-five different primary outcome domains were identified spanning seven categories (tinnitus percept, impact of tinnitus, co-occurring complaints, quality of life, body structures and function, treatment-related outcomes and unclear or not specified). Over half the studies (55 %) did not clearly define the complaint of interest. Tinnitus loudness was the domain most often reported (14 %), followed by tinnitus distress (7 %). Seventy-eight different primary outcome instruments were identified. Instruments assessing multiple attributes of the impact of tinnitus were most common (34 %). Overall, 24 different patient-reported tools were used, predominantly the Tinnitus Handicap Inventory (15 %). Loudness was measured in diverse ways including a numerical rating scale (8 %), loudness matching (4 %), minimum masking level (1 %) and loudness discomfort level (1 %). Ten percent of studies did not clearly report the instrument used. CONCLUSIONS: Our findings indicate poor appreciation of the basic principles of good trial design, particularly the importance of specifying what aspect of therapeutic benefit is the main outcome. No single outcome was reported in all studies and there was a broad diversity of outcome instruments. PROSPERO REGISTRATION: The systematic review protocol is registered on PROSPERO (International Prospective Register of Systematic Reviews): CRD42015017525. Registered on 12 March 2015 revised on 15 March 2016. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13063-016-1399-9) contains supplementary material, which is available to authorized users

Impact of technology-based interventions for children and young people with type 1 diabetes on key diabetes self-management behaviours and prerequisites: A systematic review

Background The role of technology in the self-management of type 1 diabetes mellitus (T1DM) among children and young people is not well understood. Interventions should aim to improve key diabetes self-management behaviours (self-management of blood glucose, insulin administration, physical activity and dietary behaviours) and prerequisites (psychological outcomes and HbA1c) highlighted in the UK guidelines of the National Institute for Health and Care Excellence (NICE) for management of T1DM. The purpose was to identify evidence to assess the effectiveness of technological tools in promoting aspects of these guidelines amongst children and young people. Methods A systematic review of English language articles was conducted using the following databases: Web of Science, PubMed, Scopus, NUSearch, SAGE Journals, SpringerLink, Google Scholar, Science Direct, Sport Discus, Embase, Psychinfo and Cochrane Trials. Search terms included paediatric, type one diabetes, technology, intervention and various synonyms. Included studies examined interventions which supplemented usual care with a health care strategy primarily delivered through a technology-based medium (e.g. mobile phone, website, activity monitor) with the aim of engaging children and young people with T1DM directly in their diabetes healthcare. Studies did not need to include a comparator condition and could be randomised, non-randomised or cohort studies but not single-case studies. Results Of 30 included studies (21 RCTs), the majority measured self-monitoring of blood glucose monitoring (SMBG) frequency, clinical indicators of diabetes self-management (e.g. HbA1c) and/or psychological or cognitive outcomes. The most positive findings were associated with technology-based health interventions targeting SMBG as a behavioural outcome, with some benefits found for clinical and/or psychological diabetes self-management outcomes. Technological interventions were well accepted by children and young people. For the majority of included outcomes, clinical relevance was deemed to be little or none. Conclusions More research is required to assess which elements of interventions are most likely to produce beneficial behavioural outcomes. To produce clinically relevant outcomes, interventions may need to be delivered for at least 1 year and should consider targeting individuals with poorly managed diabetes. It is not possible to determine the impact of technology-based interventions on insulin administration, dietary habits and/or physical activity behaviour due to lack of evidence

Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries

BACKGROUND: The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decision-making. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peer-reviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries. METHODS AND FINDINGS: Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%-52.4%, I2 = 94.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%-65.9%, I2 = 98.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2-3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6-2.5). The probability of publication within two years after study completion ranged from 7% to 30%. CONCLUSIONS: A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased