Barriers and opportunities for evidence-based health service planning: the example of developing a Decision Analytic Model to plan services for sexually transmitted infections in the UK

Decision Analytic Models (DAMs) are established means of evidence-synthesis to differentiate between health interventions. They have mainly been used to inform clinical decisions and health technology assessment at the national level, yet could also inform local health service planning. For this, a DAM must take into account the needs of the local population, but also the needs of those planning its services. Drawing on our experiences from stakeholder consultations, where we presented the potential utility of a DAM for planning local health services for sexually transmitted infections (STIs) in the UK, and the evidence it could use to inform decisions regarding different combinations of service provision, in terms of their costs, cost-effectiveness, and public health outcomes, we discuss the barriers perceived by stakeholders to the use of DAMs to inform service planning for local populations, including (1) a tension between individual and population perspectives; (2) reductionism; and (3) a lack of transparency regarding models, their assumptions, and the motivations of those generating models

Standardizing effect size from linear regression models with log-transformed variables for meta-analysis

Background: Meta-analysis is very useful to summarize the effect of a treatment or a risk factor for a given disease. Often studies report results based on log-transformed variables in order to achieve the principal assumptions of a linear regression model. If this is the case for some, but not all studies, the effects need to be homogenized. Methods: We derived a set of formulae to transform absolute changes into relative ones, and vice versa, to allow including all results in a meta-analysis. We applied our procedure to all possible combinations of log-transformed independent or dependent variables. We also evaluated it in a simulation based on two variables either normally or asymmetrically distributed. Results: In all the scenarios, and based on different change criteria, the effect size estimated by the derived set of formulae was equivalent to the real effect size. To avoid biased estimates of the effect, this procedure should be used with caution in the case of independent variables with asymmetric distributions that significantly differ from the normal distribution. We illustrate an application of this procedure by an application to a meta-analysis on the potential effects on neurodevelopment in children exposed to arsenic and manganese. Conclusions: The procedure proposed has been shown to be valid and capable of expressing the effect size of a linear regression model based on different change criteria in the variables. Homogenizing the results from different studies beforehand allows them to be combined in a meta-analysis, independently of whether the transformations had been performed on the dependent and/or independent variables

Developing evidence-based clinical practice guidelines in hospitals in Australia, Indonesia, Malaysia, the Philippines and Thailand: values, requirements and barriers

<p>Abstract</p> <p>Background</p> <p>Evidence-based clinical practice guidelines support clinical decision-making by making recommendations to guide clinical practice. These recommendations are developed by integrating the expertise of a multidisciplinary group of clinicians with the perspectives of consumers and the best available research evidence. However studies have raised concerns about the quality of guideline development, and particularly the link between research and recommendations. The reasons why guideline developers are not following the established development methods are not clear.</p> <p>We aimed to explore the barriers to developing evidence-based guidelines in eleven hospitals in Australia, Indonesia, Malaysia, the Philippines and Thailand, so as to better understand how evidence-based guideline development could be facilitated in these settings. The research aimed to identify the value clinicians place on guidelines, what clinicians want in guidelines developed in hospital settings and what factors limit rigorous evidence-based guideline development in these settings.</p> <p>Methods</p> <p>Semi-structured, face-to-face interviews were undertaken with senior and junior healthcare providers (nurses, midwives, doctors, allied health) from the maternal and neonatal services of the eleven participating hospitals. Interviews were audio-recorded, transcribed and a thematic analysis undertaken.</p> <p>Results</p> <p>Ninety-three individual, 25 pair and eleven group interviews were conducted. Participants were clear that they want guidelines that are based on evidence and updated regularly. They were also clear that there are major barriers to this. Most of the barriers were shared across countries, and included lack of time, lack of skills in finding, appraising and interpreting evidence, lack of access to relevant evidence and difficulty arranging meetings and achieving consensus.</p> <p>Barriers that were primarily identified in Australian hospitals include cumbersome organisational processes and a feeling that guidelines are being developed for bureaucratic ends. Barriers that were primarily identified in South East Asian hospitals include difficulty accessing evidence due to limited resources available for computers, internet and journal subscriptions and limited skills in computing and English.</p> <p>Conclusions</p> <p>The clinicians in these eleven very different hospitals want evidence-based guidelines. However they are frustrated by guideline development processes that are enormously time, skill and resource intensive. They feel strongly that "there's got to be a better way".</p> <p>The fact that the great majority of the identified barriers were shared across settings may provide an opportunity to develop a more pragmatic way of developing guidelines that can be applied in many contexts.</p

Representation and misrepresentation of scientific evidence in contemporary tobacco regulation:a review of tobacco industry submissions to the UK Government consultation on standardised packaging

BACKGROUND: Standardised packaging (SP) of tobacco products is an innovative tobacco control measure opposed by transnational tobacco companies (TTCs) whose responses to the UK government's public consultation on SP argued that evidence was inadequate to support implementing the measure. The government's initial decision, announced 11 months after the consultation closed, was to wait for 'more evidence', but four months later a second 'independent review' was launched. In view of the centrality of evidence to debates over SP and TTCs' history of denying harms and manufacturing uncertainty about scientific evidence, we analysed their submissions to examine how they used evidence to oppose SP. METHODS AND FINDINGS: We purposively selected and analysed two TTC submissions using a verification-oriented cross-documentary method to ascertain how published studies were used and interpretive analysis with a constructivist grounded theory approach to examine the conceptual significance of TTC critiques. The companies' overall argument was that the SP evidence base was seriously flawed and did not warrant the introduction of SP. However, this argument was underpinned by three complementary techniques that misrepresented the evidence base. First, published studies were repeatedly misquoted, distorting the main messages. Second, 'mimicked scientific critique' was used to undermine evidence; this form of critique insisted on methodological perfection, rejected methodological pluralism, adopted a litigation (not scientific) model, and was not rigorous. Third, TTCs engaged in 'evidential landscaping', promoting a parallel evidence base to deflect attention from SP and excluding company-held evidence relevant to SP. The study's sample was limited to sub-sections of two out of four submissions, but leaked industry documents suggest at least one other company used a similar approach. CONCLUSIONS: The TTCs' claim that SP will not lead to public health benefits is largely without foundation. The tools of Better Regulation, particularly stakeholder consultation, provide an opportunity for highly resourced corporations to slow, weaken, or prevent public health policies

Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial

BACKGROUND: Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. METHODOLOGY AND PRINCIPAL FINDINGS: Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with “no statistical expert” and “no checklist” as controls. The two interventions were crossed in a 2×2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6–24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: −0.3–+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3–6.7), showing a significant improvement in quality. CONCLUSIONS AND SIGNIFICANCE: This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines

Multi-centred mixed-methods PEPFAR HIV care & support public health evaluation: study protocol

BACKGROUND: A public health response is essential to meet the multidimensional needs of patients and families affected by HIV disease in sub-Saharan Africa. In order to appraise current provision of HIV care and support in East Africa, and to provide evidence-based direction to future care programming, and Public Health Evaluation was commissioned by the PEPFAR programme of the US Government. METHODS/DESIGN: This paper described the 2-Phase international mixed methods study protocol utilising longitudinal outcome measurement, surveys, patient and family qualitative interviews and focus groups, staff qualitative interviews, health economics and document analysis. Aim 1) To describe the nature and scope of HIV care and support in two African countries, including the types of facilities available, clients seen, and availability of specific components of care [Study Phase 1]. Aim 2) To determine patient health outcomes over time and principle cost drivers [Study Phase 2]. The study objectives are as follows. 1) To undertake a cross-sectional survey of service configuration and activity by sampling 10% of the facilities being funded by PEPFAR to provide HIV care and support in Kenya and Uganda (Phase 1) in order to describe care currently provided, including pharmacy drug reviews to determine availability and supply of essential drugs in HIV management. 2) To conduct patient focus group discussions at each of these (Phase 1) to determine care received. 3) To undertake a longitudinal prospective study of 1200 patients who are newly diagnosed with HIV or patients with HIV who present with a new problem attending PEPFAR care and support services. Data collection includes self-reported quality of life, core palliative outcomes and components of care received (Phase 2). 4) To conduct qualitative interviews with staff, patients and carers in order to explore and understand service issues and care provision in more depth (Phase 2). 5) To undertake document analysis to appraise the clinical care procedures at each facility (Phase 2). 6) To determine principle cost drivers including staff, overhead and laboratory costs (Phase 2). DISCUSSION: This novel mixed methods protocol will permit transparent presentation of subsequent dataset results publication, and offers a substantive model of protocol design to measure and integrate key activities and outcomes that underpin a public health approach to disease management in a low-income setting

A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants

Background: This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined as the extent to which two or more independent reviews of the same scientific document agree. Methodology/Principal Findings: Altogether, 70 reliability coefficients (Cohen’s Kappa, intra-class correlation [ICC], and Pearson product-moment correlation [r]) from 48 studies were taken into account in the meta-analysis. The studies were based on a total of 19,443 manuscripts; on average, each study had a sample size of 311 manuscripts (minimum: 28, maximum: 1983). The results of the meta-analysis confirmed the findings of the narrative literature reviews published to date: The level of IRR (mean ICC/r 2 =.34, mean Cohen’s Kappa =.17) was low. To explain the study-to-study variation of the IRR coefficients, meta-regression analyses were calculated using seven covariates. Two covariates that emerged in the metaregression analyses as statistically significant to gain an approximate homogeneity of the intra-class correlations indicated that, firstly, the more manuscripts that a study is based on, the smaller the reported IRR coefficients are. Secondly, if the information of the rating system for reviewers was reported in a study, then this was associated with a smaller IRR coefficient than if the information was not conveyed. Conclusions/Significance: Studies that report a high level of IRR are to be considered less credible than those with a low level o

Leaders, leadership and future primary care clinical research

Background: A strong and self confident primary care workforce can deliver the highest quality care and outcomes equitably and cost effectively. To meet the increasing demands being made of it, primary care needs its own thriving research culture and knowledge base. Methods: Review of recent developments supporting primary care clinical research. Results: Primary care research has benefited from a small group of passionate leaders and significant investment in recent decades in some countries. Emerging from this has been innovation in research design and focus, although less is known of the effect on research output. Conclusion: Primary care research is now well placed to lead a broad re-vitalisation of academic medicine, answering questions of relevance to practitioners, patients, communities and Government. Key areas for future primary care research leaders to focus on include exposing undergraduates early to primary care research, integrating this early exposure with doctoral and postdoctoral research career support, further expanding cross disciplinary approaches, and developing useful measures of output for future primary care research investment

Reporting bias in medical research - a narrative review

Reporting bias represents a major problem in the assessment of health care interventions. Several prominent cases have been described in the literature, for example, in the reporting of trials of antidepressants, Class I anti-arrhythmic drugs, and selective COX-2 inhibitors. The aim of this narrative review is to gain an overview of reporting bias in the medical literature, focussing on publication bias and selective outcome reporting. We explore whether these types of bias have been shown in areas beyond the well-known cases noted above, in order to gain an impression of how widespread the problem is. For this purpose, we screened relevant articles on reporting bias that had previously been obtained by the German Institute for Quality and Efficiency in Health Care in the context of its health technology assessment reports and other research work, together with the reference lists of these articles