Methodological quality of test accuracy studies included in systematic reviews in obstetrics and gynaecology: sources of bias

<p>Abstract</p> <p>Background</p> <p>Obstetrics and gynaecology have seen rapid growth in the development of new tests with research on these tests presented as diagnostic accuracy studies. To avoid errors in judgement it is important that the methodology of these studies is such that bias is minimised. Our objective was to determine the methodological quality of test accuracy studies in obstetrics and gynaecology using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) checklist and to assess sources of bias.</p> <p>Methods</p> <p>A prospective protocol was developed to assess the impact of QUADAS on ten systematic reviews performed over the period 2004-2007.We investigated whether there was an improvement in study quality since the introduction of QUADAS, whether a correlation existed between study sample size, country of origin of study and its quality. We also investigated whether there was a correlation between reporting and methodological quality and by the use of meta-regression analyses explored for items of quality that were associated with bias.</p> <p>Results</p> <p>A total of 300 studies were included. The overall quality of included studies was poor (> 50% compliance with 57.1% of quality items). However, the mean compliance with QUADAS showed an improvement post-publication of QUADAS (54.9% versus 61.4% p = 0.002). There was no correlation with study sample size. Gynaecology studies published from the United States of America showed higher quality (USA versus Western Europe p = 0.002; USA versus Asia p = 0.004). Meta-regression analysis showed that no individual quality item had a significant impact on accuracy. There was an association between reporting and methodological quality (r = 0.51 p < 0.0001 for obstetrics and r = 0.56 p < 0.0001 for gynaecology).</p> <p>Conclusions</p> <p>A combination of poor methodological quality and poor reporting affects the inferences that can be drawn from test accuracy studies. Further compliance with quality checklists is required to ensure that bias is minimised.</p

Are pediatric Open Access journals promoting good publication practice? An analysis of author instructions

<p>Abstract</p> <p>Background</p> <p>Several studies analyzed whether conventional journals in general medicine or specialties such as pediatrics endorse recommendations aiming to improve publication practice. Despite evidence showing benefits of these recommendations, the proportion of endorsing journals has been moderate to low and varied considerably for different recommendations. About half of pediatric journals indexed in the Journal Citation Report referred to the Uniform Requirements for Manuscripts of the International Committee of Medical Journal Editors (ICMJE) but only about a quarter recommended registration of trials. We aimed to investigate to what extent pediatric open-access (OA) journals endorse these recommendations. We hypothesized that a high proportion of these journals have adopted recommendations on good publication practice since OA electronic publishing has been associated with a number of editorial innovations aiming at improved access and transparency.</p> <p>Methods</p> <p>We identified 41 journals publishing original research in the subject category "Health Sciences, Medicine (General), Pediatrics" of the Directory of Open Access Journals <url>http://www.doaj.org</url>. From the journals' online author instructions we extracted information regarding endorsement of four domains of editorial policy: the Uniform Requirements for Manuscripts, trial registration, disclosure of conflicts of interest and five major reporting guidelines such as the CONSORT (Consolidated Standards of Reporting Trials) statement. Two investigators collected data independently.</p> <p>Results</p> <p>The Uniform Requirements were mentioned by 27 (66%) pediatric OA journals. Thirteen (32%) required or recommended trial registration prior to publication of a trial report. Conflict of interest policies were stated by 25 journals (61%). Advice about reporting guidelines was less frequent: CONSORT was referred to by 12 journals (29%) followed by other reporting guidelines (MOOSE, PRISMA or STARD) (8 journals, 20%) and STROBE (3 journals, 7%). The EQUATOR network, a platform of several guideline initiatives, was acknowledged by 4 journals (10%).</p> <p>Journals published by OA publishing houses gave more guidance than journals published by professional societies or other publishers.</p> <p>Conclusions</p> <p>Pediatric OA journals mentioned certain recommendations such as the Uniform Requirements or trial registration more frequently than conventional journals; however, endorsement is still only moderate. Further research should confirm these exploratory findings in other medical fields and should clarify what the motivations and barriers are in implementing such policies.</p

Development of a quality assessment tool for systematic reviews of observational studies (QATSO) of HIV prevalence in men having sex with men and associated risk behaviours

<p>Abstract</p> <p>Background</p> <p>Systematic reviews based on the critical appraisal of observational and analytic studies on HIV prevalence and risk factors for HIV transmission among men having sex with men are very useful for health care decisions and planning. Such appraisal is particularly difficult, however, as the quality assessment tools available for use with observational and analytic studies are poorly established.</p> <p>Methods</p> <p>We reviewed the existing quality assessment tools for systematic reviews of observational studies and developed a concise quality assessment checklist to help standardise decisions regarding the quality of studies, with careful consideration of issues such as external and internal validity.</p> <p>Results</p> <p>A pilot version of the checklist was developed based on epidemiological principles, reviews of study designs, and existing checklists for the assessment of observational studies. The Quality Assessment Tool for Systematic Reviews of Observational Studies (QATSO) Score consists of five items: External validity (1 item), reporting (2 items), bias (1 item) and confounding factors (1 item). Expert opinions were sought and it was tested on manuscripts that fulfil the inclusion criteria of a systematic review. Like all assessment scales, QATSO may oversimplify and generalise information yet it is inclusive, simple and practical to use, and allows comparability between papers.</p> <p>Conclusion</p> <p>A specific tool that allows researchers to appraise and guide study quality of observational studies is developed and can be modified for similar studies in the future.</p

STORIES Statement: publication standards for healthcare education evidence synthesis

Fully copy of the STORIES statement - a checklist of reporting guidance for health education evidence synthesis Structured approach for Reporting In health education of Evidence Synthesis Background Evidence synthesis techniques in healthcare education have been enhanced through the activities of experts in the field and the Best Evidence Medical Education (BEME) collaborative. Despite this, significant heterogeneity in techniques and reporting of healthcare education systematic review still exist and limit the usefulness of such reports. The aim of this project was to produce the STORIES (STructured apprOach to the Reporting In healthcare education of Evidence Synthesis) statement to offer a guide for reporting evidence synthesis in health education for use by authors and journal editors. Methods A review of existing published evidence synthesis consensus statements was undertaken. A modified Delphi process was used. In stage one, expert participants were asked to state whether common existing items identified were relevant, to suggest relevant texts and specify any items they feel should be included. The results were analysed and a second stage commenced where all synthesised items were presented and participants asked to state whether they should be included or amend as needed. After further analysis, the full statement was sent for final review and comment. Results Nineteen experts participated in the panel from 35 invitations. Thirteen text sources were proposed, six existing items amended and twelve new items synthesised. After stage two, 25 amended consensus items were proposed for inclusion. The final statement contains several items unique to this context, including description of relevant conceptual frameworks or theoretical constructs, description of qualitative methodologies with rationale for their choice and presenting the implications for educators in practice of the results obtained. Conclusions An international expert panel has agreed upon a consensus statement of 25 items for the reporting of evidence synthesis within healthcare education. This unique set of items is focused on context, rather than a specific methodology. This statement can be used for those writing for publication and reviewing such manuscripts to ensure reporting supports and best informs the wider healthcare education community

Protocol for the 'e-Nudge trial' : a randomised controlled trial of electronic feedback to reduce the cardiovascular risk of individuals in general practice [ISRCTN64828380]

Background: Cardiovascular disease (including coronary heart disease and stroke) is a major cause of death and disability in the United Kingdom, and is to a large extent preventable, by lifestyle modification and drug therapy. The recent standardisation of electronic codes for cardiovascular risk variables through the United Kingdom's new General Practice contract provides an opportunity for the application of risk algorithms to identify high risk individuals. This randomised controlled trial will test the benefits of an automated system of alert messages and practice searches to identify those at highest risk of cardiovascular disease in primary care databases. Design: Patients over 50 years old in practice databases will be randomised to the intervention group that will receive the alert messages and searches, and a control group who will continue to receive usual care. In addition to those at high estimated risk, potentially high risk patients will be identified who have insufficient data to allow a risk estimate to be made. Further groups identified will be those with possible undiagnosed diabetes, based either on elevated past recorded blood glucose measurements, or an absence of recent blood glucose measurement in those with established cardiovascular disease. Outcome measures: The intervention will be applied for two years, and outcome data will be collected for a further year. The primary outcome measure will be the annual rate of cardiovascular events in the intervention and control arms of the study. Secondary measures include the proportion of patients at high estimated cardiovascular risk, the proportion of patients with missing data for a risk estimate, and the proportion with undefined diabetes status at the end of the trial

Funding source, trial outcome and reporting quality: are they related? Results of a pilot study

BACKGROUND: There has been increasing concern regarding the potential effects of the commercialization of research. METHODS: In order to examine the relationships between funding source, trial outcome and reporting quality, recent issues of five peer-reviewed, high impact factor, general medical journals were hand-searched to identify a sample of 100 randomized controlled trials (20 trials/journal). Relevant data, including funding source (industry/not-for-profit/mixed/not reported) and statistical significance of primary outcome (favouring new treatment/favouring conventional treatment/neutral/unclear), were abstracted. Quality scores were assigned using the Jadad scale and the adequacy of allocation concealment. RESULTS: Sixty-six percent of trials received some industry funding. Trial outcome was not associated with funding source (p= .461). There was a preponderance of favourable statistical conclusions among published trials with 67% reporting results that favored a new treatment whereas 6% favoured the conventional treatment. Quality scores were not associated with funding source or trial outcome. CONCLUSIONS: It is not known whether the absence of significant associations between funding source, trial outcome and reporting quality reflects a true absence of an association or is an artefact of inadequate statistical power, reliance on voluntary disclosure of funding information, a focus on trials recently published in the top medical journals, or some combination thereof. Continued and expanded monitoring of potential conflicts is recommended, particularly in light of new guidelines for disclosure that have been endorsed by the ICMJE

REporting recommendations for tumour MARKer prognostic studies (REMARK)

Despite years of research and hundreds of reports on tumour markers in oncology, the number of markers that have emerged as clinically useful is pitifully small. Often initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of methodological problems have been cited to explain these discrepancies. Unfortunately, many tumour marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalisability of the study results. The development of guidelines for the reporting of tumour marker studies was a major recommendation of the US National Cancer Institute and the European Organisation for Research and Treatment of Cancer (NCI-EORTC) First International Meeting on Cancer Diagnostics in 2000. Similar to the successful CONSORT initiative for randomised trials and the STARD statement for diagnostic studies, we suggest guidelines to provide relevant information about the study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggest helpful presentations of data and important elements to include in discussions. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them to judge the usefulness of the data and understand the context in which the conclusions apply