Best practice in undertaking and reporting health technology assessments : Working Group 4 report

[Executive Summary] The aim of Working Group 4 has been to develop and disseminate best practice in undertaking and reporting assessments, and to identify needs for methodologic development. Health technology assessment (HTA) is a multidisciplinary activity that systematically examines the technical performance, safety, clinical efficacy, and effectiveness, cost, costeffectiveness, organizational implications, social consequences, legal, and ethical considerations of the application of a health technology (18). HTA activity has been continuously increasing over the last few years. Numerous HTA agencies and other institutions (termed in this report “HTA doers”) across Europe are producing an important and growing amount of HTA information. The objectives of HTA vary considerably between HTA agencies and other actors, from a strictly political decision making–oriented approach regarding advice on market licensure, coverage in benefits catalogue, or investment planning to information directed to providers or to the public. Although there seems to be broad agreement on the general elements that belong to the HTA process, and although HTA doers in Europe use similar principles (41), this is often difficult to see because of differences in language and terminology. In addition, the reporting of the findings from the assessments differs considerably. This reduces comparability and makes it difficult for those undertaking HTA assessments to integrate previous findings from other HTA doers in a subsequent evaluation of the same technology. Transparent and clear reporting is an important step toward disseminating the findings of a HTA; thus, standards that ensure high quality reporting may contribute to a wider dissemination of results. The EUR-ASSESS methodologic subgroup already proposed a framework for conducting and reporting HTA (18), which served as the basis for the current working group. New developments in the last 5 years necessitate revisiting that framework and providing a solid structure for future updates. Giving due attention to these methodologic developments, this report describes the current “best practice” in both undertaking and reporting HTA and identifies the needs for methodologic development. It concludes with specific recommendations and tools for implementing them, e.g., by providing the structure for English-language scientific summary reports and a checklist to assess the methodologic and reporting quality of HTA reports

The Politics of Evidence Use in Health Policy Making in Germany-the Case of Regulating Hospital Minimum Volumes.

This article examines the role of scientific evidence in informing health policy decisions in Germany, using minimum volumes policy as a case study. It argues that scientific evidence was used strategically at various stages of the policy process both by individual corporatist actors and by the Federal Joint Committee as the regulator. Minimum volumes regulation was inspired by scientific evidence suggesting a positive relationship between service volume and patient outcomes for complex surgical interventions. Federal legislation was introduced in 2002 to delegate the selection of services and the setting of volumes to corporatist decision makers. Yet, despite being represented in the Federal Joint Committee, hospitals affected by its decisions took the Committee to court to seek legal redress and prevent policy implementation. Evidence has been key to support, and challenge, decisions about minimum volumes, including in court. The analysis of the role of scientific evidence in minimum volumes regulation in Germany highlights the dynamic relationship between evidence use and the political and institutional context of health policy making, which in this case is characterized by the legislative nature of policy making, corporatism, and the role of the judiciary in reviewing policy decisions

Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries

Background: The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decisionmaking. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peerreviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries. Copyright:Methods and Findings: Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%-52.4%, I2594.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%-65.9%, I2598.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2-3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6- 2.5). The probability of publication within two years after study completion ranged from 7% to 30%.Conclusions: A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased

Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the eighth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on the synthesis and presentation of research evidence, focusing on four key questions. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We found two reviews of instruments for critically appraising systematic reviews, several studies of the importance of using extensive searches for reviews and determining when it is important to update reviews, and consensus statements about the reporting of reviews that informed our answers to the following questions. How should existing systematic reviews be critically appraised? • Because preparing systematic reviews can take over a year and require capacity and resources, existing reviews should be used when possible and updated, if needed. • Standard criteria, such as A MeaSurement Tool to Assess Reviews (AMSTAR), should be used to critically appraise existing systematic reviews, together with an assessment of the relevance of the review to the questions being asked. When and how should WHO undertake or commission new reviews? • Consideration should be given to undertaking or commissioning a new review whenever a relevant, up-to-date review of good quality is not available. • When time or resources are limited it may be necessary to undertake rapid assessments. The methods that are used to do these assessments should be reported, including important limitations and uncertainties and explicit consideration of the need and urgency of undertaking a full systematic review. • Because WHO has limited capacity for undertaking systematic reviews, reviews will often need to be commissioned when a new review is needed. Consideration should be given to establishing collaborating centres to undertake or support this work, similar to what some national organisations have done. How should the findings of systematic reviews be summarised and presented to committees responsible for making recommendations? • Concise summaries (evidence tables) of the best available evidence for each important outcome, including benefits, harms and costs, should be presented to the groups responsible for making recommendations. These should include an assessment of the quality of the evidence and a summary of the findings for each outcome. • The full systematic reviews, on which the summaries are based, should also be available to both those making recommendations and users of the recommendations. What additional information is needed to inform recommendations and how should this information be synthesised with information about effects and presented to committees? • Additional information that is needed to inform recommendations includes factors that might modify the expected effects, need (prevalence, baseline risk or status), values (the relative importance of key outcomes), costs and the availability of resources. • Any assumptions that are made about values or other factors that may vary from setting to setting should be made explicit. • For global guidelines that are intended to inform decisions in different settings, consideration should be given to using a template to assist the synthesis of information specific to a setting with the global evidence of the effects of the relevant interventions

Towards a model of talent development in physical education

Traditional conceptions of talent generally emphasise the construction of threshold values and the development of relatively unitary abilities, and this approach still dominates talent development programmes for elite sport. Most researchers on high ability, however, now favour domain-specific, multidimensional conceptions of ability that stress the development of behavioural potential and its interaction with personal and environmental characteristics. This paper presents a model of talent in physical education, drawing together findings from a wide range of literature on the realisation and inhibition of abilities, international studies of effective school-based identification and provision strategies, and a conception of the subject as an integration and realisation of different forms of ability. In presenting this model, the authors aim to redress the imbalance within the current debate from an almost total concern with out-of-school clubs and the preparation for adult elite sport, in favour of a more equitable and inclusive approach, premised upon the unique importance of mainstream, curricular physical education within any talent development scheme

Glioma imaging in Europe: A survey of 220 centres and recommendations for best clinical practice

Objectives: At a European Society of Neuroradiology (ESNR) Annual Meeting 2015 workshop, commonalities in practice, current controversies and technical hurdles in glioma MRI were discussed. We aimed to formulate guidance on MRI of glioma and determine its feasibility, by seeking information on glioma imaging practices from the European Neuroradiology community. Methods: Invitations to a structured survey were emailed to ESNR members (n=1,662) and associates (n=6,400), European national radiologists’ societies and distributed via social media. Results: Responses were received from 220 institutions (59% academic). Conventional imaging protocols generally include T2w, T2-FLAIR, DWI, and pre- and post-contrast T1w. Perfusion MRI is used widely (85.5%), while spectroscopy seems reserved for specific indications. Reasons for omitting advanced imaging modalities include lack of facility/software, time constraints and no requests. Early postoperative MRI is routinely carried out by 74% within 24–72 h, but only 17% report a percent measure of resection. For follow-up, most sites (60%) issue qualitative reports, while 27% report an assessment according to the RANO criteria. A minori

Bias in dissemination of clinical research findings: Structured OPEN framework of what, who and why, based on literature review and expert consensus

Objective: The aim of this study is to review highly cited articles that focus on non-publication of studies, and to develop a consistent and comprehensive approach to defining (non-) dissemination of research findings. Setting: We performed a scoping review of definitions of the term 'publication bias' in highly cited publications. Participants: Ideas and experiences of a core group of authors were collected in a draft document, which was complemented by the findings from our literature search. Interventions: The draft document including findings from the literature search was circulated to an international group of experts and revised until no additional ideas emerged and consensus was reached. Primary outcomes: We propose a new approach to the comprehensive conceptualisation of (non-) dissemination of research. Secondary outcomes: Our 'What, Who and Why?' approach includes issues that need to be considered when disseminating research findings (What?), the different players who should assume responsibility during the various stages of conducting a clinical trial and disseminating clinical trial documents (Who?), and motivations that might lead the various players to disseminate findings selectively, thereby introducing bias in the dissemination process (Why?). Conclusions: Our comprehensive framework of (non-) dissemination of research findings, based on the results of a scoping literature search and expert consensus will facilitate the development of future policies and guidelines regarding the multifaceted issue of selective publication, historically referred to as 'publication bias'