Best practice in undertaking and reporting health technology assessments : Working Group 4 report

[Executive Summary] The aim of Working Group 4 has been to develop and disseminate best practice in undertaking and reporting assessments, and to identify needs for methodologic development. Health technology assessment (HTA) is a multidisciplinary activity that systematically examines the technical performance, safety, clinical efficacy, and effectiveness, cost, costeffectiveness, organizational implications, social consequences, legal, and ethical considerations of the application of a health technology (18). HTA activity has been continuously increasing over the last few years. Numerous HTA agencies and other institutions (termed in this report “HTA doers”) across Europe are producing an important and growing amount of HTA information. The objectives of HTA vary considerably between HTA agencies and other actors, from a strictly political decision making–oriented approach regarding advice on market licensure, coverage in benefits catalogue, or investment planning to information directed to providers or to the public. Although there seems to be broad agreement on the general elements that belong to the HTA process, and although HTA doers in Europe use similar principles (41), this is often difficult to see because of differences in language and terminology. In addition, the reporting of the findings from the assessments differs considerably. This reduces comparability and makes it difficult for those undertaking HTA assessments to integrate previous findings from other HTA doers in a subsequent evaluation of the same technology. Transparent and clear reporting is an important step toward disseminating the findings of a HTA; thus, standards that ensure high quality reporting may contribute to a wider dissemination of results. The EUR-ASSESS methodologic subgroup already proposed a framework for conducting and reporting HTA (18), which served as the basis for the current working group. New developments in the last 5 years necessitate revisiting that framework and providing a solid structure for future updates. Giving due attention to these methodologic developments, this report describes the current “best practice” in both undertaking and reporting HTA and identifies the needs for methodologic development. It concludes with specific recommendations and tools for implementing them, e.g., by providing the structure for English-language scientific summary reports and a checklist to assess the methodologic and reporting quality of HTA reports

Around the Tables – Contextual Factors in Healthcare Coverage Decisions Across Western Europe

Background: Across Western Europe, procedures and formalised criteria for taking decisions on the coverage (inclusion in the benefits basket or equivalent) of healthcare technologies vary substantially. In the decision documents, which display the justification of, the rationale for, these decisions, national healthcare institutes may employ ‘contextual factors,’ defined here as situation-specific considerations. Little is known about how the use of such contextual factors compares across countries. We describe and compare contextual factors as used in coverage decisions generally and 4 decision documents specifically in Belgium, England, Germany, and the Netherlands. Methods: Four group interviews with 3 experts from the national healthcare institute of each country, document and web site analysis, and a workshop with 1 to 2 of these experts per country were followed by the examination of the documents of 4 specific decisions taken in each of the 4 countries, sampled to vary widely in type of technology and decision outcome. Results: From the available decision documents, we conclude that in every country studied, contextual factors are established ‘around the table,’ ie, in deliberation. All documents examined feature contextual factors, with similar contextual factor patterns leading to similar decisions in different countries. The Dutch decisions employ the widest variety of factors, with the exception of the societal functioning of the patient, which is relatively common in Belgium, England, and Germany. Half of the final decisions were taken in another setting, with the consequence that no documentation was retrievable for 2 decisions. Conclusion: First, we conclude that in these countries, contextual factors are actively integrated in the decision document, and that this is achieved in deliberation. Conceptualising contextual factors as both situation-specific and actively-integrated affords insight into practices of contextualisation and provides an encouragement for exchange between decision-makers on more qualitative aspects of decisions. Second, the decisions that lacked a publicly acc

Determinants of avoidable deaths from ischemic heart disease in East and West Germany

Objective: Within Germany, a significant decrease in avoidable mortality from ischemic heart disease (IHD) has been observed since the early 1990s. The objective of this paper is to identify the specific reasons that have led to the decrease in the number of avoidable deaths from IHD in West and East Germany from 1996 to 2004. Methods: We analyzed the mortality rate from IHD of the male population aged less than 65 years on the regional level of German counties over the 1996–2004 period. Methodologically, after adjusting for a number of health structure variables, the socioeconomic structure of each region, and yearly time trends in avoidable mortality, we accounted for unobservable differences among regions by using a fixed-effect estimator. Results: Our main result reveals that the number of intracardiac catheter facilities, an important diagnostic tool for IHD, significantly accounts for decreases in avoidable mortality from IHD. This is important, as the modernization of the East German health sector included a considerable catching-up process in the number of IC facilities provided relative to West Germany. Conclusion: Our results suggest that the modernization of the East German health sector may have contributed to saving people from premature deaths

Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries

Background: The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decisionmaking. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peerreviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries. Copyright:Methods and Findings: Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%-52.4%, I2594.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%-65.9%, I2598.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2-3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6- 2.5). The probability of publication within two years after study completion ranged from 7% to 30%.Conclusions: A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased

The Politics of Evidence Use in Health Policy Making in Germany-the Case of Regulating Hospital Minimum Volumes.

This article examines the role of scientific evidence in informing health policy decisions in Germany, using minimum volumes policy as a case study. It argues that scientific evidence was used strategically at various stages of the policy process both by individual corporatist actors and by the Federal Joint Committee as the regulator. Minimum volumes regulation was inspired by scientific evidence suggesting a positive relationship between service volume and patient outcomes for complex surgical interventions. Federal legislation was introduced in 2002 to delegate the selection of services and the setting of volumes to corporatist decision makers. Yet, despite being represented in the Federal Joint Committee, hospitals affected by its decisions took the Committee to court to seek legal redress and prevent policy implementation. Evidence has been key to support, and challenge, decisions about minimum volumes, including in court. The analysis of the role of scientific evidence in minimum volumes regulation in Germany highlights the dynamic relationship between evidence use and the political and institutional context of health policy making, which in this case is characterized by the legislative nature of policy making, corporatism, and the role of the judiciary in reviewing policy decisions

The value of health care – a matter of discussion in Germany

BACKGROUND: Interest in assessing the value of health-care services in Germany has considerably increased since the foundation of the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG (Institute for Quality and Efficiency in Health Care). The practical application of value assessment illustrates how problematic the process can be. In all decisions made for the provision of health care, data concerning the measurable dimensions (quantity and quality of efficacy and effectiveness, validity of the results and costs) flow into a complex and not yet standardized decision-making process concerning public financing. Some of these decisions are based on data of uncertain validity, unknown reproducibility and unclear appropriateness. DISCUSSION: In this paper we describe the theoretical aspects of value from psychological and economic viewpoints and discuss national and international approaches. Methodic details and difficulties in assessing the value of health-care services are analysed. A definition of the intangible value of health-care services will be proposed which contains only three factors: the absolute risk reduction (usually a measure of efficacy), the validity of the scientific papers examined and the type of the expected effectiveness (prevention of death and disability, restitution of well-being). The intangible value describes the additional benefit when comparing two possible actions, like treatment or observation only. CONCLUSION: The description of intangible value from the viewpoint of different stakeholders is a useful measure for subsequent steps (not discussed here) – the evaluation of costs and of patient benefit. A standardised, transparent, fair and democratic evaluation is essential for the definition of a basic benefit package

Improving the use of research evidence in guideline development: 14. Reporting guidelines

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 14(th )of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on reporting guidelines and recommendations. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: There is little empirical evidence that addresses these questions. Our answers are based on logical arguments and standards put forward by other groups. What standard types of recommendations or reports should WHO use? • WHO should develop standard formats for reporting recommendations to facilitate recognition and use by decision makers for whom the recommendations are intended, and to ensure that all the information needed to judge the quality of a guideline, determine its applicability and, if needed, adapt it, is reported. • WHO should develop standard formats for full systematically developed guidelines that are sponsored by WHO, rapid assessments, and guidelines that are endorsed by WHO. • All three formats should include the same information as full guidelines, indicating explicitly what the group preparing the guideline did not do, as well as the methods that were used. • These formats should be used across clinical, public health and health systems recommendations. How should recommendations be formulated and reported? • Reports should be structured, using headings that correspond to those suggested by the Conference on Guideline Standardization or similar headings. • The quality of evidence and strength of recommendations should be reported explicitly using a standard approach. • The way in which recommendations are formulated should be adapted to the specific characteristics of a specific guideline. • Urgent attention should be given to developing a template that provides decision makers with the relevant global evidence that is needed to inform a decision and offers practical methods for incorporating the context specific evidence and judgements that are needed

Glioma imaging in Europe: A survey of 220 centres and recommendations for best clinical practice

Objectives: At a European Society of Neuroradiology (ESNR) Annual Meeting 2015 workshop, commonalities in practice, current controversies and technical hurdles in glioma MRI were discussed. We aimed to formulate guidance on MRI of glioma and determine its feasibility, by seeking information on glioma imaging practices from the European Neuroradiology community. Methods: Invitations to a structured survey were emailed to ESNR members (n=1,662) and associates (n=6,400), European national radiologists’ societies and distributed via social media. Results: Responses were received from 220 institutions (59% academic). Conventional imaging protocols generally include T2w, T2-FLAIR, DWI, and pre- and post-contrast T1w. Perfusion MRI is used widely (85.5%), while spectroscopy seems reserved for specific indications. Reasons for omitting advanced imaging modalities include lack of facility/software, time constraints and no requests. Early postoperative MRI is routinely carried out by 74% within 24–72 h, but only 17% report a percent measure of resection. For follow-up, most sites (60%) issue qualitative reports, while 27% report an assessment according to the RANO criteria. A minori

Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the eighth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on the synthesis and presentation of research evidence, focusing on four key questions. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We found two reviews of instruments for critically appraising systematic reviews, several studies of the importance of using extensive searches for reviews and determining when it is important to update reviews, and consensus statements about the reporting of reviews that informed our answers to the following questions. How should existing systematic reviews be critically appraised? • Because preparing systematic reviews can take over a year and require capacity and resources, existing reviews should be used when possible and updated, if needed. • Standard criteria, such as A MeaSurement Tool to Assess Reviews (AMSTAR), should be used to critically appraise existing systematic reviews, together with an assessment of the relevance of the review to the questions being asked. When and how should WHO undertake or commission new reviews? • Consideration should be given to undertaking or commissioning a new review whenever a relevant, up-to-date review of good quality is not available. • When time or resources are limited it may be necessary to undertake rapid assessments. The methods that are used to do these assessments should be reported, including important limitations and uncertainties and explicit consideration of the need and urgency of undertaking a full systematic review. • Because WHO has limited capacity for undertaking systematic reviews, reviews will often need to be commissioned when a new review is needed. Consideration should be given to establishing collaborating centres to undertake or support this work, similar to what some national organisations have done. How should the findings of systematic reviews be summarised and presented to committees responsible for making recommendations? • Concise summaries (evidence tables) of the best available evidence for each important outcome, including benefits, harms and costs, should be presented to the groups responsible for making recommendations. These should include an assessment of the quality of the evidence and a summary of the findings for each outcome. • The full systematic reviews, on which the summaries are based, should also be available to both those making recommendations and users of the recommendations. What additional information is needed to inform recommendations and how should this information be synthesised with information about effects and presented to committees? • Additional information that is needed to inform recommendations includes factors that might modify the expected effects, need (prevalence, baseline risk or status), values (the relative importance of key outcomes), costs and the availability of resources. • Any assumptions that are made about values or other factors that may vary from setting to setting should be made explicit. • For global guidelines that are intended to inform decisions in different settings, consideration should be given to using a template to assist the synthesis of information specific to a setting with the global evidence of the effects of the relevant interventions

Towards a model of talent development in physical education

Traditional conceptions of talent generally emphasise the construction of threshold values and the development of relatively unitary abilities, and this approach still dominates talent development programmes for elite sport. Most researchers on high ability, however, now favour domain-specific, multidimensional conceptions of ability that stress the development of behavioural potential and its interaction with personal and environmental characteristics. This paper presents a model of talent in physical education, drawing together findings from a wide range of literature on the realisation and inhibition of abilities, international studies of effective school-based identification and provision strategies, and a conception of the subject as an integration and realisation of different forms of ability. In presenting this model, the authors aim to redress the imbalance within the current debate from an almost total concern with out-of-school clubs and the preparation for adult elite sport, in favour of a more equitable and inclusive approach, premised upon the unique importance of mainstream, curricular physical education within any talent development scheme