Project Retrosight. Understanding the returns from cardiovascular and stroke research: Methodology Report

Copyright @ 2011 RAND Europe. All rights reserved. The full text article is available via the link below.This project explores the impacts arising from cardiovascular and stroke research funded 15-20 years ago and attempts to draw out aspects of the research, researcher or environment that are associated with high or low impact. The project is a case study-based review of 29 cardiovascular and stroke research grants, funded in Australia, Canada and UK between 1989 and 1993. The case studies focused on the individual grants but considered the development of the investigators and ideas involved in the research projects from initiation to the present day. Grants were selected through a stratified random selection approach that aimed to include both high- and low-impact grants. The key messages are as follows: 1) The cases reveal that a large and diverse range of impacts arose from the 29 grants studied. 2) There are variations between the impacts derived from basic biomedical and clinical research. 3) There is no correlation between knowledge production and wider impacts 4) The majority of economic impacts identified come from a minority of projects. 5) We identified factors that appear to be associated with high and low impact. This report presents the key observations of the study and an overview of the methods involved. It has been written for funders of biomedical and health research and health services, health researchers, and policy makers in those fields. It will also be of interest to those involved in research and impact evaluation.This study was initiated with internal funding from RAND Europe and HERG, with continuing funding from the UK National Institute for Health Research, the Canadian Institutes of Health Research, the Heart and Stroke Foundation of Canada and the National Heart Foundation of Australia. The UK Stroke Association and the British Heart Foundation provided support in kind through access to their archives

Serious violent offenders : developing a risk assessment framework

In order to establish a complementary language of risk across all agencies, it is recommended that the Scottish Government and the Risk Management Authority actively disseminate MAPPA guidance through the RMA's specialist training programme and through the development of protocols and memoranda of agreement. Prior to a violent offender framework being implemented, an audit of existing numbers, staffing, budgetary and other resources should be undertaken across the Community Justice Authorities to ascertain projected needs

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

Best practice in undertaking and reporting health technology assessments : Working Group 4 report

[Executive Summary] The aim of Working Group 4 has been to develop and disseminate best practice in undertaking and reporting assessments, and to identify needs for methodologic development. Health technology assessment (HTA) is a multidisciplinary activity that systematically examines the technical performance, safety, clinical efficacy, and effectiveness, cost, costeffectiveness, organizational implications, social consequences, legal, and ethical considerations of the application of a health technology (18). HTA activity has been continuously increasing over the last few years. Numerous HTA agencies and other institutions (termed in this report “HTA doers”) across Europe are producing an important and growing amount of HTA information. The objectives of HTA vary considerably between HTA agencies and other actors, from a strictly political decision making–oriented approach regarding advice on market licensure, coverage in benefits catalogue, or investment planning to information directed to providers or to the public. Although there seems to be broad agreement on the general elements that belong to the HTA process, and although HTA doers in Europe use similar principles (41), this is often difficult to see because of differences in language and terminology. In addition, the reporting of the findings from the assessments differs considerably. This reduces comparability and makes it difficult for those undertaking HTA assessments to integrate previous findings from other HTA doers in a subsequent evaluation of the same technology. Transparent and clear reporting is an important step toward disseminating the findings of a HTA; thus, standards that ensure high quality reporting may contribute to a wider dissemination of results. The EUR-ASSESS methodologic subgroup already proposed a framework for conducting and reporting HTA (18), which served as the basis for the current working group. New developments in the last 5 years necessitate revisiting that framework and providing a solid structure for future updates. Giving due attention to these methodologic developments, this report describes the current “best practice” in both undertaking and reporting HTA and identifies the needs for methodologic development. It concludes with specific recommendations and tools for implementing them, e.g., by providing the structure for English-language scientific summary reports and a checklist to assess the methodologic and reporting quality of HTA reports

Assessing the applicability of participatory multi-criteria analysis (MCA) methodologies to the appraisal of mega transport infrastructure

The topic of ‘how mega transport projects should be assessed’ continues to generate disputes amongst academics, infrastructure specialists, investors and governments alike. This Ph.D. research sought to explore the applicability and effectiveness of participatory MCA methodologies to the (economic, environmental and social) appraisal of such projects. Although very popular amongst academics, participatory MCA methodologies seem to have enjoyed limited practical application and there are no studies assessing their effectiveness. This research entailed several steps, where different methods of investigation and multiple sources of information, both primary and secondary, were combined together in an effort to increase the reliability of the results. Firstly, a comprehensive review of the literature, whose findings were validated and integrated by means of an expert focus group interview, was undertaken in the attempt to determine the key features of mega transport projects. An investigation into transport appraisal practice and traditional appraisal methodologies was then carried out through a comprehensive analysis of the existing literature, complemented with unstructured interviews and informal discussions with some international experts. Successively, an examination of over 60 publications on participatory MCA methodologies was performed with the objective of determining the main attributes and critical aspects of such methods. An analysis of the current trends in mega transport infrastructure planning and decision-making practice was also undertaken based on a series of interviews with several infrastructure practitioners and the analysis of three large-scale port and port-related projects, namely the Alameda freight rail corridor (US), the expansion of the Port of Rotterdam (Holland) and the London Gateway port complex (UK). The London Gateway port complex was also adopted as case study for a practical application of participatory MCA methodologies. Overall, the outcomes of this multi-actor multi-criteria appraisal exercise, together with the findings from the previous steps of research, led to the identification of several (methodological and non-methodological) issues potentially surrounding the practical application of such methods. These issues were ultimately explored through a survey questionnaire carried out amongst specialists and proponents of participatory techniques. The overall conclusion of the thesis is that participatory MCA methodologies represent an under researched area, where some critical themes have received limited consideration; contrasting views still exist on many fundamental aspects; and where, in many cases, the disconnection between theory and practice is apparent. In particular, the key principles, steps and structure of such methods are (directly or indirectly) based on the rationalcomprehensive planning model, which hardly reflects the way projects are developed. Therefore, while, in principle, multi-actor multi-criteria procedures might be carried out to complement and integrate the information derived from conventional ex-ante analyses, it is difficult to see how such methods could significantly improve the decision-making process of major transport infrastru

Report No. 28: Review of Methodologies Applied for the Assessment of Employment and Social Impacts

Joint report with ECORYS Netherlands for the DG Employment, Social Affairs and Equal Opportunities of the European Commission, Bonn 2010 (217 pages)

Pain assessment for people with dementia: a systematic review of systematic reviews of pain assessment tools.

BACKGROUND: There is evidence of under-detection and poor management of pain in patients with dementia, in both long-term and acute care. Accurate assessment of pain in people with dementia is challenging and pain assessment tools have received considerable attention over the years, with an increasing number of tools made available. Systematic reviews on the evidence of their validity and utility mostly compare different sets of tools. This review of systematic reviews analyses and summarises evidence concerning the psychometric properties and clinical utility of pain assessment tools in adults with dementia or cognitive impairment. METHODS: We searched for systematic reviews of pain assessment tools providing evidence of reliability, validity and clinical utility. Two reviewers independently assessed each review and extracted data from them, with a third reviewer mediating when consensus was not reached. Analysis of the data was carried out collaboratively. The reviews were synthesised using a narrative synthesis approach. RESULTS: We retrieved 441 potentially eligible reviews, 23 met the criteria for inclusion and 8 provided data for extraction. Each review evaluated between 8 and 13 tools, in aggregate providing evidence on a total of 28 tools. The quality of the reviews varied and the reporting often lacked sufficient methodological detail for quality assessment. The 28 tools appear to have been studied in a variety of settings and with varied types of patients. The reviews identified several methodological limitations across the original studies. The lack of a 'gold standard' significantly hinders the evaluation of tools' validity. Most importantly, the samples were small providing limited evidence for use of any of the tools across settings or populations. CONCLUSIONS: There are a considerable number of pain assessment tools available for use with the elderly cognitive impaired population. However there is limited evidence about their reliability, validity and clinical utility. On the basis of this review no one tool can be recommended given the existing evidence

Applying economic evaluation to public health interventions: The case of interventions to promote physical activity

Copyright @ 2012 The Authors. This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/2.5/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.This article has been made available through the Brunel Open Access Publishing Fund.BACKGROUND: This paper explores the application of alternative approaches to economic evaluation of public health interventions, using a worked example of exercise referral schemes (ERSs). METHODS: Cost-utility (CUA) and cost-consequence analyses (CCA) were used to assess the cost-effectiveness of ERSs. For the CUA, evidence was synthesized using a decision analytic model that adopts a lifetime horizon and NHS/Personal Social Services perspective. Outcomes were expressed as incremental cost per quality-adjusted life-year (QALY). CCA was conducted from a partial-societal perspective, including health and non-healthcare costs and benefits. Outcomes were reported in natural units, such as cases of strokes or CHD avoided. RESULTS: Compared with usual care, the incremental cost per QALY of ERS is £20 876. Based on a cohort of 100 000 individuals, CCA estimates cost of ERS at £22 million to the healthcare provider and £12 million to participants. The benefits of ERS include additional 3900 people becoming physically active, 51 cases of CHD avoided, 16 cases of stroke avoided, 86 cases of diabetes avoided and a gain of ∼800 QALYs. CONCLUSIONS: CCA might provide greater transparency than CUA in reporting the outcomes of public health interventions and have greater resonance with stakeholders involved in commissioning these interventions.This work was supported by the NIHR Health Technology Assessment programme (project number 08/72/01). This article is made available through the Brunel Open Access Publishing Fund