Quality of life returns from basic research

<p>Abstract</p> <p>Background</p> <p>Assessing the consequences of research is an increasingly important task in research and innovation policy. This paper takes a broader view of those consequences than the conventional economic approach, placing researchers and their activities in the centre of the assessment process and examining results for professional practice and general education as well as contributions to knowledge.</p> <p>Methods</p> <p>The paper uses historical and documentary analysis to illustrate the approach, focusing on U.S. biomedicine over the past century. At aggregate level, the analysis attributes portions of the change in aggregate health indicators to research and research-based institutions, through several available types of logic: either through correlations between timing of institutional changes and changes in the indicators or through direct or indirect causal connections.</p> <p>Results</p> <p>The analysis shows that while biomedical research has certainly contributed to improved health in the United States, other factors have also contributed. In some ways the institutional structure of science-based medicine has worked against creating benefits for some groups in U.S. society.</p> <p>Conclusions</p> <p>The paper concludes with a call for more strategic attention to dimensions of impact other than knowledge outcomes and for participatory planning for research.</p

The impact of Cochrane Systematic Reviews : a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research

© 2014 Bunn et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Background: There has been a growing emphasis on evidence-informed decision making in health care. Systematic reviews, such as those produced by the Cochrane Collaboration, have been a key component of this movement. The UK National Institute for Health Research (NIHR) Systematic Review Programme currently supports 20 Cochrane Review Groups (CRGs). The aim of this study was to identify the impacts of Cochrane reviews published by NIHR funded CRGs during the years 2007-11. Methods: We sent questionnaires to CRGs and review authors, interviewed guideline developers and used bibliometrics and documentary review to get an overview of CRG impact and to evaluate the impact of a sample of 60 Cochrane reviews. We used a framework with four categories (knowledge production, research targeting, informing policy development, and impact on practice/services). Results: A total of 1502 new and updated reviews were produced by the 20 NIHR funded CRGs between 2007-11. The clearest impacts were on policy with a total of 483 systematic reviews cited in 247 sets of guidance; 62 were international, 175 national (87 from the UK) and 10 local. Review authors and CRGs provided some examples of impact on practice or services, for example safer use of medication, the identification of new effective drugs or treatments and potential economic benefits through the reduction in the use of unproven or unnecessary procedures. However, such impacts are difficult to objectively document and the majority of reviewers were unsure if their review had produced specific impacts. Qualitative data suggested that Cochrane reviews often play an instrumental role in informing guidance although a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and guideline developers were barriers to their use. Conclusions: Health and economic impacts of research are generally difficult to measure. We found that to be the case with this evaluation. Impacts on knowledge production and clinical guidance were easier to identify and substantiate than those on clinical practice. Questions remain about how we define and measure impact and more work is needed to develop suitable methods for impact analysis.Peer reviewe

The utilisation of health research in policy-making: Concepts, examples and methods of assessment

The importance of health research utilisation in policy-making, and of understanding the mechanisms involved, is increasingly recognised. Recent reports calling for more resources to improve health in developing countries, and global pressures for accountability, draw greater attention to research-informed policy-making. Key utilisation issues have been described for at least twenty years, but the growing focus on health research systems creates additional dimensions. The utilisation of health research in policy-making should contribute to policies that may eventually lead to desired outcomes, including health gains. In this article, exploration of these issues is combined with a review of various forms of policy-making. When this is linked to analysis of different types of health research, it assists in building a comprehensive account of the diverse meanings of research utilisation. Previous studies report methods and conceptual frameworks that have been applied, if with varying degrees of success, to record utilisation in policy-making. These studies reveal various examples of research impact within a general picture of underutilisation. Factors potentially enhancing utilisation can be identified by exploration of: priority setting; activities of the health research system at the interface between research and policy-making; and the role of the recipients, or 'receptors', of health research. An interfaces and receptors model provides a framework for analysis. Recommendations about possible methods for assessing health research utilisation follow identification of the purposes of such assessments. Our conclusion is that research utilisation can be better understood, and enhanced, by developing assessment methods informed by conceptual analysis and review of previous studies

The Calculus of Committee Composition

Modern institutions face the recurring dilemma of designing accurate evaluation procedures in settings as diverse as academic selection committees, social policies, elections, and figure skating competitions. In particular, it is essential to determine both the number of evaluators and the method for combining their judgments. Previous work has focused on the latter issue, uncovering paradoxes that underscore the inherent difficulties. Yet the number of judges is an important consideration that is intimately connected with the methodology and the success of the evaluation. We address the question of the number of judges through a cost analysis that incorporates the accuracy of the evaluation method, the cost per judge, and the cost of an error in decision. We associate the optimal number of judges with the lowest cost and determine the optimal number of judges in several different scenarios. Through analytical and numerical studies, we show how the optimal number depends on the evaluation rule, the accuracy of the judges, the (cost per judge)/(cost per error) ratio. Paradoxically, we find that for a panel of judges of equal accuracy, the optimal panel size may be greater for judges with higher accuracy than for judges with lower accuracy. The development of any evaluation procedure requires knowledge about the accuracy of evaluation methods, the costs of judges, and the costs of errors. By determining the optimal number of judges, we highlight important connections between these quantities and uncover a paradox that we show to be a general feature of evaluation procedures. Ultimately, our work provides policy-makers with a simple and novel method to optimize evaluation procedures

Describing the impact of health research: a Research Impact Framework

BACKGROUND: Researchers are increasingly required to describe the impact of their work, e.g. in grant proposals, project reports, press releases and research assessment exercises. Specialised impact assessment studies can be difficult to replicate and may require resources and skills not available to individual researchers. Researchers are often hard-pressed to identify and describe research impacts and ad hoc accounts do not facilitate comparison across time or projects. METHODS: The Research Impact Framework was developed by identifying potential areas of health research impact from the research impact assessment literature and based on research assessment criteria, for example, as set out by the UK Research Assessment Exercise panels. A prototype of the framework was used to guide an analysis of the impact of selected research projects at the London School of Hygiene and Tropical Medicine. Additional areas of impact were identified in the process and researchers also provided feedback on which descriptive categories they thought were useful and valid vis-à-vis the nature and impact of their work. RESULTS: We identified four broad areas of impact: I. Research-related impacts; II. Policy impacts; III. Service impacts: health and intersectoral and IV. Societal impacts. Within each of these areas, further descriptive categories were identified. For example, the nature of research impact on policy can be described using the following categorisation, put forward by Weiss: Instrumental use where research findings drive policy-making; Mobilisation of support where research provides support for policy proposals; Conceptual use where research influences the concepts and language of policy deliberations and Redefining/wider influence where research leads to rethinking and changing established practices and beliefs. CONCLUSION: Researchers, while initially sceptical, found that the Research Impact Framework provided prompts and descriptive categories that helped them systematically identify a range of specific and verifiable impacts related to their work (compared to ad hoc approaches they had previously used). The framework could also help researchers think through implementation strategies and identify unintended or harmful effects. The standardised structure of the framework facilitates comparison of research impacts across projects and time, which is useful from analytical, management and assessment perspectives

Reflections on the entrepreneurial state, innovation and social justice

The state and its role in technological innovation and social justice have become, once again, fashionable topics of political and economic debate. A number of innovation theorists argue that never more than today, it is necessary to rethink the state’s entrepreneurial role in society and welfare. Their argument provides justification for the existence of the state, going beyond classical political theory and especially contractarian accounts of legitimacy and obligation. It emphasises the ability and willingness of the state to take risks and reduce uncertainty of economic agents for the sake of innovation that can make everyone better off. This paper insists that although the risk-taking argument of innovation theorists deserves further attention and analysis, it should not be abstracted from a holistic politico-theoretical approach to the state. Such an approach is necessary for a critical understanding of the complex set of predominantly political institutions which compose the state and which have been historically developed to guarantee social evolution. Any risk-taking for innovative enterprise and mission-oriented investment ought to be justified and legitimised on the grounds of principled democratic procedures. This implies that innovation itself is a value-laden political process, requiring participation in the decision-making and standards of fairness

‘Capacity for what? Capacity for whom?’ A decolonial deconstruction of research capacity development practices in the Global South and a proposal for a value-centred approach

Whilst North to South knowledge transfer patterns have been extensively problematised by Southern and decolonial perspectives, there is very little reflection on the practice of research capacity development (RCD), still strongly focused on technoscientific solutionism, yet largely uncritical of its underlying normative directions and power asymmetries. Without making transparent these normative and epistemological dimensions, RCD practices will continue to perpetuate approaches that are likely to be narrow, technocratic and unreflexive of colonial legacies, thus failing to achieve the aims of RCD, namely, the equitable and development-oriented production of knowledge in low- and middle-income societies. Informed by the authors’ direct experience of RCD approaches and combining insights from decolonial works and other perspectives from the margins with Science and Technology Studies, the paper undertakes a normative and epistemological deconstruction of RCD mainstream practice. Highlighting asymmetries of power and material resources in knowledge production, the paper’s decolonial lens seeks to aid the planning, implementation and evaluation of RCD interventions. Principles of cognitive justice and epistemic pluralism, accessibility enabled by systems thinking and sustainability grounded on localisation are suggested as the building blocks for more reflexive and equitable policies that promote research capacity for the purpose of creating social value and not solely for the sake of perpetuating technoscience