Assessing the impact of health technology assessment in the Netherlands

Copyright © Cambridge University Press 2008Objectives: Investments in health research should lead to improvements in health and health care. This is also the remit of the main HTA program in the Netherlands. The aims of this study were to assess whether the results of this program have led to such improvements and to analyze how best to assess the impact from health research.Methods: We assessed the impact of individual HTA projects by adapting the "payback framework" developed in the United Kingdom. We conducted dossier reviews and sent a survey to principal investigators of forty-three projects awarded between 2000 and 2003. We then provided an overview of documented output and outcome that was assessed by ten HTA experts using a scoring method. Finally, we conducted five case studies using information from additional dossier review and semistructured key informant interviews.Results: The findings confirm that the payback framework is a useful approach to assess the impact of HTA projects. We identified over 101 peer reviewed papers, more than twenty-five PhDs, citations of research in guidelines (six projects), and implementation of new treatment strategies (eleven projects). The case studies provided greater depth and understanding about the levels of impact that arise and why and how they have been achieved.Conclusions: It is generally too early to determine whether the HTA program led to actual changes in healthcare policy and practice. However, the results can be used as a baseline measurement for future evaluation and can help funding organizations or HTA agencies consider how to assess impact, possibly routinely. This, in turn, could help inform research strategies and justify expenditure for health research.This research is funded by ZonMw, the Netherlands organization for health research and development (project 945-15-001)

Temporal album

Transient synchronization has been used as a mechanism of recognizing auditory patterns using integrate-and-fire neural networks. We first extend the mechanism to vision tasks and investigate the role of spike dependent learning. We show that such a temporal Hebbian learning rule significantly improves accuracy of detection. We demonstrate how multiple patterns can be identified by a single pattern selective neuron and how a temporal album can be constructed. This principle may lead to multidimensional memories, where the capacity per neuron is considerably increased with accurate detection of spike synchronization

Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's Arthritis Research Campaign

Background: External and internal factors are increasingly encouraging research funding bodies to demonstrate the outcomes of their research. Traditional methods of assessing research are still important, but can be merged into broader multi-dimensional categorisations of research benefits. The onus has hitherto been on public sector funding bodies, but in the UK the role of medical charities in funding research is particularly important and the Arthritis Research Campaign, the leading medical charity in its field in the UK, commissioned a study to identify the outcomes from research that it funds. This article describes the methods to be used. Methods: A case study approach will enable narratives to be told, illuminating how research funded in the early 1990s was (or was not) translated into practice. Each study will be organised using a common structure, which, with careful selection of cases, should enable cross-case analysis to illustrate the strengths of different modes and categories of research. Three main interdependent methods will be used: documentary and literature review; semi-structured interviews; and bibliometric analysis. The evaluative framework for organising the studies was previously used for assessing the benefits from health services research. Here, it has been specifically amended for a medical charity that funds a wide range of research and is concerned to develop the careers of researchers. It was further refined in three pilot studies. The framework has two main elements. First, a multi-dimensional categorisation of benefits going from the knowledge produced in peer reviewed journal articles through to the health and potential economic gain. The second element is a logic model, which, with various stages, should provide a way of organising the studies. The stock of knowledge is important: much research, especially basic, will feed into it and influence further research rather than directly lead to health gains. The cross-case analysis will look for factors associated with outcomes. Conclusions: The pilots confirmed the applicability of the methods for a full study which should assist the Arthritis Research Campaign to demonstrate the outcomes from its funding, and provide it with evidence to inform its own policies

A lattice spring model of heterogeneous materials with plasticity

A three-dimensional lattice spring model of a heterogeneous material is presented. For small deformations, the model is shown to recover the governing equations for an isotropic elastic medium. The model gives reasonable agreement with theoretical predictions for the elastic fields generated by a spherical inclusion, although for small particle sizes the discretization of the underlying lattice causes some departures from the predicted values. Plasticity is introduced by decreasing the elastic moduli locally whilst maintaining stress continuity. Results are presented for a spherical inclusion in a plastic matrix and are found to be in good agreement with the predictions of Wilner (1988 J. Mech. Phys. Solids 36 141-65).</p

Physical activity in England: Who is meeting the recommended level of participation through sports and exercise?

This article is available through the Brunel Open Access Publishing Fund. Copyright © 2012 Anokye et al.Background: Little is known about the correlates of meeting recommended levels of participation in physical activity (PA) and how this understanding informs public health policies on behaviour change. Objective: To analyse who meets the recommended level of participation in PA in males and females separately by applying ‘process’ modelling frameworks (single vs. sequential 2-step process). Methods: Using the Health Survey for England 2006, (n = 14 142; ≥16 years), gender-specific regression models were estimated using bivariate probit with selectivity correction and single probit models. A ‘sequential, 2-step process’ modelled participation and meeting the recommended level separately, whereas the ‘single process’ considered both participation and level together. Results: In females, meeting the recommended level was associated with degree holders [Marginal effect (ME) = 0.013] and age (ME = −0.001), whereas in males, age was a significant correlate (ME = −0.003 to −0.004). The order of importance of correlates was similar across genders, with ethnicity being the most important correlate in both males (ME = −0.060) and females (ME = −0.133). In females, the ‘sequential, 2-step process’ performed better (ρ = −0.364, P < 0.001) than that in males (ρ = 0.154). Conclusion: The degree to which people undertake the recommended level of PA through vigorous activity varies between males and females, and the process that best predicts such decisions, i.e. whether it is a sequential, 2-step process or a single-step choice, is also different for males and females. Understanding this should help to identify subgroups that are less likely to meet the recommended level of PA (and hence more likely to benefit from any PA promotion intervention).This study was funded by the Department of Health’s Policy Research Programme

Using remote sensing to assess ecosystem resilience

Vegetation ecosystems are increasingly under pressure from both direct human influence and indirect anthropogenically-driven climate change. Increasing amounts of data are made available from satellite systems which can image these ecosystems from afar. The work in this thesis provides several examples of the utility of remotely sensed data from satellites to assess the resilience of ecosystems. This notion of resilience is measured by considering the return rate following a perturbation, with statistical metrics such as AR(1) and variance providing an indication of system resilience and the proximity to a potential tipping point. The first focus of this work is on direct human environmental intervention through community-based agroforestry groups in Kenya. These results show that the efforts of these groups can be detected with satellite data as a greening trend which occurs both within designated tree planting groves and in the surrounding landscape. These groups provide a case study for the power of positive social tipping points to achieve environmental improvement. Following this, the potential of high-resolution satellite data from Sentinel-2 to quantify patterned vegetation in the Sahel is explored. These striking patterns have often been associated with vegetation resilience in drylands. No correlation is found between pattern morphology and resilience, contrary to a previously held hypothesis from the literature. Precipitation is also identified as a key driver of these patterns. Moving beyond drylands, satellite data is utilised at a global scale to assess the link between vegetation resilience and climatic variables across the world. There is a clear relationship between average resilience, as measured by AR(1), and precipitation, which is evident at three spatial scales; the local (pixel), ecoregion and biome. There is also a temperature component, with hotter, drier locations displaying lower levels of resilience. This thesis finishes with a discussion of the potential for a resilience sensing framework constructed by combining remote sensing data with new cloud computing technologies. This will enable the monitoring of resilience change across the world and the identification of regions which require further investigation and intervention.Leverhulme Trus

Project Retrosight. Understanding the returns from cardiovascular and stroke research: Case Studies

Copyright @ 2011 RAND Europe. All rights reserved. The full text article is available via the link below.This project explores the impacts arising from cardiovascular and stroke research funded 15-20 years ago and attempts to draw out aspects of the research, researcher or environment that are associated with high or low impact. The project is a case study-based review of 29 cardiovascular and stroke research grants, funded in Australia, Canada and UK between 1989 and 1993. The case studies focused on the individual grants but considered the development of the investigators and ideas involved in the research projects from initiation to the present day. Grants were selected through a stratified random selection approach that aimed to include both high- and low-impact grants. The key messages are as follows: 1) The cases reveal that a large and diverse range of impacts arose from the 29 grants studied. 2) There are variations between the impacts derived from basic biomedical and clinical research. 3) There is no correlation between knowledge production and wider impacts 4) The majority of economic impacts identified come from a minority of projects. 5) We identified factors that appear to be associated with high and low impact. This report presents the key observations of the study and an overview of the methods involved. It has been written for funders of biomedical and health research and health services, health researchers, and policy makers in those fields. It will also be of interest to those involved in research and impact evaluation.This study was initiated with internal funding from RAND Europe and HERG, with continuing funding from the UK National Institute for Health Research, the Canadian Institutes of Health Research, the Heart and Stroke Foundation of Canada and the National Heart Foundation of Australia. The UK Stroke Association and the British Heart Foundation provided support in kind through access to their archives

From bench to bedside: Tracing the payback forwards from basic or early clinical research – A preliminary exercise and proposals for a future study

EXECUTIVE SUMMARY Chapter 1 : Introduction • The members of the research team from HERG and the Wellcome Trust have conducted previous studies showing that it is possible both to assess the payback from applied health research, and to use bibliometrics to trace the links between generations of research and clinical guidelines. In another of the team’s studies, however, it proved difficult to replicate the major study by Comroe and Dripps (1976) that had identified clinical advances and then worked backwards to show that they had relied on earlier basic research. Therefore, the study reported here sets out to use the methods developed in our previous studies of payback to undertake analysis that starts with more basic or early clinical research and traces the research lines forwards to clinical applications. Whilst this preliminary study involved preparation for a future large-scale study, it was hoped that it would also provide an interesting case study. • Starting with the research outputs of one team 20 years ago, called the 1st generation papers, the preliminary study has three main elements: standard bibliometric analysis through several generations of papers; categorisation of the citations; and qualitative analysis using questionnaires, critical pathway analysis and interviews to trace the impact of the 1st generation of research. • Diabetes and cardiology were suggested as possible topics on which to base the study. Initial reviews identified two bodies of research in diabetes as being potentially suitable for reasons such as the continuing activity of key members of the team. • The research into diabetes conducted in 1981 by George Alberti and his team at Newcastle, and collaborators elsewhere, was selected to provide the case study for this preliminary stage for several reasons. It was thought to have been important science and there was a belief that some of it had made a contribution to clinical practice. Chapter 2 : Bibliometric analysis • An original plan to look at publications produced over a three year period was changed to looking at the output of just one year, 1981, because in that year alone Alberti and colleagues published 29 articles. These form the 1st generation papers and the average number of citations they received is high. Identifying the citations given to these 29 papers resulted in 799 2nd generation papers and 12,891 3rd generation papers. The numbers involved meant that it was impractical to go beyond the 3rd generation. Within the high overall average, the variation in the number of citations per paper was iii considerable going from 76 to just one. Similarly, the half-lives of the 29 papers, ie the time taken for an article to receive 50% of its citations, ranged from two years to 11. • Articles can be given a Research Level (ie one of four levels from clinical observation to basic) based on the journals in which they appear. Such analysis demonstrates the breadth of Alberti’s work because the 29 articles are spread across all four Research Levels. Crucially, there was not a shift from basic to more clinical levels across the generations. The higher than average number of authors and addresses per paper is testimony to Alberti’s extensive collaborations. • The funding acknowledgements reveal the high proportion of papers supported, at least partially, by one funder: the British Diabetic Association, now Diabetes UK, which provided core support for Alberti’s Newcastle team. Chapter 3 : Categorisation of citations • Traditional citation analysis does not allow identification of the importance of the cited article to the citing article, and therefore limits the ability to use citation analysis to trace the impact of basic or early research on later research. We conducted a review of the literature of the meaning of citations. • From this review, a template was devised that allowed the location, nature and importance of citations to be recorded as well as the type of research (basic or clinical) described in the paper. This was used by six assessors on a sample of papers and inter-rater reliability was tested. Further work is required to refine the template and its definitions, and to improve its consistency in application. • Nevertheless, for initial analysis, it was applied to 623 out of the 799 2nd generation papers. A four point scale was used for the importance of the cited paper to the citing paper. In just 9% of cases was the cited 1st generation paper thought to be in one of the top two categories, ie of Considerable or Essential importance to the citing paper. • Statistical analysis revealed no relationship between the number of citations a paper received and the proportion of citations where the cited paper was classified as being of high (ie. Considerable or Essential) importance to the citing paper. Self-citations, however, were shown to be significantly more likely to be in this category. • The classification of the type of research (basic or clinical) by our analysis of each paper broadly agreed with the classification of the journals by Research Level. • The time constraints involved in applying the template, plus the lack of any overall pattern in terms of correlations between number and importance of citations, might point to the desirability of adopting a more selective approach, guided by qualitative analysis. In any selective approach, however, it is likely that self-citations should feature. iv Chapter 4 : Qualitative analysis • Given the number of co-authors, it seemed appropriate to send them a questionnaire rather than attempt to interview them. Therefore the interviewing was rather more concentrated than originally intended. Only one formal critical pathway was created, but it was undertaken by an expert in the field who worked with Alberti at Newcastle. • Some problems emerged in taking 1981 as the starting point for the study. Alberti identified 10 selected papers from the 1970s and 1980s that he felt had had most impact on clinical practice. These helped to give us both a better understanding of the payback from our 1st generation, or 1981, papers, and provided further material for analysis. • Attempting to describe the impact from the 1981 body of work, and from the 10 selected papers, underlines the complex reality of how science advances and influences clinical practice. If they make a contribution at all, most studies make a small, incremental one. • A few papers, however, have been shown to have a considerably greater impact. A possible key to the level of payback indicated is the enormous breadth of Alberti’s contacts, and fields and methods of working, to which various references were made. This is well illustrated in the account of how the idea for subcutaneous pumps came about. Similarly, the ability to produce the very important guidelines on treating diabetics during surgery, and diabetic coma, partly resulted from the application to clinical problems of the understandings gained from some of the basic/early clinical studies. It is significant that the key papers on these issues, all of which come from the list of 10 selected papers from the 1970s and 1980s, were having an impact on the 1981 work. • How far the collection of papers from 1981 have been drawn upon in similar ways is less clear. Nevertheless, papers on treating diabetics during open heart surgery, and on bolus delivery of insulin at meal times, were key parts of these wider streams, despite variable citation levels. Furthermore, various papers, including on acarbose, on portal infusion of insulin, and on semi-human insulin, were important steps in bodies of work in their respective areas. The complexity was illustrated by a paper that helped debunk the Chlorpropamide alcohol flushing hypothesis, and thus end a line of scientific enquiry: there was payback in stopping an incorrect line of inquiry, but nothing on which to build. • Each technique in the qualitative study produced information about the successful subsequent careers followed by many researchers trained through working with Alberti. • Historical perspectives, and insider expert opinions, were important in the qualitative analysis. Overall, the qualitative methods highlighted some limitations in the bibliometric approach but also showed how aspects of the citation analysis can complement the opinions expressed, for example about the importance of the breadth of Alberti’s work. v Chapter 5 : Lessons learnt and the way forward • Lessons learnt: a variety of methods can be used successfully to gather considerable data about the payback from a body of research undertaken 20 years ago. Traditional citation analysis alone, however, is not sufficient: the importance of the surgery papers despite their relatively low citation rates illustrates this. The qualitative methods are important and much of the analysis is strengthened by drawing on multiple approaches. Several problems remain, including: identifying a coherent starting point for the analysis; coping with the enormous number of papers involved in later generations; and refining the template for categorising citations and developing ways of fully utilising the results from applying it. • Preparing for the large-scale study: this preliminary study provides a basis on which to attempt to undertake the larger study we envisaged. Issues now being addressed include identification of the level of bibliometric/citation analysis necessary to complement any qualitative studies. To provide confidence in the findings from an eventual large-scale study, we will need to expand the focus. The study will need to cover at least four sets of case studies. Ideally, each set should focus on a number of research groups working in a country in the same field. We hope there will be sets of case studies in two or three fields and in at least two countries. The issues to be explored will include ones highlighted by this study such as breadth of work, level of collaboration, and the role of core funding. • Methods for the large-scale study: for each case study we now propose to employ two methodological elements based on the qualitative and quantitative techniques adopted in the preliminary study. They will work in parallel but the quantitative bibliometric analysis would be applied selectively to parts of ‘research lines’ (ie discrete themes of research) identified in the qualitative studies as being important in influencing clinical practice. • Presenting the findings: each research line could be written-up in a standardised document that would use the HERG payback model and categories to describe the impact of that research. We shall use the qualitative and quantitative data to compare and contrast the ‘payback’ of research lines by country and disease, and then identify common factors that correlate with the translation of basic or early clinical research. • Concluding comments: in the era of ‘evidence based policy’, research funders are looking for value for money in the research they support and for evidence on the effectiveness of different research strategies. In this study we have begun developing a methodology that will allow us to understand the complexity of research development over a series of generations. The utility of the policy research we propose here will only be realised when it is scaled up to cover a number of different fields in different settings.NHS Executive, London Regio

Project Retrosight. Understanding the returns from cardiovascular and stroke research: Methodology Report

Copyright @ 2011 RAND Europe. All rights reserved. The full text article is available via the link below.This project explores the impacts arising from cardiovascular and stroke research funded 15-20 years ago and attempts to draw out aspects of the research, researcher or environment that are associated with high or low impact. The project is a case study-based review of 29 cardiovascular and stroke research grants, funded in Australia, Canada and UK between 1989 and 1993. The case studies focused on the individual grants but considered the development of the investigators and ideas involved in the research projects from initiation to the present day. Grants were selected through a stratified random selection approach that aimed to include both high- and low-impact grants. The key messages are as follows: 1) The cases reveal that a large and diverse range of impacts arose from the 29 grants studied. 2) There are variations between the impacts derived from basic biomedical and clinical research. 3) There is no correlation between knowledge production and wider impacts 4) The majority of economic impacts identified come from a minority of projects. 5) We identified factors that appear to be associated with high and low impact. This report presents the key observations of the study and an overview of the methods involved. It has been written for funders of biomedical and health research and health services, health researchers, and policy makers in those fields. It will also be of interest to those involved in research and impact evaluation.This study was initiated with internal funding from RAND Europe and HERG, with continuing funding from the UK National Institute for Health Research, the Canadian Institutes of Health Research, the Heart and Stroke Foundation of Canada and the National Heart Foundation of Australia. The UK Stroke Association and the British Heart Foundation provided support in kind through access to their archives