Assessing the impact of health technology assessment in the Netherlands

Copyright © Cambridge University Press 2008Objectives: Investments in health research should lead to improvements in health and health care. This is also the remit of the main HTA program in the Netherlands. The aims of this study were to assess whether the results of this program have led to such improvements and to analyze how best to assess the impact from health research.Methods: We assessed the impact of individual HTA projects by adapting the "payback framework" developed in the United Kingdom. We conducted dossier reviews and sent a survey to principal investigators of forty-three projects awarded between 2000 and 2003. We then provided an overview of documented output and outcome that was assessed by ten HTA experts using a scoring method. Finally, we conducted five case studies using information from additional dossier review and semistructured key informant interviews.Results: The findings confirm that the payback framework is a useful approach to assess the impact of HTA projects. We identified over 101 peer reviewed papers, more than twenty-five PhDs, citations of research in guidelines (six projects), and implementation of new treatment strategies (eleven projects). The case studies provided greater depth and understanding about the levels of impact that arise and why and how they have been achieved.Conclusions: It is generally too early to determine whether the HTA program led to actual changes in healthcare policy and practice. However, the results can be used as a baseline measurement for future evaluation and can help funding organizations or HTA agencies consider how to assess impact, possibly routinely. This, in turn, could help inform research strategies and justify expenditure for health research.This research is funded by ZonMw, the Netherlands organization for health research and development (project 945-15-001)

Steps toward preregistration of research on research integrity

Abstract Background A proposal to encourage the preregistration of research on research integrity was developed and adopted as the Amsterdam Agenda at the 5th World Conference on Research Integrity (Amsterdam, 2017). This paper reports on the degree to which abstracts of the 6th World Conference in Research Integrity (Hong Kong, 2019) reported on preregistered research. Methods Conference registration data on participants presenting a paper or a poster at 6th WCRI were made available to the research team. Because the data set was too small for inferential statistics this report is limited to a basic description of results and some recommendations that should be considered when taking further steps to improve preregistration. Results 19% of the 308 presenters preregistered their research. Of the 56 usable cases, less than half provided information on the six key elements of the Amsterdam Agenda. Others provided information that invalidated their data, such as an uninformative URL. There was no discernable difference between qualitative and quantitative research. Conclusions Some presenters at the WCRI have preregistered their research on research integrity, but further steps are needed to increase frequency and completeness of preregistration. One approach to increase preregistration would be to make it a requirement for research presented at the World Conferences on Research Integrity.http://deepblue.lib.umich.edu/bitstream/2027.42/174042/1/41073_2021_Article_108.pd

How to combine rules and commitment in fostering research integrity?

Research integrity (RI) is crucial for producing research that is trustworthy and of high quality. Rules are important in setting RI standards, improving research practice and fostering responsible research practices. At the same time, rules can lead to increased bureaucracy, which without commensurate increased commitment amongst researchers towards RI is unlikely to lead to more responsible research behavior. In this paper, we explore the question: How can rules and commitment be combined to foster RI? There are three ways that research institutions can govern RI: markets (governing through incentives), hierarchies or bureaucracies (governing through rules), and network processes (governing through commitment and agreement at group level). Based on Habermas’ Theory of Communicative Action, we argue that network processes focusing on consensus, as part of the lifeworld, are necessary to legitimize and support systems, i.e. market and bureaucratic modes of governance. We analyze the institutional response to a serious RI case to illustrate how network processes can create a context in which rules can foster RI. Specifically, we analyze how the Science Committee established at Tilburg University in 2012 has navigated and combined different modes of governance to foster RI. Based on this case analysis, we formulate recommendations to research institutions on how to combine rules and commitment

Prevalence of questionable research practices, research misconduct and their potential explanatory factors:A survey among academic researchers in The Netherlands

Prevalence of research misconduct, questionable research practices (QRPs) and their associations with a range of explanatory factors has not been studied sufficiently among academic researchers. The National Survey on Research Integrity targeted all disciplinary fields and academic ranks in the Netherlands. It included questions about engagement in fabrication, falsification and 11 QRPs over the previous three years, and 12 explanatory factor scales. We ensured strict identity protection and used the randomized response method for questions on research misconduct. 6,813 respondents completed the survey. Prevalence of fabrication was 4.3% (95% CI: 2.9, 5.7) and of falsification 4.2% (95% CI: 2.8, 5.6). Prevalence of QRPs ranged from 0.6% (95% CI: 0.5, 0.9) to 17.5% (95% CI: 16.4, 18.7) with 51.3% (95% CI: 50.1, 52.5) of respondents engaging frequently in at least one QRP. Being a PhD candidate or junior researcher increased the odds of frequently engaging in at least one QRP, as did being male. Scientific norm subscription (odds ratio (OR) 0.79; 95% CI: 0.63, 1.00) and perceived likelihood of detection by reviewers (OR 0.62, 95% CI: 0.44, 0.88) were associated with engaging in less research misconduct. Publication pressure was associated with more often engaging in one or more QRPs frequently (OR 1.22, 95% CI: 1.14, 1.30). We found higher prevalence of misconduct than earlier surveys. Our results suggest that greater emphasis on scientific norm subscription, strengthening reviewers in their role as gatekeepers of research quality and curbing the “publish or perish” incentive system promotes research integrity

Selective citation in the literature on the hygiene hypothesis:A citation analysis on the association between infections and rhinitis

Objective Our objective was to assess the occurrence and determinants of selective citation in scientific publications on Strachan's original hygiene hypothesis. His hypothesis states that lack of exposure to infections in early childhood increases the risk of rhinitis. Setting Web of Science Core Collection. Participants We identified 110 publications in this network, consisting of 5551 potential citations. Primary and secondary outcome measures Whether a citation occurs or not, measured and analysed according to the preregistered protocol. Results We found evidence for citation bias in this field: publications supportive of the hypothesis were cited more often than non-supportive publications (OR adjusted for study design [adjOR] 2.2, 95% CI 1.6 to 3.1), and the same was the case for publications with mixed findings (adjOR 3.1, 95% CI 2.2 to 4.5). Other relevant determinants for citation were type of exposure, specificity, journal impact factor, authority and self-citation. Surprisingly, prospective cohort studies were cited less often than other empirical studies. Conclusions There is clear evidence for selective citation in this research field, and particularly for citation bias