Lost in Translation: Piloting a Novel Framework to Assess the Challenges in Translating Scientific Uncertainty From Empirical Findings to WHO Policy Statements

Background Calls for evidence-informed public health policy, with implicit promises of greater program effectiveness, have intensified recently. The methods to produce such policies are not self-evident, requiring a conciliation of values and norms between policy-makers and evidence producers. In particular, the translation of uncertainty from empirical research findings, particularly issues of statistical variability and generalizability, is a persistent challenge because of the incremental nature of research and the iterative cycle of advancing knowledge and implementation. This paper aims to assess how the concept of uncertainty is considered and acknowledged in World Health Organization (WHO) policy recommendations and guidelines. Methods We selected four WHO policy statements published between 2008-2013 regarding maternal and child nutrient supplementation, infant feeding, heat action plans, and malaria control to represent topics with a spectrum of available evidence bases. Each of these four statements was analyzed using a novel framework to assess the treatment of statistical variability and generalizability. Results WHO currently provides substantial guidance on addressing statistical variability through GRADE (Grading of Recommendations Assessment, Development, and Evaluation) ratings for precision and consistency in their guideline documents. Accordingly, our analysis showed that policy-informing questions were addressed by systematic reviews and representations of statistical variability (eg, with numeric confidence intervals). In contrast, the presentation of contextual or “background” evidence regarding etiology or disease burden showed little consideration for this variability. Moreover, generalizability or “indirectness” was uniformly neglected, with little explicit consideration of study settings or subgroups. Conclusion In this paper, we found that non-uniform treatment of statistical variability and generalizability factors that may contribute to uncertainty regarding recommendations were neglected, including the state of evidence informing background questions (prevalence, mechanisms, or burden or distributions of health problems) and little assessment of generalizability, alternate interventions, and additional outcomes not captured by systematic review. These other factors often form a basis for providing policy recommendations, particularly in the absence of a strong evidence base for intervention effects. Consequently, they should also be subject to stringent and systematic evaluation criteria. We suggest that more effort is needed to systematically acknowledge (1) when evidence is missing, conflicting, or equivocal, (2) what normative considerations were also employed, and (3) how additional evidence may be accrued

Developing a decision aid to guide public sector health policy decisions: A study protocol

<p>Abstract</p> <p>Background</p> <p>Decision aids have been developed in a number of health disciplines to support evidence-informed decision making, including patient decision aids and clinical practice guidelines. However, policy contexts differ from clinical contexts in terms of complexity and uncertainty, requiring different approaches for identifying, interpreting, and applying many different types of evidence to support decisions. With few studies in the literature offering decision guidance specifically to health policymakers, the present study aims to facilitate the structured and systematic incorporation of research evidence and, where there is currently very little guidance, values and other non-research-based evidence, into the policy making process. The resulting decision aid is intended to help public sector health policy decision makers who are tasked with making evidence-informed decisions on behalf of populations. The intent is not to develop a decision aid that will yield uniform recommendations across jurisdictions, but rather to facilitate more transparent policy decisions that reflect a balanced consideration of all relevant factors.</p> <p>Methods/design</p> <p>The study comprises three phases: a modified meta-narrative review, the use of focus groups, and the application of a Delphi method. The modified meta-narrative review will inform the initial development of the decision aid by identifying as many policy decision factors as possible and other features of methodological guidance deemed to be desirable in the literatures of all relevant disciplines. The first of two focus groups will then seek to marry these findings with focus group members' own experience and expertise in public sector population-based health policy making and screening decisions. The second focus group will examine issues surrounding the application of the decision aid and act as a sounding board for initial feedback and refinement of the draft decision aid. Finally, the Delphi method will be used to further inform and refine the decision aid with a larger audience of potential end-users.</p> <p>Discussion</p> <p>The product of this research will be a working version of a decision aid to support policy makers in population-based health policy decisions. The decision aid will address the need for more structured and systematic ways of incorporating various evidentiary sources where applicable.</p

Access to electronic health records by care setting and provider type: perceptions of cancer care providers in Ontario, Canada

<p>Abstract</p> <p>Background</p> <p>The use of electronic health records (EHRs) to support the organization and delivery of healthcare is evolving rapidly. However, little is known regarding potential variation in access to EHRs by provider type or care setting. This paper reports on observed variation in the perceptions of access to EHRs by a wide range of cancer care providers covering diverse cancer care settings in Ontario, Canada.</p> <p>Methods</p> <p>Perspectives were sought regarding EHR access and health record completeness for cancer patients as part of an internet survey of 5663 cancer care providers and administrators in Ontario. Data were analyzed using a multilevel logistic regression model. Provider type, location of work, and access to computer or internet were included as covariates in the model.</p> <p>Results</p> <p>A total of 1997 of 5663 (35%) valid responses were collected. Focusing on data from cancer care providers (N = 1247), significant variation in EHR access and health record completeness was observed between provider types, location of work, and level of computer access. Providers who worked in community hospitals were half as likely as those who worked in teaching hospitals to have access to their patients' EHRs (OR 0.45 95% CI: 0.24–0.85, p < 0.05) and were six times less likely to have access to other organizations' EHRs (OR 0.15 95% CI: 0.02–1.00, p < 0.05). Compared to surgeons, nurses (OR 3.47 95% CI: 1.80–6.68, p < 0.05), radiation therapists/physicists (OR 7.86 95% CI: 2.54–25.34, p < 0.05), and other clinicians (OR 4.92 95% CI: 2.15–11.27, p < 0.05) were more likely to report good access to their organization's EHRs.</p> <p>Conclusion</p> <p>Variability in access across different provider groups, organization types, and geographic locations illustrates the fragmented nature of EHR adoption in the cancer system. Along with focusing on technological aspects of EHR adoption within organizations, it is essential that there is cross-organizational and cross-provider access to EHRs to ensure patient continuity of care, system efficiency, and high quality care.</p

An exploratory cluster randomised controlled trial of knowledge translation strategies to support evidence-informed decision-making in local governments (The KT4LG study)

Background: Childhood overweight and obesity is the most prevalent and, arguably, politically complex child health problem internationally. Governments, communities and industry have important roles to play, and are increasingly expected to deliver an evidence-informed system-wide prevention program. However, efforts are impeded by a lack of organisational access to and use of research evidence. This study aims to identify feasible, acceptable and ideally, effective knowledge translation (KT) strategies to increase evidence-informed decision making in local governments, within the context of childhood obesity prevention as a national policy priority.Methods/Design: This paper describes the methods for KT4LG, a cluster randomised controlled trial which is exploratory in nature, given the limited evidence base and methodological advances. KT4LG aims to examine a program of KT strategies to increase the use of research evidence in informing public health decisions in local governments. KT4LG will also assess the feasibility and acceptability of the intervention. The intervention program comprises a facilitated program of evidence awareness, access to tailored research evidence, critical appraisal skills development, networking and evidence summaries and will be compared to provision of evidence summaries alone in the control program. 28 local governments were randomised to intervention or control, using computer generated numbers, stratified by budget tertile (high, medium or low). Questionnaires will be used to measure impact, costs, and outcomes, and key informant interviews will be used to examine processes, feasibility, and experiences. Policy tracer studies will be included to examine impact of intervention on policies within relevant government policy documents.Discussion: Knowledge translation intervention studies with a focus on public health and prevention are very few in number. Thus, this study will provide essential data on the experience of program implementation and evaluation of a system-integrated intervention program employed within the local government public health context. Standardised programs of system, organisational and individual KT strategies have not been described or rigorously evaluated. As such, the findings will make a significant contribution to understanding whether a facilitated program of KT strategies hold promise for facilitating evidence-informed public health decision making within complex multisectoral government organisations.<br /

A systematic review of frameworks for the interrelationships of mental health evidence and policy in low- and middle-income countries

Background: The interrelationships between research evidence and policy-making are complex. Different theoretical frameworks exist to explain general evidence–policy interactions. One largely unexplored element of these interrelationships is how evidence interrelates with, and influences, policy/political agenda-setting. This review aims to identify the elements and processes of theories, frameworks and models on interrelationships of research evidence and health policy-making, with a focus on actionability and agenda-setting in the context of mental health in low- and middle-income countries (LMICs). Methods: A systematic review of theories was conducted based on the BeHeMOTh search method, using a tested and refined search strategy. Nine electronic databases and other relevant sources were searched for peer-reviewed and grey literature. Two reviewers screened the abstracts, reviewed full-text articles, extracted data and performed quality assessments. Analysis was based on a thematic analysis. The included papers had to present an actionable theoretical framework/model on evidence and policy interrelationships, such as knowledge translation or evidence-based policy, specifically target the agenda-setting process, focus on mental health, be from LMICs and published in English. Results: From 236 publications included in the full text analysis, no studies fully complied with our inclusion criteria. Widening the focus by leaving out ‘agenda-setting’, we included ten studies, four of which had unique conceptual frameworks focusing on mental health and LMICs but not agenda-setting. The four analysed frameworks confirmed research gaps from LMICs and mental health, and a lack of focus on agenda-setting. Frameworks and models from other health and policy areas provide interesting conceptual approaches and lessons with regards to agenda-setting. Conclusion: Our systematic review identified frameworks on evidence and policy interrelations that differ in their elements and processes. No framework fulfilled all inclusion criteria. Four actionable frameworks are applicable to mental health and LMICs, but none specifically target agenda-setting. We have identified agenda-setting as a research theory gap in the context of mental health knowledge translation in LMICs. Frameworks from other health/policy areas could offer lessons on agenda-setting and new approaches for creating policy impact for mental health and to tackle the translational gap in LMICs

Prioritising between direct observation of therapy and case-finding interventions for tuberculosis: use of population impact measures

BACKGROUND: Population impact measures (PIMs) have been developed as tools to help policy-makers with locally relevant decisions over health risks and benefits. This involves estimating and prioritising potential benefits of interventions in specific populations. Using tuberculosis (TB) in India as an example, we examined the population impact of two interventions: direct observation of therapy and increasing case-finding. METHODS: PIMs were calculated using published literature and national data for India, and applied to a notional population of 100 000 people. Data included the incidence or prevalence of smear-positive TB and the relative risk reduction from increasing case finding and the use of direct observation of therapy (applied to the baseline risks over the next year), and the incremental proportion of the population eligible for the proposed interventions. RESULTS: In a population of 100 000 people in India, the directly observed component of the Directly Observed Treatment, Short-course (DOTS) programme may prevent 0.188 deaths from TB in the next year compared with 1.79 deaths by increasing TB case finding. The costs of direct observation are (in international dollars) I$5960 and of case finding are I$4839 or I$31702 and I$2703 per life saved respectively. CONCLUSION: Increasing case-finding for TB will save nearly 10 times more lives than will the use of the directly observed component of DOTS in India, at a smaller cost per life saved. The demonstration of the population impact, using simple and explicit numbers, may be of value to policy-makers as they prioritise interventions for their populations

Research, evidence and policymaking: the perspectives of policy actors on improving uptake of evidence in health policy development and implementation in Uganda

<p>Abstract</p> <p>Background</p> <p>Use of evidence in health policymaking plays an important role, especially in resource-constrained settings where informed decisions on resource allocation are paramount. Several knowledge translation (KT) models have been developed, but few have been applied to health policymaking in low income countries. If KT models are expected to explain evidence uptake and implementation, or lack of it, they must be contextualized and take into account the specificity of low income countries for example, the strong influence of donors. The main objective of this research is to elaborate a Middle Range Theory (MRT) of KT in Uganda that can also serve as a reference for other low- and middle income countries.</p> <p>Methods</p> <p>This two-step study employed qualitative approaches to examine the principal barriers and facilitating factors to KT. Step 1 involved a literature review and identification of common themes. The results informed the development of the initial MRT, which details the facilitating factors and barriers to KT at the different stages of research and policy development. In Step 2, these were further refined through key informant interviews with policymakers and researchers in Uganda. Deductive content and thematic analysis was carried out to assess the degree of convergence with the elements of the initial MRT and to identify other emerging issues.</p> <p>Results</p> <p>Review of the literature revealed that the most common emerging facilitating factors could be grouped under institutional strengthening for KT, research characteristics, dissemination, partnerships and political context. The analysis of interviews, however, showed that policymakers and researchers ranked institutional strengthening for KT, research characteristics and partnerships as the most important. New factors emphasized by respondents were the use of mainstreamed structures within MoH to coordinate and disseminate research, the separation of roles between researchers and policymakers, and the role of the community and civil society in KT.</p> <p>Conclusions</p> <p>This study refined an initial MRT on KT in policymaking in the health sector in Uganda that was based on a literature review. It provides a framework that can be used in empirical research of the process of KT on specific policy issues.</p

Looking for interaction: quantitative measurement of research utilization by Dutch local health officials

Background: In the Netherlands, local authorities are required by law to develop local health memoranda, based on epidemiological analyses. The purpose of this study was to assess the actual use of these epidemiological reports by municipal health officials and associated factors that affect this use.Method: Based on a conceptual framework, we designed a questionnaire in which we operationalized instrumental, conceptual, and symbolic use, the interaction between researchers and local health officials, and four clusters of barriers in this interaction process. We conducted an internet survey among 155 Dutch local health officials representing 35% of all Dutch municipalities. By means of multiple regression analyses, we gained insight into the related factors for each of the three types of research utilization.Results: The results show that local health officials use epidemiological research more often in a conceptual than an instrumental or symbolic way. This can be explained by the complexity of the local policy process which is often linked to policies in other areas, and the various policy actors involved. Conceptual use was statistically associated with a presentation given by the epidemiologist during the policy process, the presence of obstructions regarding the report's accessibility, and the local official's personal belief systems and interests originating from different professional values and responsibilities. Instrumental and symbolic use increased with the involvement of local officials in the research process.Conclusions: The results of this study provide a partial solution to understanding and influencing research utilization. The quantitative approach underpins earlier qualitative findings on this topic. The outcomes suggest that RPHS epidemiologists can use different strategies to improve research utilization. 'Blurring the boundaries', and the enhancement of interfaces between epidemiologists and local health officials, like direct interactions into each other's work processes, is expected to create better possibilities for optimizing research use

Knowledge in process? Exploring barriers between epidemiological research and local health policy development

The Redes de Trueque (RT) thrived during the economic crisis of 2001 – 2002 in Argentina and still stand out as one of the largest Complementary Currency System in the world. These local exchange networks reach a large scale during times of severe economic distress, but as large non-state initiatives, they pose a governance problem. Four types of governance systems were structured within the Argentine RT, of varying degrees of sustainability: a) loosely regulated market systems, b) hierarchies, c) associational regional networks, and d) local communities. Based on a four dimensional analytical framework, this paper discusses the rules of governance and sustainability of the governance systems in the RT. It found that some became more sustainable than others in terms of achieving combinations of scale and organisational modes