Designing evaluation studies to optimally inform policy: what factors do policy-makers in China consider when making resource allocation decisions on healthcare worker training programmes?

BACKGROUND: In light of the gap in evidence to inform future resource allocation decisions about healthcare provider (HCP) training in low- and middle-income countries (LMICs), and the considerable donor investments being made towards training interventions, evaluation studies that are optimally designed to inform local policy-makers are needed. The aim of our study is to understand what features of HCP training evaluation studies are important for decision-making by policy-makers in LMICs. We investigate the extent to which evaluations based on the widely used Kirkpatrick model - focusing on direct outcomes of training, namely reaction of trainees, learning, behaviour change and improvements in programmatic health indicators - align with policy-makers' evidence needs for resource allocation decisions. We use China as a case study where resource allocation decisions about potential scale-up (using domestic funding) are being made about an externally funded pilot HCP training programme. METHODS: Qualitative data were collected from high-level officials involved in resource allocation at the national and provincial level in China through ten face-to-face, in-depth interviews and two focus group discussions consisting of ten participants each. Data were analysed manually using an interpretive thematic analysis approach. RESULTS: Our study indicates that Chinese officials not only consider information about the direct outcomes of a training programme, as captured in the Kirkpatrick model, but also need information on the resources required to implement the training, the wider or indirect impacts of training, and the sustainability and scalability to other settings within the country. In addition to considering findings presented in evaluation studies, we found that Chinese policy-makers pay close attention to whether the evaluations were robust and to the composition of the evaluation team. CONCLUSIONS: Our qualitative study indicates that training programme evaluations that focus narrowly on direct training outcomes may not provide sufficient information for policy-makers to make decisions on future training programmes. Based on our findings, we have developed an evidence-based framework, which incorporates but expands beyond the Kirkpatrick model, to provide conceptual and practical guidance that aids in the design of training programme evaluations better suited to meet the information needs of policy-makers and to inform policy decisions

What steps can researchers take to increase research uptake by policymakers? A case study in China.

Empirical analysis of the connections between research and health policymaking is scarce in middle-income countries. In this study, we focused on a national multidrug-resistant tuberculosis (TB) healthcare provider training programme in China as a case study to examine the role that research plays in influencing health policy. We specifically focused on the factors that influence research uptake within the complex Chinese policymaking process. Qualitative data were collected from 34 participants working at multilateral organizations, funding agencies, academia, government agencies and hospitals through 14 in-depth interviews and 2 focus group discussions with 10 participants each. Themes were derived inductively from data and grouped based on the 'Research and Policy in Developing countries' framework developed by the Overseas Development Institute. We further classified how actors derive their power to influence policy decisions following the six sources of power identified by Sriram et al. We found that research uptake by policymakers in China is influenced by perceived importance of the health issues addressed in the research, relevance of the research to policymakers' information needs and government's priorities, the research quality and the composition of the research team. Our analysis identified that international donors are influential in the TB policy process through their financial power. Furthermore, the dual roles of two government agencies as both evidence providers and actors who have the power to influence policy decisions through their technical expertise make them natural intermediaries in the TB policy process. We concluded that resolving the conflict of interests between researchers and policymakers, as suggested in the 'two-communities theory', is not enough to improve evidence use by policymakers. Strategies such as framing research to accommodate the fast-changing policy environment and making alliances with key policy actors can be effective to improve the communication of research findings into the policy process, particularly in countries undergoing rapid economic and political development

Variations in regulations to control standards for training and licensing of physicians: A multi-country comparison

Background: To strengthen health systems, the shortage of physicians globally needs to be addressed. However, efforts to increase the numbers of physicians must be balanced with controls on medical education imparted and the professionalism of doctors licensed to practise medicine.Methods: We conducted a multi-country comparison of mandatory regulations and voluntary guidelines to control standards for medical education, clinical training, licensing and re-licensing of doctors. We purposively selected seven case-study countries with differing health systems and income levels: Canada, China, India, Iran, Pakistan, UK and USA. Using an analytical framework to assess regulations at four sequential stages of the medical education to relicensing pathway, we extracted information from: systematically collected scientific and grey literature and online news articles, websites of regulatory bodies in study countries, and standardised input from researchers and medical professionals familiar with rules in the study countries.Results: The strictest controls we identified to reduce variations in medical training, licensing and re-licensing of doctors between different medical colleges, and across different regions within a country, include: medical education delivery restricted to public sector institutions; uniform, national examinations for medical college admission and licensing; and standardised national requirements for relicensing linked to demonstration of competence. However, countries analysed used different combinations of controls, balancing the strictness of controls across the four stages.Conclusions: While there is no gold standard model for medical education and practise regulation, examining the combinations of controls used in different countries enables identification of innovations and regulatory approaches to address specific contextual challenges, such as decentralisation of regulations to sub-national bodies or privatisation of medical education. Looking at the full continuum from medical education to licensing is valuable to understand how countries balance the strictness of controls at different stages. Further research is needed to understand how regulating authorities, policy-makers and medical associations can find the right balance of standardisation and context-based flexibility to produce well-rounded physicians

Is knowledge retained by healthcare providers after training? A pragmatic evaluation of drug-resistant tuberculosis management in China.

OBJECTIVES: Considering the urgent need of training to improve standardised management of drug-resistant infectious disease and the lack of evidence on the impact of training, this study evaluates whether training participants' knowledge on multidrug-resistant tuberculosis (MDR-TB) is improved immediately and a year after training. SETTING AND PARTICIPANTS: The study involved 91 MDR-TB healthcare providers (HCPs), including clinical doctors, nurses and CDC staff, who attended a new MDR-TB HCP training programme in Liaoning and Jiangxi provinces, China. MAIN OUTCOME MEASURES: A phone-based assessment of participants' long-term retention of knowledge about MDR-TB management was conducted in July 2017, approximately 1 year after training. The proportion of correct responses in the long-term knowledge assessment was compared with a pretraining test and an immediate post-training test using a χ2 test. Factors influencing participants' performance in the long-term knowledge assessment were analysed using linear regression. RESULTS: Across both provinces, knowledge of definitions of drug-resistant TB, standardised MDR-TB case detection protocols and laboratory diagnosis was improved 1 year after the training by 14.5% (p=0.037), 32.4% (p<0.001) and 31% (p<0.001) relative to pretraining. However, compared with immediately after training, the knowledge of the three topics declined by 26.5% (p=0.003), 19.8% (p=0.018) and 52.7% (p<0.001) respectively in Jiangxi, while no significant decline was observed in Liaoning. Additionally, we found that obtaining a higher score in the long-term knowledge assessment was associated with longer years of clinical experience (coefficient=0.51; 95 CI% 0.02 to 0.99; p=0.041) and attending training in Liaoning (coefficient=0.50; 95% CI 0.14 to 0.85; p=0.007). CONCLUSION: Our study, the first to assess knowledge retention of MDR-TB HCPs 1 year after training, showed an overall positive long-term impact of lecture-style group training on participants' knowledge. Knowledge decline 1 year after training was observed in one province, Jiangxi, and this may be partly addressed by targeted support to HCPs with fewer years of clinical experience

Understanding factors influencing the length of hospital stay among non-severe COVID-19 patients: A retrospective cohort study in a Fangcang shelter hospital.

As a novel concept of responding to disease epidemics, Fangcang shelter hospitals were deployed to expand the health system's capacity and provide medical services for non-severe COVID-19 patients during the outbreak in Wuhan. To give insights on patient management within Fangcang hospitals, we conducted a retrospective analysis to: 1) describe the characteristics of the patients admitted to Fangcang hospitals and 2) explore risk factors for longer length of stay (LOS). We enrolled 136 confirmed COVID-19 patients, including asymptomatic patients and those with mild symptoms, who were hospitalized in the Wuti Fangcang Hospital. 58 patients completed the treatment and discharged before 1 March 2020. After describing patients' demographic and clinical characteristics, exposure history, treatment received and time course of the disease, we conducted linear regression analysis to identify factors influencing LOS. We found that patients having fever before admission were hospitalized 3.5 days (95%CI 1.39 to 5.63, p = 0.002) longer than those without fever and that patients having bilateral pneumonia were hospitalized 3.4 days (95%CI 0.49 to 6.25, p = 0.023) longer than those with normal CT scan results. We also found weak evidence suggesting that patients with diabetes were hospitalized 3.2 days longer than those without diabetes (95%CI -0.2 to 6.56, p = 0.065). However, we observed no significant differences in LOS between symptomatic and asymptomatic patients and between patients who received treatment and those without treatment. Longer duration of hospitalization among non-severe COVID-19 patients is associated with having fever, bilateral pneumonia on CT scan and diabetes. However, being asymptomatic and using supportive medications at the early stage of infection do not have significant influences on LOS. Our study is a single-centered study with relatively small sample size. The findings provide evidence for predicting hospital bed demand in a novel response scenario and may help decision-makers in preparing for ramping up the health system capacity

Framing global discourses on non-communicable diseases: a scoping review.

BACKGROUND: The choices that policymakers make are shaped by how their problems are framed. At last, non-communicable diseases (NCDs) have risen high on the global policy agenda, but there are many disputed issues. First, what are they? Their name refers not to what they are but what they are not. Second, where do their boundaries lie? What diseases are included? Third, should we view their causes as mainly biomedical, behavioural, or social, or a combination? Our failure to resolve these issues has been invoked as a reason for our limited progress in developing and implementing effective remedies. In this scoping review, we ask "What is known from the existing literature about how NCDs are framed in the global policy discourses?" We answer it by reviewing the frames employed in policy and academic discourses. METHODS: We searched nine electronic databases for articles published since inception to 31 May 2019. We also reviewed websites of eight international organisations to identify global NCDs policies. We extracted data and synthesised findings to identify key thematic frames. RESULTS: We included 36 articles and nine policy documents on global NCDs policies. We identified five discursive domains that have been used and where there are differing perspectives. These are: "Expanding the NCDs frame to include mental health and air pollution"; "NCDs and their determinants"; "A rights-based approach to NCDs"; "Approaches to achieving policy coherence in NCDs globally"; and "NCDs as part of Sustainable Socio-economic Development". We further identified 12 frames within the five discursive domains. CONCLUSIONS: This scoping review identifies issues that remain unresolved and points to a need for alignment of perspectives among global health policy actors, as well as synergies with those working on mental health, maternal health, and child health. The current COVID-19 pandemic warrants greater consideration of its impact on global NCDs policies. Future global strategies for NCDs need to consider explicitly how NCDs are framed in a changing global health discourse and ensure adequate alignment with implementation and global health issues. There is a need for global strategies to recognise the pertinent role of actors in shaping policy discourses

Invisible medicine sellers and their use of antibiotics: a qualitative study in Cambodia.

BACKGROUND: Global attention to antimicrobial resistance has increased interest in tackling the widespread inappropriate dispensing of antibiotics by informal, for-profit healthcare providers (HCPs). We provide new evidence on an understudied group of informal HCPs: invisible medicine sellers (IMS) who operate without any marked facility. We investigated factors that influence community decisions on which HCPs to purchase medicines from, focusing on reasons for using IMS, and compared different HCPs' knowledge of antibiotic use. METHODS: We conducted community focus group discussions (FGDs) in seven purposively selected villages representing high and low informal HCPs use in two peri-urban districts in Phnom Penh, Cambodia. Using information from the FGDs to identify HCPs that sell medicines, we interviewed 35 participants: 21 HCPs (including five IMS) and 14 key informants, including government HCPs and village leaders. We adopted an interpretative approach and conducted a thematic analysis. RESULTS: Community members typically knew of several formal and informal HCPs selling medicines nearby, and IMS were common, as were doctors that sell medicines covertly. Two factors were most salient in influencing the choice of HCP for medicine purchasing. The first was trust in the effectiveness of medicines provided, judged by the speed of symptomatic relief. This pushed HCPs to provide several medicines, including antibiotics, at the first consultation. The second was the convenience offered by IMS and other informal HCPs: supplying medicines when other facilities are closed, accepting delayed payments, providing incomplete courses of medication and selling human antibiotics for animal use. CONCLUSION: This first study focusing on IMS indicates that it is important, but challenging, for public health agencies to engage with them to reduce inappropriate use of antibiotics. Although public health facilities must fill some gaps that informal HCPs are currently addressing, such as access to medicines at night, reducing demand for unnecessary antibiotics is also critical

Health financing policies during the COVID-19 pandemic and implications for universal health care: a case study of 15 countries

Background The COVID-19 pandemic was a health emergency requiring rapid scal resource mobilisation to support national responses. The use of e ective health nancing mechanisms and policies, or lack thereof, a ected the impact of the pandemic on the population, particularly vulnerable groups and individuals. We provide an overview and illustrative examples of health nancing policies adopted in 15 countries during the pandemic, develop a framework for resilient health nancing, and use this pandemic to argue a case to move towards universal health coverage (UHC). Methods In this case study, we examined the national health nancing policy responses of 15 countries, which were purposefully selected countries to represent all WHO regions and have a range of income levels, UHC index scores, and health system typologies. We did a systematic literature review of peer-reviewed articles, policy documents, technical reports, and publicly available data on policy measures undertaken in response to the pandemic and complemented the data obtained with 61 in-depth interviews with health systems and health nancing experts. We did a thematic analysis of our data and organised key themes into a conceptual framework for resilient health nancing. Findings Resilient health nancing for health emergencies is characterised by two main phases: (1) absorb and recover, where health systems are required to absorb the initial and subsequent shocks brought about by the pandemic and restabilise from them; and (2) sustain, where health systems need to expand and maintain scal space for health to move towards UHC while building on resilient health nancing structures that can better prepare health systems for future health emergencies. We observed that ve key nancing policies were implemented across the countries— namely, use of extra-budgetary funds for a swift initial response, repurposing of existing funds, e cient fund disbursement mechanisms to ensure rapid channelisation to the intended personnel and general population, mobilisation of the private sector to mitigate the gaps in public settings, and expansion of service coverage to enhance the protection of vulnerable groups. Accountability and monitoring are needed at every stage to ensure e cient and accountable movement and use of funds, which can be achieved through strong governance and coordination, information technology, and community engagement. Interpretation Our ndings suggest that health systems need to leverage the COVID-19 pandemic as a window of opportunity to make health nancing policies robust and need to politically commit to public nancing mechanisms that work to prepare for future emergencies and as a lever for UHC.We thank the management team of the Bill & Melinda Gates Foundation and the Saw Swee Hock School of Public Health (National University of Singapore, Singapore) for all the administrative support given. This research was funded by the Bill & Melinda Gates Foundation (Investment ID INV-005598)

Social media and sensemaking patterns in new product development: demystifying the customer sentiment

Artificial intelligence by principle is developed to assist but also support decision making processes. In our study, we explore how information retrieved from social media can assist decision-making processes for new product development (NPD). We focus on consumers’ emotions that are expressed through social media and analyse the variations of their sentiments in all the stages of NPD. We collect data from Twitter that reveal consumers’ appreciation of aspects of the design of a newly launched model of an innovative automotive company. We adopt the sensemaking approach coupled with the use of fuzzy logic for text mining. This combinatory methodological approach enables us to retrieve consensus from the data and to explore the variations of sentiments of the customers about the product and define the polarity of these emotions for each of the NPD stages. The analysis identifies sensemaking patterns in Twitter data and explains the NPD process and the associated steps where the social interactions from customers can have an iterative role. We conclude the paper by outlining an agenda for future research in the NPD process and the role of the customer opinion through sensemaking mechanisms