Analyses of the return on investment of public health interventions: a scoping review and recommendations for future studies

Return on investment (ROI) analysis is increasingly being used for evaluating the value for money of public health interventions. Given its potential role for informing health policies, it is important that there is a more comprehensive understanding of ROI analysis within the global health field. To address this gap in the literature, we conducted a scoping review of recent research articles reporting an ROI metric for a health intervention within the public sector in any country setting. The database search was limited to literature published in English and studies published between 1 January 2018 and 14 June 2021. Uses and settings where the ROI metric is being applied, key methodological features of the calculations and the types of economic benefits included were extracted. 118 relevant studies were included within this scoping review. We found that ROI analyses of health interventions differed between those that only included fiscal savings (such as prevented medical expenses) and those which incorporated a wider range of benefits (such as monetised health benefits). This highlights the variation in the definition of ROI analyses and supports the finding that ROI analyses are used for a range of different research questions/purposes within the healthcare sector. We also found that the methodologies used in ROI calculations were inconsistent and often poorly reported. This review demonstrates that there is notable variation in the methodology surrounding recent ROI calculations of healthcare interventions, as well as the definition of ROI analysis. We recommend that ROI metrics should be carefully interpreted before they are used to inform policy decisions regarding the allocation of healthcare resources. To improve the consistency of future studies, we also set out recommended use cases for ROI analysis and a reporting checklist

The Development of the Guide to Economic Analysis and Research (GEAR) Online Resource for Low- and Middle-Income Countries' Health Economics Practitioners: A Commentary

Public health authorities around the world are increasingly using economic evaluation to set priorities and inform decision making in health policy, especially in the development of health benefit packages. Nevertheless, researchers in low- and middle-income countries (LMICs) encounter many barriers when conducting economic evaluations. In 2015, the Health Intervention and Technology Assessment Program identified key technical and context-specific challenges faced in conducting and using health economic evaluations in LMICs. On the basis of these research findings, the Guide to Economic Analysis and Research (GEAR) online resource (www.gear4health.com) was developed as a reliable aid to researchers in LMICs that would help overcome those challenges. Funded by the Thailand Research Fund and the Bill and Melinda Gates Foundation, GEAR is a free online resource that provides a visual aid tool for planning economic evaluation studies (GEAR mind maps), a repository of national and international economic evaluation guidelines (GEAR guideline comparison), and an active link to a network of volunteer international experts (GEAR: Ask an expert). GEAR will evolve over time to provide relevant, reliable, and up-to-date information through inputs from its users (e.g., periodic survey on methodological challenges) and experts (e.g., in responding to users' questions). The objective of this commentary was to give a brief description of the development and key features of this unique collective information hub aimed at facilitating high-quality research and empowering health care decision makers and stakeholders to use economic evaluation evidence

Methodological variation in economic evaluations conducted in low- and middle- income countries: Information for reference case development

© 2015 Santatiwongchai et al. Information generated from economic evaluation is increasingly being used to inform health resource allocation decisions globally, including in low- and middle- income countries. However, a crucial consideration for users of the information at a policy level, e.g. funding agencies, is whether the studies are comparable, provide sufficient detail to inform policy decision making, and incorporate inputs from data sources that are reliable and relevant to the context. This review was conducted to inform a methodological standardisation workstream at the Bill and Melinda Gates Foundation (BMGF) and assesses BMGF-funded cost-per-DALY economic evaluations in four programme areas (malaria, tuberculosis, HIV/AIDS and vaccines) in terms of variation in methodology, use of evidence, and quality of reporting. The findings suggest that there is room for improvement in the three areas of assessment, and support the case for the introduction of a standardised methodology or reference case by the BMGF. The findings are also instructive for all institutions that fund economic evaluations in LMICs and who have a desire to improve the ability of economic evaluations to inform resource allocation decisions

Cost-effectiveness modelling studies of all preventive measures against rabies: A systematic review.

Rabies is one of the most feared infectious diseases worldwide, predominantly occurring in Asia and Africa where rabies is endemic in domestic dog populations. Whereas previous studies have demonstrated mass dog vaccination and post-exposure prophylaxis (PEP) as the most effective control strategies, successful rabies elimination has yet to be realized as these recognized effective interventions continue to face challenges of limited accessibility. In the light of new evidence towards improving programmatic feasibility and clinical practice in rabies control especially among endemic countries, a systematic review was undertaken to identify cost-effectiveness modelling studies of rabies preventive measures and to provide a critical review of published evidence through comparative evaluation and model quality assessment, and a synthesis of key findings based thereon. Our search through MEDLINE and SCOPUS identified a total of 17 studies which mostly focused on estimating the impact of increasing PEP and pre-exposure prophylaxis (PrEP) access, human rabies elimination scenarios using mass dog vaccinations only or complemented with PEP strategy. While no significant methodological inconsistency across studies was identified and the extent of reporting is generally high, we note several points for quality and internal validity improvement. Assessment of modelling approach showed that decision tree models had similar pathways. The results of the studies suggest that interventions would be cost-effective at the cost-effectiveness threshold of 1 to 3 times per capita Gross Domestic Product (GDP) as recommended by the Commission on Macroeconomics and Health's GDP based thresholds, compared with no intervention in rabies endemic countries. When compared across studies which reported incremental cost-effectiveness ratio (ICER) as cost per QALY gained or DALY averted in international dollars adjusted by purchasing power parity conversion rate, PEP vaccination yields less cost per DALY averted or QALY gained due to one year-horizon assessment compared to canine vaccination at 4- or 10-year-time horizon

Tackling the 3 big challenges confronting health technology assessment development in Asia : a commentary

There has been continuous development in the field of health technology assessment (HTA) owing to the added value of HTA in supporting healthcare reimbursement decisions. Collaboration and engagement among countries in Asia has been carried out to share experiences and learning on the barriers and factors facilitating the implementation and use of HTA in policy making. A symposium on the topic of Health Technology Assessment (HTA): Selecting the Highest Value Care was held on January 10, 2019 at the National University of Singapore, during which 3 major challenges confronting HTA development in Asia were identified. The symposium also offered possible ways to overcome the challenges

An overview of the perspectives used in health economic evaluations

The term ‘perspective’ in the context of economic evaluations and costing studies in healthcare refers to the viewpoint that an analyst has adopted to define the types of costs and outcomes to consider in their studies. However, there are currently notable variations in terms of methodological recommendations, definitions, and applications of different perspectives, depending on the objective or intended user of the study. This can make it a complex area for stakeholders when interpreting these studies. Consequently, there is a need for a comprehensive overview regarding the different types of perspectives employed in such analyses, along with the corresponding implications of their use. This is particularly important, in the context of low-and-middle-income countries (LMICs), where practical guidelines may be less well-established and infrastructure for conducting economic evaluations may be more limited. This article addresses this gap by summarising the main types of perspectives commonly found in the literature to a broad audience (namely the patient, payer, health care providers, healthcare sector, health system, and societal perspectives), providing their most established definitions and outlining the corresponding implications of their uses in health economic studies, with examples particularly from LMIC settings. We then discuss important considerations when selecting the perspective and present key arguments to consider when deciding whether the societal perspective should be used. We conclude that there is no one-size-fits-all answer to what perspective should be used and the perspective chosen will be influenced by the context, policymakers'/stakeholders’ viewpoints, resource/data availability, and intended use of the analysis. Moving forward, considering the ongoing issues regarding the variation in terminology and practice in this area, we urge that more standardised definitions of the different perspectives and the boundaries between them are further developed to support future studies and guidelines, as well as to improve the interpretation and comparison of health economic evidence

Variation of health-related quality of life assessed by caregivers and patients affected by severe childhood infections.

BACKGROUND: The agreement between self-reported and proxy measures of health status in ill children is not well established. This study aimed to quantify the variation in health-related quality of life (HRQOL) derived from young patients and their carers using different instruments. METHODS: A hospital-based cross-sectional survey was conducted between August 2010 and March 2011. Children with meningitis, bacteremia, pneumonia, acute otitis media, hearing loss, chronic lung disease, epilepsy, mild mental retardation, severe mental retardation, and mental retardation combined with epilepsy, aged between five to 14 years in seven tertiary hospitals were selected for participation in this study. The Health Utilities Index Mark 2 (HUI2), and Mark 3 (HUI3), and the EuroQoL Descriptive System (EQ-5D) and Visual Analogue Scale (EQ-VAS) were applied to both paediatric patients (self-assessment) and caregivers (proxy-assessment). RESULTS: The EQ-5D scores were lowest for acute conditions such as meningitis, bacteremia, and pneumonia, whereas the HUI3 scores were lowest for most chronic conditions such as hearing loss and severe mental retardation. Comparing patient and proxy scores (n = 74), the EQ-5D exhibited high correlation (r = 0.77) while in the HUI2 and HUI3 patient and caregiver scores were moderately correlated (r = 0.58 and 0.67 respectively). The mean difference between self and proxy-assessment using the HUI2, HUI3, EQ-5D and EQ-VAS scores were 0.03, 0.05, -0.03 and -0.02, respectively. In hearing-impaired and chronic lung patients the self-rated HRQOL differed significantly from their caregivers. CONCLUSIONS: The use of caregivers as proxies for measuring HRQOL in young patients affected by pneumococcal infection and its sequelae should be employed with caution. Given the high correlation between instruments, each of the HRQOL instruments appears acceptable apart from the EQ-VAS which exhibited low correlation with the others

Comparing 3 Approaches for Making Vaccine Adoption Decisions in Thailand

Background: The World Health Organization (WHO) has developed the Total System Effectiveness (TSE) framework to assist national policy-makers in prioritizing vaccines. The pilot was launched in Thailand to explore the potential use of TSE in a country with established governance structures and accountable decision-making processes for immunization policy. While the existing literature informs vaccine adoption decisions in GAVI-eligible countries, this study attempts to address a gap in the literature by examining the policy process of a non-GAVI eligible country.Methods: A rotavirus vaccine (RVV) test case was used to compare the decision criteria made by the existing processes (Expanded Program on Immunization [EPI], and National List of Essential Medicines [NLEM]) for vaccine prioritization and the TSE-pilot model, using Thailand specific data. Results: The existing decision-making processes in Thailand and TSE were found to offer similar recommendations on the selection of a RVV product. Conclusion: The authors believe that TSE can provide a well-reasoned and step by step approach for countries, especially low- and middle-income countries (LMICs), to develop a systematic and transparent decision-making process for immunization policy

Comparison of Economic Evaluation Methods Across Low-income, Middle-income and High-income Countries: What are the Differences and Why?

There are marked differences in methods used for undertaking economic evaluations across low-income, middle-income, and high-income countries. We outline the most apparent dissimilarities and reflect on their underlying reasons. We randomly sampled 50 studies from each of three country income groups from a comprehensive database of 2844 economic evaluations published between January 2012 and May 2014. Data were extracted on ten methodological areas: (i) availability of guidelines; (ii) research questions; (iii) perspective; (iv) cost data collection methods; (v) cost data analysis; (vi) outcome measures; (vii) modelling techniques; (viii) cost-effectiveness thresholds; (ix) uncertainty analysis; and (x) applicability. Comparisons were made across income groups and odds ratios calculated. Contextual heterogeneity rightly drives some of the differences identified. Other differences appear less warranted and may be attributed to variation in government health sector capacity, in health economics research capacity and in expectations of funders, journals and peer reviewers. By highlighting these differences, we seek to start a debate about the underlying reasons why they have occurred and to what extent the differences are conducive for methodological advancements. We suggest a number of specific areas in which researchers working in countries of differing environments could learn from one another