How Can We Better Evaluate Complex Global Health Initiatives? Reflections From the January 2014 Institute of Medicine Workshop.

An IOM workshop on evaluation design drew on recent evaluations of 4 complex initiatives (PEPFAR; the Global Fund to Fight AIDS, TB and Malaria; the President’s Malaria Initiative; and the Affordable Medicines Facility-malaria). Key components for good evaluations: (1) a robust theory of change to understand how and why programs should work; (2) use of multiple analytic methods; and (3) triangulation of evidence to validate and deepen understanding of results as well as synthesis of findings to identify lessons for scale-up or broader application

Using a theory of change approach to analyze global health diplomacy practice in Myanmar

The current picture of global health diplomacy (GHD) is one of increasing complexity, with multiple actors striving for multiple objectives, using a multitude of strategies and activities. Analysis and documentation of GHD practice is urgently needed to help identify the opportunities GHD provides for jointly improving global health and international relations, as well as possible unintended consequences. Systematic analysis of GHD practice is challenging without a conceptual framework. We identified a Theory of Change approach as one way to conceptualize GHD practice and potential impacts that could also depict its complexity and identify relationships and pathways for measuring success. We hypothesized that an implicit ToC underlying GHD practice existed, that could be identified and made explicit. In this paper, we develop a template ToC for GHD based on existing literature. We discuss the concepts and methods used to develop this “implicit” ToC template for GHD, and use the case of US-Myanmar relations to test the ToC against an example of real practice. We conclude with a discussion of how a ToC approach can: provide clarity on the complex relationships in GHD; help articulate desired outcomes for GHD; and, systematically capture contextual factors, stakeholder motivations, and contributions to GHD objectives. We argue that this single case of making an explicit ToC for GHD demonstrates the potential for developing a more generally relevant ToC for future GHD efforts. Finally, we propose three immediate ways in which the ToC approach could contribute to future GHD practice and assessment

Tracking Global Fund HIV/AIDS resources used for sexual and reproductive health service integration: case study from Ethiopia

Objective/Background The Global Fund to Fight AIDS, Tuberculosis & Malaria (GF) strives for high value for money, encouraging countries to integrate synergistic services and systems strengthening to maximize investments. The GF needs to show how, and how much, its grants support more than just HIV/AIDS, TB and malaria. Sexual and Reproductive Health (SRH) has been part of HIV/AIDS grants since 2007. Previous studies showed the GF PBF system does not allow resource tracking for SRH integration within HIV/AIDS grants. We present findings from a resource tracking case study using primary data collected at country level. Methods Ethiopia was the study site. We reviewed data from four HIV/AIDS grants from January 2009-June 2011 and categorized SDAs and activities as directly, indirectly, or not related to SRH integration. Data included: GF PBF data; financial, performance, in-depth interview and facility observation data from Ethiopia. Results All HIV/AIDS grants in Ethiopia support SRH integration activities (12-100%). Using activities within SDAs, expenditures directly supporting SRH integration increased from 25% to 66% for the largest HIV/AIDS grant, and from 21% to 34% for the smaller PMTCT-focused grant. Using SDAs to categorize expenditures underestimated direct investments in SRH integration; activity-based categorization is more accurate. The important finding is that primary data collection could not resolve the limitations in using GF GPR data for resource tracking. The remedy is to require existing activity-based budgets and expenditure reports as part of PBF reporting requirements, and make them available in the grant portfolio database. The GF should do this quickly, as it is a serious shortfall in the GF guiding principle of transparency. Conclusions Showing high value for money is important for maximizing impact and replenishments. The Global Fund should routinely track HIV/AIDs grant expenditures to disease control, service integration, and overall health systems strengthening. The current PBF system will not allow this. Real-time expenditure analysis could be achieved by integrating existing activity-based financial data into the routine PBF system. The GF’s New Funding Model and the 2012-2016 strategy present good opportunities for over-hauling the PBF system to improve transparency and allow the GF to monitor and maximize value for money

Hospital ships adrift? Part 2: The role of U.S. Navy hospital ship humanitarian assistance missions in building partnerships

Introduction US Navy hospital ships are used as a foreign policy instrument to achieve various objectives that include building partnerships. Despite substantial resource investment by the Department of Defense (DoD) in these missions, their impact is unclear. The purpose of this study was to understand how and why hospital ship missions influence partnerships among the different participants. Methods An embedded case study was used and included the hospital ship Mercy\u27s mission to Timor-Leste in 2008 and 2010 with four units of analysis: the US government, partner nation, host nation, and nongovernmental organizations. Key stakeholders representing each unit were interviewed using open-ended questions that explored the experiences of each participant and their organization. Findings were analyzed using a priori domains from a proposed partnership theoretical framework. A documentary review of key policy, guidance, and planning documents was also conducted. Results Fifteen themes related to how and why hospital ship missions influence partnerships emerged from the 37 interviews and documentary review. The five most prominent included: developing relationships, developing new perspectives, sharing resources, understanding partner constraints, and developing credibility. Facilitators to joining the mission included partner nations seeking a regional presence and senior executive relationships. Enablers included historical relationships and host nation receptivity. The primary barrier to joining was the military leading the mission. Internal constraints included the short mission duration, participant resentment, and lack of personnel continuity. External constraints included low host nation and United States Agency for International Development (USAID) capacity. Conclusion The research finds the idea of building partnerships exists among most units of analysis. However, the results show a delay in downstream effects of generating action and impact among the participants. Without a common partnership definition and policy, guidance, and planning documents reinforcing these constructs, achieving the partnership goal will remain challenging. Efforts should be made to magnify the facilitators and enablers while developing mitigation strategies for the barriers and constraints. This is the first study to scientifically assess the partnership impact of hospital ship missions and could support the DoD\u27s effort to establish, enable, and sustain meaningful partnerships. Application of the findings to improve partnerships in contexts beyond hospital ship missions may be warranted and require further analysis. This unique opportunity could bridge the rift with humanitarian actors and establish, enable, and sustain meaningful partnerships with the DoD

Scaling up context-tailored clinical guidelines and training to improve childbirth care in urban, low-resource maternity units in Tanzania: A protocol for a stepped-wedged cluster randomized trial with embedded qualitative and economic analyses (The PartoMa Scale-Up Study)

While facility births are increasing in many low-resource settings, quality of care often does not follow suit; maternal and perinatal mortality and morbidity remain unacceptably high. Therefore, realistic, context-tailored clinical support is crucially needed to assist birth attendants in resource-constrained realities to provide best possible evidence-based and respectful care. Our pilot study in Zanzibar suggested that co-created clinical practice guidelines (CPGs) and low-dose, high-frequency training (PartoMa intervention) were associated with improved childbirth care and survival. We now aim to modify, implement, and evaluate this multi-faceted intervention in five high-volume, urban maternity units in Dar es Salaam, Tanzania (approximately 60,000 births annually). This PartoMa Scale-up Study will include four main steps: I. Mixed-methods situational analysis exploring factors affecting care; II. Co-created contextual modifications to the pilot CPGs and training, based on step I; III. Implementation and evaluation of the modified intervention; IV. Development of a framework for co-creation of context-specific CPGs and training, of relevance in comparable fields. The implementation and evaluation design is a theory-based, stepped-wedged cluster-randomised trial with embedded qualitative and economic assessments. Women in active labour and their offspring will be followed until discharge to assess provided and experienced care, intra-hospital perinatal deaths, Apgar scores, and caesarean sections that could potentially be avoided. Birth attendants\u27 perceptions, intervention use and possible associated learning will be analysed. Moreover, as further detailed in the accompanying article, a qualitative in-depth investigation will explore behavioural, biomedical, and structural elements that might interact with non-linear and multiplying effects to shape health providers\u27 clinical practices. Finally, the incremental cost-effectiveness of co-creating and implementing the PartoMa intervention is calculated. Such real-world scale-up of context-tailored CPGs and training within an existing health system may enable a comprehensive understanding of how impact is achieved or not, and how it may be translated between contexts and sustained.Trial registration number: NCT04685668

How Can We Better Evaluate Complex Global Health Initiatives? Reflections From the January 2014 Institute of Medicine Workshop.

An IOM workshop on evaluation design drew on recent evaluations of 4 complex initiatives (PEPFAR; the Global Fund to Fight AIDS, TB and Malaria; the President’s Malaria Initiative; and the Affordable Medicines Facility-malaria). Key components for good evaluations: (1) a robust theory of change to understand how and why programs should work; (2) use of multiple analytic methods; and (3) triangulation of evidence to validate and deepen understanding of results as well as synthesis of findings to identify lessons for scale-up or broader application

Health System Determinants of Maternal Mortality in Tanzania: An analysis of regional disparities in health system capacity and its impact on maternal mortality in Tanzania.

Introduction: The Government of Tanzania has made a commitment to reduce the maternal mortality ratio (MMR) in Tanzania from 410 deaths per 100,000 live births in 2015, to 292 by 2020. This target is based on raising the percentage of facility-assisted deliveries and improving access to and quality of emergency obstetric care at the dispensary and health center level (MOHSW, 2015). Census data from 2012 show great regional disparity in MMR with the highest and lowest regions differing by a factor of 5. This cross-sectional study investigates whether regional health systems capacity contributes to the variability in regional MMR. Methods: The study uses regression analysis to correlate regional health system usage and quality factors identified in the Service Provision Assessment (SPA) and Demographic and Health Survey (DHS), both from 2015, with sub-national maternal mortality data collected in Tanzania\u27s thirty administrative regions during the 2012 Household and Population Census. Health system factors included quality variables for dispensary and health center readiness to deliver basic emergency obstetric care (BEmOC), which were created based on fifty-six factors taken from the SPA. Socio-economic and variables related to service access barriers from the DHS were included to isolate the partial contribution of health system factors to variability in regional MMR. Results: Facility-assisted Delivery was the health system factor most highly correlated with MMR, but not with a linear relationship. In fact, the direction of the correlation only becomes negative after facility-assisted delivery reaches 65% coverage. (p=0.001) the factor most significantly correlated to higher levels of coverage of facility-assisted delivery was contraceptive prevalence (mCPR) (p=0.000). Dispensary readiness was also a contributor to facility-assisted delivery (p=0.001), even when controlling for maternal education. Conclusions: Facility-assisted delivery is a critical path to achieving targeted reductions in MMR; however, drivers of increased facility-assisted delivery are not only the readiness of facilities to provide quality BEmOC, but reducing the burden on facilities and physical risk factors for women by increasing the use of modern contraception. While the findings from this study support the strategic priorities of Tanzania\u27s One Plan II, an impact on MMR cannot be expected until all regions of the country achieve a 65% level of facility-assisted delivery. This will likely require regionally specific strategic plans to increase mCPR and dispensary readiness

Factors influencing the sustainability of neglected tropical disease elimination programs: A multi-case study of the Kenya national program for elimination of lymphatic filariasis

© 2020 by The American Society of Tropical Medicine and Hygiene Elimination of a disease is the holy grail in global health. The pathology of several neglected tropical diseases (NTDs) such as lymphatic filariasis (LF) makes elimination a reality. However, successful elimination requires that NTD programs be sustainable-the ability to confirm that the disease has been eliminated and the capacity to ensure that it does not return. The WHO\u27s guidelines on NTDs thoroughly detail how to reach elimination. Notwithstanding, comprehensive guidance regarding contextual and programmatic factors that influence sustainability is lacking. Moreover, a comprehensive NTD sustainability framework that includes these factors is nonexistent. This research aimed to develop a framework that identified the critical programmatic and contextual factors influencing sustainability of NTD elimination programs. The methodology included a literature review and a multiple case study. The literature review had two objectives: the first was to identify unique attributes of NTD elimination programs that should be considered when planning for sustainability and the second was to identify an existing sustainability framework that could be adapted to NTD elimination programs. The literature review resulted in a draft sustainability framework that was then tested, using a multicase study methodology, on the Kenya National Program for Elimination of Lymphatic Filariasis. The result is the first comprehensive NTD sustainability framework that identifies key contextual and programmatic factors that influence NTD sustainability. This framework is an invaluable resource for practitioners and policy makers alike as many countries start to eliminate NTDs