Finagle’s laws of information: lessons learnt evaluating a complex health intervention in Nigeria

Evaluations cannot support evidence-informed decision making if they do not provide the information needed by decision-makers. In this article, we reflect on our own difficulties evaluating the Geo-Referenced Infrastructure and Demographic Data for Development (GRID3) approach, an intervention that provides high-resolution demographic and geographical information to support health service delivery. GRID3 was implemented in Nigeria’s northern states to support polio (2012–2019) and measles immunisation campaigns (2017–2018). Generalising from our experience we argue that Finagle’s four laws of information capture a particular set of challenges when evaluating complex interventions: the weak causal claims derived from quasi-experimental studies and secondary analyses of existing data (the information we have is not what we want); the limited external validity of counterfactual impact evaluations (the information we want is not what we need); the absence of reliable monitoring data on implementation processes (the information we need is not what we can obtain) and the overly broad scope of evaluations attempting to generate both proof of concept and evidence for upscaling (the information we can obtain costs more than we want to pay). Evaluating complex interventions requires a careful selection of methods, thorough analyses and balanced judgements. Funders, evaluators and implementers share a joint responsibility for their success

Response and resilience in rural Bangladesh in the context of COVID-19

This record includes an extended abstract and MP4 presentation. Presented at the 42nd WEDC International Conference

A Spatial Analysis Framework to Monitor and Accelerate Progress towards SDG 3 to End TB in Bangladesh

Global efforts to end the tuberculosis (TB) epidemic by 2030 (SDG3.3) through improved TB case detection and treatment have not been effective to significantly reduce the global burden of the TB epidemic. This study presents an analytical framework to evaluate the use of TB case notification rates (CNR) to monitor and to evaluate TB under-detection and under-diagnoses in Bangladesh. Local indicators of spatial autocorrelation (LISA) were calculated to assess the presence and scale of spatial clusters of TB CNR across 489 upazilas in Bangladesh. Simultaneous autoregressive models were fit to the data to identify associations between TB CNR and poverty, TB testing rates and retreatment rates. CNRs were found to be significantly spatially clustered, negatively correlated to poverty rates and positively associated to TB testing and retreatment rates. Comparing the observed pattern of CNR with model-standardized rates made it possible to identify areas where TB under-detection is likely to occur. These results suggest that TB CNR is an unreliable proxy for TB incidence. Spatial variations in TB case notifications and subnational variations in TB case detection should be considered when monitoring national TB trends. These results provide useful information to target and prioritize context specific interventions

WASH, nutrition and health-seeking behavior during COVID-19 lockdowns: Evidence from rural Bangladesh

A general lockdown to minimize to slow transmission of COVID-19 in Bangladesh came into effect on March 26th and lasted until May 30th. The lockdown had far-reaching economic implications for the population, with many facing economic hardship due to loss of income. Despite the attempt of the government to ease economic hardship by means of social safety net packages, people suffered from poor access to health services, and financial and food insecurity. This is likely to have disastrous consequences for the nutritional status of young children. This cross-sectional study measured the impact of the first general lockdown on food consumption of young children, access to water, handwashing and health seeking behavior, and the ability to maintain livelihood among households with children under the age of 5, in rural Bangladesh. The result of the analysis suggest that loss of income was reported by almost all respondents across all socio-economic groups. However, the poorest households were less likely to provide for sufficient food for their families and had to reduce consumption of food. Diet diversity and food intake–particularly animal protein sources—for young children were severely affected. On the other, increased awareness of handwashing and access to soap were also reported. The pandemic is likely to be detrimental to the nutritional status of children in Bangladesh and can exacerbate existing health inequities. Strong social safety net programs are needed to protect vulnerable populations to consequences of restrictive measures, supported in design and implementation by non-governmental organizations

Third party monitoring for health in Afghanistan: the good, the bad and the ugly

Third party monitoring (TPM) is used in development programming to assess deliverables in a contract relationship between purchasers (donors or government) and providers (non-governmental organisations or non-state entities). In this paper, we draw from our experience as public health professionals involved in implementing and monitoring the Basic Package of Health Services (BPHS) and the Essential Package of Hospital Services (EPHS) as part of the SEHAT and Sehatmandi programs in Afghanistan between 2013 and 2021. We analyse our own TPM experience through the lens of the three parties involved: the Ministry of Public Health; the service providers implementing the BPHS/EPHS; and the TPM agency responsible for monitoring the implementation. Despite the highly challenging and fragile context, our findings suggest that the consistent investments and strategic vision of donor programmes in Afghanistan over the past decades have led to a functioning and robust system to monitor the BPHS/EPHS implementation in Afghanistan. To maximise the efficiency, effectiveness and impact of this system, it is important to promote local ownership and use of the data, to balance the need for comprehensive information with the risk of jamming processes, and to address political economy dynamics in pay-for-performance schemes. Our findings are likely to be emblematic of TPM issues in other sectors and other fragile and conflicted affected settings and offer a range of lessons learnt to inform the implementation of TPM schemes

COVID-19 morbidity in Afghanistan: a nationwide, population-based seroepidemiological study

Objective The primary objectives were to determine the magnitude of COVID-19 infections in the general population and age-specific cumulative incidence, as determined by seropositivity and clinical symptoms of COVID-19, and to determine the magnitude of asymptomatic or subclinical infections.Design, setting and participants We describe a population-based, cross-sectional, age-stratified seroepidemiological study conducted throughout Afghanistan during June/July 2020. Participants were interviewed to complete a questionnaire, and rapid diagnostic tests were used to test for SARS-CoV-2 antibodies. This national study was conducted in eight regions of Afghanistan plus Kabul province, considered a separate region. The total sample size was 9514, and the number of participants required in each region was estimated proportionally to the population size of each region. For each region, 31–44 enumeration areas (EAs) were randomly selected, and a total of 360 clusters and 16 households per EA were selected using random sampling. To adjust the seroprevalence for test sensitivity and specificity, and seroreversion, Bernoulli’s model methodology was used to infer the population exposure in Afghanistan.Outcome measures The main outcome was to determine the prevalence of current or past COVID-19 infection.Results The survey revealed that, to July 2020, around 10 million people in Afghanistan (31.5% of the population) had either current or previous COVID-19 infection. By age group, COVID-19 seroprevalence was reported to be 35.1% and 25.3% among participants aged ≥18 and 5–17 years, respectively. This implies that most of the population remained at risk of infection. However, a large proportion of the population had been infected in some localities, for example, Kabul province, where more than half of the population had been infected with COVID-19.Conclusion As most of the population remained at risk of infection at the time of the study, any lifting of public health and social measures needed to be considered gradually