The Causal Roadmap and simulation studies to inform the Statistical Analysis Plan for real-data applications

The Causal Roadmap outlines a systematic approach to our research endeavors: define quantity of interest, evaluate needed assumptions, conduct statistical estimation, and carefully interpret of results. At the estimation step, it is essential that the estimation algorithm be chosen thoughtfully for its theoretical properties and expected performance. Simulations can help researchers gain a better understanding of an estimator's statistical performance under conditions unique to the real-data application. This in turn can inform the rigorous pre-specification of a Statistical Analysis Plan (SAP), not only stating the estimand (e.g., G-computation formula), the estimator (e.g., targeted minimum loss-based estimation [TMLE]), and adjustment variables, but also the implementation of the estimator -- including nuisance parameter estimation and approach for variance estimation. Doing so helps ensure valid inference (e.g., 95% confidence intervals with appropriate coverage). Failing to pre-specify estimation can lead to data dredging and inflated Type-I error rates

Applying the causal roadmap to longitudinal national Danish registry data: a case study of second-line diabetes medication and dementia

The causal roadmap is a formal framework for causal and statistical inference that supports clear specification of the causal question, interpretable and transparent statement of required causal assumptions, robust inference, and optimal precision. The roadmap is thus particularly well-suited to evaluating longitudinal causal effects using large scale registries; however, application of the roadmap to registry data also introduces particular challenges. In this paper we provide a detailed case study of the longitudinal causal roadmap applied to the Danish National Registry to evaluate the comparative effectiveness of second-line diabetes drugs on dementia risk. Specifically, we evaluate the difference in counterfactual five-year cumulative risk of dementia if a target population of adults with type 2 diabetes had initiated and remained on GLP-1 receptor agonists (a second-line diabetes drug) compared to a range of active comparator protocols. Time-dependent confounding is accounted for through use of the iterated conditional expectation representation of the longitudinal g-formula as a statistical estimand. Statistical estimation uses longitudinal targeted maximum likelihood, incorporating machine learning. We provide practical guidance on the implementation of the roadmap using registry data, and highlight how rare exposures and outcomes over long-term follow up can raise challenges for flexible and robust estimators, even in the context of the large sample sizes provided by the registry. We demonstrate how simulations can be used to help address these challenges by supporting careful estimator pre-specification. We find a protective effect of GLP-1RAs compared to some but not all other second-line treatments

Management practices in community-based HIV prevention organizations in Nigeria

Background: Nigeria has one of the largest Human Immunodeficiency Virus (HIV) epidemics in the world. Addressing the epidemic of HIV in such a high-burden country has necessitated responses of a multidimensional nature. Historically, community-based organizations (CBOs) have played an essential role in targeting key populations (eg. men who have sex with men, sex workers) that are particularly burdened by HIV. CBOs are an essential part of the provision of health services in sub-Saharan Africa, but very little is known about the management practices of CBOs that provide HIV prevention interventions. Methods: We interviewed 31 CBO staff members and other key stakeholders in January 2017 about management practices in CBOs. Management was conceptualized under the classical management process perspective; these four management phases—planning, organizing, leading, and evaluating—guided the interview process and code development. Data analysis was conducted thematically using Atlas.ti software. The protocol was approved by the ethics committees of the National Institute of Public Health of Mexico (INSP), the National Agency for the Control of AIDS in Nigeria (NACA), and the Nigerian Institute for Medical Research (NIMR). Results: We found that CBOs implement variable management practices that can either hinder or facilitate the efficient provision of HIV prevention services. Long-standing CBOs had relatively strong organizational infrastructure and capacity that positively influenced service planning. In contrast, fledgling CBOs were deficient of organizational infrastructure and lacked program planning capacity. The delivery of HIV services can become more efficient if management practices are taken into account. Conclusions: The delivery of HIV services by CBOs in Nigeria was largely influenced by inherent issues related to skills, organizational structure, talent retention, and sanction application. These, in turn, affected management practices such as planning, organizing, leading, and evaluating. This study shows that KP-led CBOs are evolving and have strong potentials and capacity for growth, and can become more efficient and effective if attention is paid to issues such as hierarchy, staff recruitment, and talent retention

Evaluating the impact of community health worker integration into prevention of mother-to-child transmission of HIV services in Tanzania

De-identified Stata dataset and do-files used to publish the report to 3ie on the project, "Evaluating the impact of community health worker integration into prevention of mother-to-child transmission of HIV services in Tanzania" (project code TW7.18). This project was funded under Thematic Window 7 on integrated HIV services

Lifestyle-related education and counseling resource utilization and cardiovascular biomarkers in midlife women with low physical activity

Health plan-based resources are promising avenues for decreasing cardiovascular disease risk. This study examined associations of lifestyle-related resource utilization within a healthcare delivery system and cardiovascular biomarkers among midlife women with low physical activity. Midlife women (45-55 years old) with <10 min/week of reported physical activity at a primary care visit within a large integrated healthcare delivery system in Northern California in 2015 (n = 55,393) were identified. Within this cohort, subsequent lifestyle-related health education and individual coaching resource utilization, and the next recorded physical activity, weight, systolic blood pressure, plasma glucose, HDL and LDL cholesterol measures up to 2 years after the index primary care visit were identified from electronic health records. We used a multilevel linear model to estimate associations. About 3% (n = 1587) of our cohort had ≥1 lifestyle-related resource encounter; 0.3% (n = 178) had ≥ 4 encounters. Participation in ≥4 lifestyle-related resource encounters (compared to none) was associated with 51 more minutes/week of physical activity (95% CI: 33,69) at the next clinical measurement in all women, 6.2 kg lower weight (95% CI: −7.0,-5.5) at the next measurement in women with obesity, and 8–10 mg/dL lower plasma glucose (95% CI: −30,14 and −23,2, respectively) at the next measurement in women with diabetes or prediabetes. Our results support the sustained utilization of health plan-based lifestyle-related resources for improving physical activity, weight, and plasma glucose in high-risk midlife women. Given the observed low utilization, health system-wide efforts may be warranted to increase utilization of lifestyle-related resources in this population

The relationship between management practices and the efficiency and quality of voluntary medical male circumcision services in four African countries.

INTRODUCTION:Given constrained funding for Human Immunodeficiency Virus (HIV) programs across Sub-Saharan Africa, delivering services efficiently is paramount. Voluntary medical male circumcision (VMMC) is a key intervention that can substantially reduce heterosexual transmission-the primary mode of transmission across the continent. There is limited research, however, on what factors may contribute to the efficient and high-quality execution of such programs. METHODS:We analyzed a multi-country, multi-stage random sample of 108 health facilities providing VMMC services in sub-Saharan Africa in 2012 and 2013. The survey collected information on inputs, outputs, process quality and management practices from facilities providing VMMC services. We analyzed the relationship between management practices, quality (measured through provider vignettes) and efficiency (estimated through data envelopment analysis) using Generalized Linear Models and Mixed-effects Models. Applying multivariate regression models, we assessed the relationship between management indices and efficiency and quality of VMMC services. RESULTS:Across countries, both efficiency and quality varied widely. After adjusting for type of facility, country and scale, performance-base funding was negatively correlated with efficiency -0.156 (p < 0.05). In our analysis, we did not find any significant relationships between quality and management practices. CONCLUSIONS:No significant relationship was found between process quality and management practices across 108 VMMC facilities. This study is the first to analyze the potential relationships between management and service quality and efficiency among a sample of VMMC health facilities in sub-Saharan Africa and can potentially inform policy-relevant hypotheses to later test through prospective experimental studies

Short-term effectiveness of a community health worker intervention for HIV-infected pregnant women in Tanzania to improve treatment adherence and retention in care: A cluster-randomized trial.

IntroductionCommunity health workers (CHWs) are lay workers who have the potential to enhance services to prevent mother-to-child HIV transmission (PMTCT) and improve the health of women living with HIV infection. We conducted a cluster-randomized trial of an intervention to integrate CHWs with 'Option B+' PMTCT services in Shinyanga Region, Tanzania.MethodsThe intervention was implemented for 11 months and included four integrated components: 1) formal linkage of CHWs to health facilities; 2) CHW-led antiretroviral therapy (ART) adherence counseling; 3) loss to follow-up tracing by CHWs; and 4) distribution of Action Birth Cards (ABCs), a birth planning tool. We cluster-randomized 32 facilities offering PMTCT services, within strata of size, to the intervention (n = 15) or comparison (standard of care, n = 17) groups. Intervention effectiveness was determined with a difference-in-differences strategy based on clinical and pharmacy data from HIV-infected postpartum women at baseline (births in 2014) and endline (births April-Oct 2015). The primary outcome was retention in care between 60 and 120 days postpartum. Secondary outcomes included ART initiation, timing of ART initiation (as measured by week of gestation), and ART adherence 90 days postpartum, measured using the medication possession ratio (MPR≥95%).ResultsIntervention and comparison facilities were similar at baseline. Data were collected from 1,152 and 678 mother-infant pairs at baseline and endline, respectively. There were no significant differences in retention in care, ART initiation, or timing of ART initiation between the intervention and control groups. Adherence (MPR≥95%) at 90 days postpartum was 11.3 percentage points higher in the intervention group in ITT analyses (95% CI: -0.7, 23.3, p = 0.06), though this effect was attenuated after adjusting for baseline imbalance (9.5 percentage points, 95% CI: -2.9, 22.0, p = 0.13). Among only sites that had the greatest fidelity to the intervention, however, we found a stronger effect on adherence (13.6 percentage points, 95% CI: 2.5, 24.6).ConclusionsDespite being feasible and acceptable, the CHW-based intervention did not have strong effects on most PMTCT indicators. CHW involvement in PMTCT programs may improve ART adherence among HIV-infected postpartum women, however, and success appears heavily dependent on program implementation.Trial registrationRegistry for International Development Impact Evaluations (RIDIE, ID 552553838b402) and ClinicalTrials.gov (NCT03058484)

The relationship between management practices and the efficiency and quality of voluntary medical male circumcision services in four African countries

INTRODUCTION:Given constrained funding for Human Immunodeficiency Virus (HIV) programs across Sub-Saharan Africa, delivering services efficiently is paramount. Voluntary medical male circumcision (VMMC) is a key intervention that can substantially reduce heterosexual transmission-the primary mode of transmission across the continent. There is limited research, however, on what factors may contribute to the efficient and high-quality execution of such programs. METHODS:We analyzed a multi-country, multi-stage random sample of 108 health facilities providing VMMC services in sub-Saharan Africa in 2012 and 2013. The survey collected information on inputs, outputs, process quality and management practices from facilities providing VMMC services. We analyzed the relationship between management practices, quality (measured through provider vignettes) and efficiency (estimated through data envelopment analysis) using Generalized Linear Models and Mixed-effects Models. Applying multivariate regression models, we assessed the relationship between management indices and efficiency and quality of VMMC services. RESULTS:Across countries, both efficiency and quality varied widely. After adjusting for type of facility, country and scale, performance-base funding was negatively correlated with efficiency -0.156 (p &lt; 0.05). In our analysis, we did not find any significant relationships between quality and management practices. CONCLUSIONS:No significant relationship was found between process quality and management practices across 108 VMMC facilities. This study is the first to analyze the potential relationships between management and service quality and efficiency among a sample of VMMC health facilities in sub-Saharan Africa and can potentially inform policy-relevant hypotheses to later test through prospective experimental studies