Performance of InterVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards

Background: InterVA is a widely disseminated tool for cause of death attribution using information from verbal autopsies. Several studies have attempted to validate the concordance and accuracy of the tool, but the main limitation of these studies is that they compare cause of death as ascertained through hospital record review or hospital discharge diagnosis with the results of InterVA. This study provides a unique opportunity to assess the performance of InterVA compared to physician-certified verbal autopsies (PCVA) and alternative automated methods for analysis.Methods: Using clinical diagnostic gold standards to select 12,542 verbal autopsy cases, we assessed the performance of InterVA on both an individual and population level and compared the results to PCVA, conducting analyses separately for adults, children, and neonates. Following the recommendation of Murray et al., we randomly varied the cause composition over 500 test datasets to understand the performance of the tool in different settings. We also contrasted InterVA with an alternative Bayesian method, Simplified Symptom Pattern (SSP), to understand the strengths and weaknesses of the tool.Results: Across all age groups, InterVA performs worse than PCVA, both on an individual and population level. On an individual level, InterVA achieved a chance-corrected concordance of 24.2% for adults, 24.9% for children, and 6.3% for neonates (excluding free text, considering one cause selection). On a population level, InterVA achieved a cause-specific mortality fraction accuracy of 0.546 for adults, 0.504 for children, and 0.404 for neonates. The comparison to SSP revealed four specific characteristics that lead to superior performance of SSP. Increases in chance-corrected concordance are attained by developing cause-by-cause models (2%), using all items as opposed to only the ones that mapped to InterVA items (7%), assigning probabilities to clusters of symptoms (6%), and using empirical as opposed to expert probabilities (up to 8%).Conclusions: Given the widespread use of verbal autopsy for understanding the burden of disease and for setting health intervention priorities in areas that lack reliable vital registrations systems, accurate analysis of verbal autopsies is essential. While InterVA is an affordable and available mechanism for assigning causes of death using verbal autopsies, users should be aware of its suboptimal performance relative to other methods

Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies

<p>Abstract</p> <p>Background</p> <p>Verbal autopsy (VA) is an important method for obtaining cause of death information in settings without vital registration and medical certification of causes of death. An array of methods, including physician review and computer-automated methods, have been proposed and used. Choosing the best method for VA requires the appropriate metrics for assessing performance. Currently used metrics such as sensitivity, specificity, and cause-specific mortality fraction (CSMF) errors do not provide a robust basis for comparison.</p> <p>Methods</p> <p>We use simple simulations of populations with three causes of death to demonstrate that most metrics used in VA validation studies are extremely sensitive to the CSMF composition of the test dataset. Simulations also demonstrate that an inferior method can appear to have better performance than an alternative due strictly to the CSMF composition of the test set.</p> <p>Results</p> <p>VA methods need to be evaluated across a set of test datasets with widely varying CSMF compositions. We propose two metrics for assessing the performance of a proposed VA method. For assessing how well a method does at individual cause of death assignment, we recommend the average chance-corrected concordance across causes. This metric is insensitive to the CSMF composition of the test sets and corrects for the degree to which a method will get the cause correct due strictly to chance. For the evaluation of CSMF estimation, we propose CSMF accuracy. CSMF accuracy is defined as one minus the sum of all absolute CSMF errors across causes divided by the maximum total error. It is scaled from zero to one and can generalize a method's CSMF estimation capability regardless of the number of causes. Performance of a VA method for CSMF estimation by cause can be assessed by examining the relationship across test datasets between the estimated CSMF and the true CSMF.</p> <p>Conclusions</p> <p>With an increasing range of VA methods available, it will be critical to objectively assess their performance in assigning cause of death. Chance-corrected concordance and CSMF accuracy assessed across a large number of test datasets with widely varying CSMF composition provide a robust strategy for this assessment.</p

Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets

Background: Verbal autopsy methods are critically important for evaluating the leading causes of death in populations without adequate vital registration systems. With a myriad of analytical and data collection approaches, it is essential to create a high quality validation dataset from different populations to evaluate comparative method performance and make recommendations for future verbal autopsy implementation. This study was undertaken to compile a set of strictly defined gold standard deaths for which verbal autopsies were collected to validate the accuracy of different methods of verbal autopsy cause of death assignment.Methods: Data collection was implemented in six sites in four countries: Andhra Pradesh, India; Bohol, Philippines; Dar es Salaam, Tanzania; Mexico City, Mexico; Pemba Island, Tanzania; and Uttar Pradesh, India. The Population Health Metrics Research Consortium (PHMRC) developed stringent diagnostic criteria including laboratory, pathology, and medical imaging findings to identify gold standard deaths in health facilities as well as an enhanced verbal autopsy instrument based on World Health Organization (WHO) standards. A cause list was constructed based on the WHO Global Burden of Disease estimates of the leading causes of death, potential to identify unique signs and symptoms, and the likely existence of sufficient medical technology to ascertain gold standard cases. Blinded verbal autopsies were collected on all gold standard deaths.Results: Over 12,000 verbal autopsies on deaths with gold standard diagnoses were collected (7,836 adults, 2,075 children, 1,629 neonates, and 1,002 stillbirths). Difficulties in finding sufficient cases to meet gold standard criteria as well as problems with misclassification for certain causes meant that the target list of causes for analysis was reduced to 34 for adults, 21 for children, and 10 for neonates, excluding stillbirths. To ensure strict independence for the validation of methods and assessment of comparative performance, 500 test-train datasets were created from the universe of cases, covering a range of cause-specific compositions.Conclusions: This unique, robust validation dataset will allow scholars to evaluate the performance of different verbal autopsy analytic methods as well as instrument design. This dataset can be used to inform the implementation of verbal autopsies to more reliably ascertain cause of death in national health information systems

Nutritional characterisation of low-income households of Nairobi: socioeconomic, livestock and gender considerations and predictors of malnutrition from a cross-sectional survey

Background: In sub-Saharan Africa, urban informal settlements are rapidly expanding, leading to overcrowding and constituting challenging environments for food and water supplies, health and nutrition. The study objectives were to characterise and compare two low-income areas of Nairobi according to socioeconomic (including livestock and gender) indicators and the nutritional status of non-pregnant women of reproductive age and 1 to 3 year-old children; and to investigate socioeconomic predictors of malnutrition in these areas. Methods: In this cross-sectional survey 205 low-income households in deprived areas of Dagoretti and Korogocho (Nairobi) were randomly selected. Socioeconomic data were collected via an interviewer-administered questionnaire. Maternal and child dietary data were collected by a 24-h dietary recall. Maternal and child anthropometric and haemoglobin measurements were taken. Chi-square, t-test and Wilcoxon-Mann–Whitney test were used to compare groups and multivariable linear regression to assess predictors of malnutrition. Results: Dagoretti consistently showed better socioeconomic indicators including: income, education and occupation of household head, land ownership, housing quality and domestic asset ownership. Animal ownership was more than twice as high in Dagoretti as in Korogocho (53.0 % vs 22.9 % of households; p-value < 0.0001). A double burden of malnutrition existed: 41.5 % of children were stunted, and 29.0 % of women were overweight. In addition, 74.0 % of the children and 25.9 % of the women were anaemic, and were at risk of inadequate intakes for a number of micronutrients. Nutritional status and nutrient intakes were consistently better in Dagoretti than Korogocho; height-for-age (0.47 Z-scores higher; p-value = 0.004), the minimum dietary diversity (80.0 % vs 57.7 % in children, p-value = 0.001) and intakes of several nutrients were significantly higher. Positive predictors of maternal nutritional status were income, age and not having a premature delivery. Positive predictors of child nutritional status were area, household head education, mother not being married, female animal ownership and child’s sex (female). Conclusions: Malnutrition is prevalent in these settings, which could be partly due to low nutrient intakes, and to socioeconomic factors (including poverty), thus requiring comprehensive approaches that include increased accessibility and affordability of nutrient-dense foods. This study indicates that differences among low-income areas may need consideration for prioritisation and design of interventions

Introducing a model of cardiovascular prevention in Nairobi&#x0027;s slums by integrating a public health and private-sector approach: the SCALE-UP study

Introduction: Cardiovascular disease (CVD) is a leading cause of death in sub-Saharan Africa (SSA), with annual deaths expected to increase to 2 million by 2030. Currently, most national health systems in SSA are not adequately prepared for this epidemic. This is especially so in slum settlements where access to formal healthcare and resources is limited. Objective: To develop and introduce a model of cardiovascular prevention in the slums of Nairobi by integrating public health and private sector approaches. Study design: Two non-profit organizations that conduct public health research, Amsterdam Institute for Global Health and Development (AIGHD) and African Population and Health Research Center (APHRC), collaborated with private-sector Boston Consulting Group (BCG) to develop a service delivery package for CVD prevention in slum settings. A theoretic model was designed based on the integration of public and private sector approaches with the focus on costs and feasibility. Results: The final model includes components that aim to improve community awareness, a home-based screening service, patient and provider incentives to seek and deliver treatment specifically for hypertension, and adherence support. The expected outcomes projected by this model could prove potentially cost effective and affordable (1 USD/person/year). The model is currently being implemented in a Nairobi slum and is closely followed by key stakeholders in Kenya including the Ministry of Health, the World Health Organization (WHO), and leading non-governmental organizations (NGOs). Conclusion: Through the collaboration of public health and private sectors, a theoretically cost-effective model was developed for the prevention of CVD and is currently being implemented in the slums of Nairobi. If results are in line with the theoretical projections and first impressions on the ground, scale-up of the service delivery package could be planned in other poor urban areas in Kenya by relevant policymakers and NGOs