Estimating sensitivity and specificity of diagnostic tests using latent class models that account for conditional dependence between tests: a simulation study

BACKGROUND: Latent class models are increasingly used to estimate the sensitivity and specificity of diagnostic tests in the absence of a gold standard, and are commonly fitted using Bayesian methods. These models allow us to account for 'conditional dependence' between two or more diagnostic tests, meaning that the results from tests are correlated even after conditioning on the person's true disease status. The challenge is that it is not always clear to researchers whether conditional dependence exists between tests and whether it exists in all or just some latent classes. Despite the increasingly widespread use of latent class models to estimate diagnostic test accuracy, the impact of the conditional dependence structure chosen on the estimates of sensitivity and specificity remains poorly investigated. METHODS: A simulation study and a reanalysis of a published case study are used to highlight the impact of the conditional dependence structure chosen on estimates of sensitivity and specificity. We describe and implement three latent class random-effect models with differing conditional dependence structures, as well as a conditional independence model and a model that assumes perfect test accuracy. We assess the bias and coverage of each model in estimating sensitivity and specificity across different data generating mechanisms. RESULTS: The findings highlight that assuming conditional independence between tests within a latent class, where conditional dependence exists, results in biased estimates of sensitivity and specificity and poor coverage. The simulations also reiterate the substantial bias in estimates of sensitivity and specificity when incorrectly assuming a reference test is perfect. The motivating example of tests for Melioidosis highlights these biases in practice with important differences found in estimated test accuracy under different model choices. CONCLUSIONS: We have illustrated that misspecification of the conditional dependence structure leads to biased estimates of sensitivity and specificity when there is a correlation between tests. Due to the minimal loss in precision seen by using a more general model, we recommend accounting for conditional dependence even if researchers are unsure of its presence or it is only expected at minimal levels

Accuracy of the direct agglutination test for diagnosis of visceral leishmaniasis: a systematic review and meta-analysis

Background:Parasitological investigation of bone marrow, splenic or lymph node aspirations is the gold standard for the diagnosis of visceral leishmaniasis (VL). However, this invasive test requires skilled clinical and laboratory staff and adequate facilities, and sensitivity varies depending on the tissue used. The direct agglutination test (DAT) is a serological test that does not need specialised staff, with just minimal training required. While previous meta-analysis has shown DAT to have high sensitivity and specificity when using parasitology as the reference test for diagnosis, meta-analysis of DAT compared to other diagnostic techniques, such as PCR and ELISA, that are increasingly used in clinical and research settings, has not been done. Methods: We conducted a systematic review to determine the diagnostic performance of DAT compared to all available tests for the laboratory diagnosis of human VL. We searched electronic databases including Medline, Embase, Global Health, Scopus, WoS Science Citation Index, Wiley Cochrane Central Register of Controlled Trials, Africa-Wide Information, LILACS and WHO Global Index. Three independent reviewers screened reports and extracted data from eligible studies. A meta-analysis estimated the diagnostic sensitivity and specificity of DAT. Results: Of 987 titles screened, 358 were selected for full data extraction and 78 were included in the analysis, reporting on 32,822 participants from 19 countries. Studies included were conducted between 1987–2020. Meta-analysis of studies using serum and DAT compared to any other test showed pooled sensitivity of 95% (95%CrI 90–98%) and pooled specificity of 95% (95%CrI 88–98%). Results were similar for freeze-dried DAT and liquid DAT when analysed separately. Sensitivity was lower for HIV-positive patients (90%, CrI 59–98%) and specificity was lower for symptomatic patients (70%, CrI 43–89%). When comparing different geographical regions, the lowest median sensitivity (89%, CrI 67–97%) was in Western Asia (five studies). Conclusions: This systematic review and meta-analysis demonstrates high estimated pooled sensitivity and specificity of DAT for diagnosis of VL, although sensitivity and specificity were lower for different patient groups and geographical locations. This review highlights the lack of standardisation of DAT methods and preparations, and the lack of data from some important geographical locations. Future well-reported studies could provide better evidence to inform test implementation for different patient populations and use cases. PROSPERO registration: CRD4202124083

Biotic factors limit the invasion of the plague pathogen ( <i>Yersinia pestis</i> ) in novel geographical settings

Aim: The distribution of Yersinia pestis, the pathogen that causes plague in humans, is reliant upon transmission between host species; however, the degree to which host species distributions dictate the distribution of Y. pestis, compared with limitations imposed by the environmental niche of Y. pestis per se, is debated. We test whether the present-day environmental niche of Y. pestis differs between its native range and an invaded range and whether biotic factors (host distributions) can explain observed discrepancies. Location: North America and Central Asia. Major taxa studied: Yersinia pestis. Methods: We use environmental niche models to determine whether the current climatic niche of Y. pestis differs between its native range in Asia and its invaded range in North America. We then test whether the inclusion of information on the distribution of host species improves the ability of models to capture the North American niche. We use geographical null models to guard against spurious correlations arising from spatially autocorrelated occurrence points. Results: The current climatic niche of Y. pestis differs between its native and invaded regions. The Asian niche overpredicted the distribution of Y. pestis across North America. Including biotic factors along with the native climatic niche increased niche overlap between the native and invaded models, and models containing only biotic factors performed better than the native climatic niche alone. Geographical null models confirmed that the increased niche overlap through inclusion of biotic factors did not, with a couple of exceptions, arise solely from spatially autocorrelated occurrences. Main conclusions: The current climatic niche in Central Asia differs from the current climatic niche in North America. Inclusion of biotic factors improved the fit of models to the Y. pestis distribution data in its invaded region better than climate variables alone. This highlights the importance of host species when investigating zoonotic disease introductions and suggests that climatic variables alone are insufficient to predict disease distribution in novel environments

Diagnostic performance of the IMMY cryptococcal antigen lateral flow assay on serum and cerebrospinal fluid for diagnosis of cryptococcosis in HIV-negative patients: a systematic review.

BACKGROUND: The incidence of cryptococcosis amongst HIV-negative persons is increasing. Whilst the excellent performance of the CrAg testing in people living with HIV is well described, the diagnostic performance of the CrAg LFA has not been systematically evaluated in HIV-negative cohorts on serum or cerebrospinal fluid. METHODS: We performed a systematic review to characterise the diagnostic performance of IMMY CrAg® LFA in HIV-negative populations on serum and cerebrospinal fluid. A systematic electronic search was performed using Medline, Embase, Global Health, CENTRAL, WoS Science Citation Index, SCOPUS, Africa-Wide Information, LILACS and WHO Global Health Library. Studies were screened and data extracted from eligible studies by two independent reviewers. A fixed effect meta-analysis was used to estimate the diagnostic sensitivity and specificity. RESULTS: Of 447 records assessed for eligibility, nine studies met our inclusion criteria, including 528 participants overall. Amongst eight studies that evaluated the diagnostic performance of the IMMY CrAg® LFA on serum, the pooled median sensitivity was 96% (95% Credible Interval (CrI) 68-100%) with a pooled specificity estimate of 96% (95%CrI 84-100%). Amongst six studies which evaluated the diagnostic performance of IMMY CrAg® LFA on CSF, the pooled median sensitivity was 99% (95%CrI 95-100%) with a pooled specificity median of 99% (95%CrI 95-100%). CONCLUSIONS: This review demonstrates a high pooled sensitivity and specificity for the IMMY CrAg® LFA in HIV-negative populations, in keeping with findings in HIV-positive individuals. The review was limited by the small number of studies. Further studies using IMMY CrAg® LFA in HIV-negative populations would help to better determine the diagnostic value of this test

Diagnostic accuracy of multiplex respiratory pathogen panels for influenza or respiratory syncytial virus infections: systematic review and meta-analysis.

Respiratory syncytial virus (RSV) and influenza viruses are important global causes of morbidity and mortality. We evaluated the diagnostic accuracy of the Luminex NxTAG respiratory pathogen panels (RPPs)™ (index) against other RPPs (comparator) for detection of RSV and influenza viruses. Studies comparing human clinical respiratory samples tested with the index and at least one comparator test were included. A random-effect latent class meta-analysis was performed to assess the specificity and sensitivity of the index test for RSV and influenza. Risk of bias was assessed using the QUADAS-2 tool and certainty of evidence using GRADE. Ten studies were included. For RSV, predicted sensitivity was 99% (95% credible interval [CrI] 96-100%) and specificity 100% (95% CrI 98-100%). For influenza A and B, predicted sensitivity was 97% (95% CrI 89-100) and 98% (95% CrI 88-100) respectively; specificity 100% (95% CrI 99-100) and 100% (95% CrI 99-100), respectively. Evidence was low certainty. Although index sensitivity and specificity were excellent, comparators' performance varied. Further research with clear patient recruitment strategies could ascertain performance across different populations.Protocol Registration: Prospero CRD42021272062

Statistical analysis plan for a cluster randomised trial in Madhya Pradesh, India: community health promotion and medical provision and impact on neonates (CHAMPION2).

BACKGROUND: Neonatal mortality in India has fallen steadily and was estimated to be 24 per 1000 live births in the year 2017. However, neonatal mortality remains high in rural parts of the country. The Community Health Promotion and Medical Provision and Impact On Neonates (CHAMPION2) trial investigates the effect of a complex health intervention on neonatal mortality in the Satna District of Madhya Pradesh. METHODS/DESIGN: The CHAMPION2 trial forms one part of a cluster-randomised controlled trial with villages (clusters) randomised to receive either a health (CHAMPION2) or education (STRIPES2) intervention. Villages receiving the health intervention are controls for the education intervention and vice versa. The primary outcome is neonatal mortality. The effect of the active intervention on the primary outcome (compared to usual care) will be expressed as a risk ratio, estimated using a generalised estimating equation approach with robust standard errors that take account of clustering at village level. Secondary outcomes include maternal mortality, stillbirths, perinatal deaths, causes of death, health care and knowledge, hospital admissions of enrolled women during pregnancy or in the immediate post-natal care period or of their babies (during the neonatal period), maternal blood transfusions, and the cost effectiveness of the intervention. A total of 196 villages have been randomised and over 34,000 women have been recruited in CHAMPION2. DISCUSSION: This update to the published trial protocol gives a detailed plan for the statistical analysis of the CHAMPION2 trial. TRIAL REGISTRATION: Registry of India: CTRI/2019/05/019296. Registered on 23 May 2019. https://ctri.nic.in/Clinicaltrials/pmaindet2.php?EncHid=MzExOTg=&Enc=&userName=champion2

Mapping malaria by sharing spatial information between incidence and prevalence data sets

As malaria incidence decreases and more countries move towards elimination, maps of malaria risk in low-prevalence areas are increasingly needed. For low-burden areas, disaggregation regression models have been developed to estimate risk at high spatial resolution from routine surveillance reports aggregated by administrative unit polygons. However, in areas with both routine surveillance data and prevalence surveys, models that make use of the spatial information from prevalence point-surveys might make more accurate predictions. Using case studies in Indonesia, Senegal and Madagascar, we compare the out-of-sample mean absolute error for two methods for incorporating point-level, spatial information into disaggregation regression models. The first simply fits a binomial-likelihood, logit-link, Gaussian random field to prevalence point-surveys to create a new covariate. The second is a multi-likelihood model that is fitted jointly to prevalence point-surveys and polygon incidence data. We find that in most cases there is no difference in mean absolute error between models. In only one case, did the new models perform the best. More generally, our results demonstrate that combining these types of data has the potential to reduce absolute error in estimates of malaria incidence but that simpler baseline models should always be fitted as a benchmark

Mapping malaria seasonality in Madagascar using health facility data.

BACKGROUND: Many malaria-endemic areas experience seasonal fluctuations in case incidence as Anopheles mosquito and Plasmodium parasite life cycles respond to changing environmental conditions. Identifying location-specific seasonality characteristics is useful for planning interventions. While most existing maps of malaria seasonality use fixed thresholds of rainfall, temperature, and/or vegetation indices to identify suitable transmission months, we construct a statistical modelling framework for characterising the seasonal patterns derived directly from monthly health facility data. METHODS: With data from 2669 of the 3247 health facilities in Madagascar, a spatiotemporal regression model was used to estimate seasonal patterns across the island. In the absence of catchment population estimates or the ability to aggregate to the district level, this focused on the monthly proportions of total annual cases by health facility level. The model was informed by dynamic environmental covariates known to directly influence seasonal malaria trends. To identify operationally relevant characteristics such as the transmission start months and associated uncertainty measures, an algorithm was developed and applied to model realisations. A seasonality index was used to incorporate burden information from household prevalence surveys and summarise 'how seasonal' locations are relative to their surroundings. RESULTS: Positive associations were detected between monthly case proportions and temporally lagged covariates of rainfall and temperature suitability. Consistent with the existing literature, model estimates indicate that while most parts of Madagascar experience peaks in malaria transmission near March-April, the eastern coast experiences an earlier peak around February. Transmission was estimated to start in southeast districts before southwest districts, suggesting that indoor residual spraying should be completed in the same order. In regions where the data suggested conflicting seasonal signals or two transmission seasons, estimates of seasonal features had larger deviations and therefore less certainty. CONCLUSIONS: Monthly health facility data can be used to establish seasonal patterns in malaria burden and augment the information provided by household prevalence surveys. The proposed modelling framework allows for evidence-based and cohesive inferences on location-specific seasonal characteristics. As health surveillance systems continue to improve, it is hoped that more of such data will be available to improve our understanding and planning of intervention strategies

Spatiotemporal mapping of malaria prevalence in Madagascar using routine surveillance and health survey data.

Malaria transmission in Madagascar is highly heterogeneous, exhibiting spatial, seasonal and long-term trends. Previous efforts to map malaria risk in Madagascar used prevalence data from Malaria Indicator Surveys. These cross-sectional surveys, conducted during the high transmission season most recently in 2013 and 2016, provide nationally representative prevalence data but cover relatively short time frames. Conversely, monthly case data are collected at health facilities but suffer from biases, including incomplete reporting and low rates of treatment seeking. We combined survey and case data to make monthly maps of prevalence between 2013 and 2016. Health facility catchment populations were estimated to produce incidence rates from the case data. Smoothed incidence surfaces, environmental and socioeconomic covariates, and survey data informed a Bayesian prevalence model, in which a flexible incidence-to-prevalence relationship was learned. Modelled spatial trends were consistent over time, with highest prevalence in the coastal regions and low prevalence in the highlands and desert south. Prevalence was lowest in 2014 and peaked in 2015 and seasonality was widely observed, including in some lower transmission regions. These trends highlight the utility of monthly prevalence estimates over the four year period. By combining survey and case data using this two-step modelling approach, we were able to take advantage of the relative strengths of each metric while accounting for potential bias in the case data. Similar modelling approaches combining large datasets of different malaria metrics may be applicable across sub-Saharan Africa

Indirect effects of the COVID-19 pandemic on malaria intervention coverage, morbidity, and mortality in Africa: a geospatial modelling analysis.

BACKGROUND: Substantial progress has been made in reducing the burden of malaria in Africa since 2000, but those gains could be jeopardised if the COVID-19 pandemic affects the availability of key malaria control interventions. The aim of this study was to evaluate plausible effects on malaria incidence and mortality under different levels of disruption to malaria control. METHODS: Using an established set of spatiotemporal Bayesian geostatistical models, we generated geospatial estimates across malaria-endemic African countries of the clinical case incidence and mortality of malaria, incorporating an updated database of parasite rate surveys, insecticide-treated net (ITN) coverage, and effective treatment rates. We established a baseline estimate for the anticipated malaria burden in Africa in the absence of COVID-19-related disruptions, and repeated the analysis for nine hypothetical scenarios in which effective treatment with an antimalarial drug and distribution of ITNs (both through routine channels and mass campaigns) were reduced to varying extents. FINDINGS: We estimated 215·2 (95% uncertainty interval 143·7-311·6) million cases and 386·4 (307·8-497·8) thousand deaths across malaria-endemic African countries in 2020 in our baseline scenario of undisrupted intervention coverage. With greater reductions in access to effective antimalarial drug treatment, our model predicted increasing numbers of cases and deaths: 224·1 (148·7-326·8) million cases and 487·9 (385·3-634·6) thousand deaths with a 25% reduction in antimalarial drug coverage; 233·1 (153·7-342·5) million cases and 597·4 (468·0-784·4) thousand deaths with a 50% reduction; and 242·3 (158·7-358·8) million cases and 715·2 (556·4-947·9) thousand deaths with a 75% reduction. Halting planned 2020 ITN mass distribution campaigns and reducing routine ITN distributions by 25%-75% also increased malaria burden to a total of 230·5 (151·6-343·3) million cases and 411·7 (322·8-545·5) thousand deaths with a 25% reduction; 232·8 (152·3-345·9) million cases and 415·5 (324·3-549·4) thousand deaths with a 50% reduction; and 234·0 (152·9-348·4) million cases and 417·6 (325·5-553·1) thousand deaths with a 75% reduction. When ITN coverage and antimalarial drug coverage were synchronously reduced, malaria burden increased to 240·5 (156·5-358·2) million cases and 520·9 (404·1-691·9) thousand deaths with a 25% reduction; 251·0 (162·2-377·0) million cases and 640·2 (492·0-856·7) thousand deaths with a 50% reduction; and 261·6 (167·7-396·8) million cases and 768·6 (586·1-1038·7) thousand deaths with a 75% reduction. INTERPRETATION: Under pessimistic scenarios, COVID-19-related disruption to malaria control in Africa could almost double malaria mortality in 2020, and potentially lead to even greater increases in subsequent years. To avoid a reversal of two decades of progress against malaria, averting this public health disaster must remain an integrated priority alongside the response to COVID-19. FUNDING: Bill and Melinda Gates Foundation; Channel 7 Telethon Trust, Western Australia