ROC estimation and threshold selection criteria in three-class classification problems for clustered data

Statistical evaluation of diagnostic tests, and, more generally, of biomarkers, is a constantly developing field, in which complexity of the assessment increases with complexity of the design under which data are collected. One particularly prevalent type of data is clustered data, where individual units are naturally nested into clusters. In these cases, bias can arise from omission, in the evaluation process, of cluster-level effects and/or individual covariates. Focussing on the three-class case and for continuous-valued diagnostic tests, we investigate how to exploit the clustered structure of data within a linear-mixed model approach, both when the assumption of normality holds and when it does not. We provide a method for estimation of covariate-specific ROC surfaces and discuss methods for the choice of optimal thresholds, proposing three possible estimators. A proof of consistency and asymptotic normality of the proposed threshold estimators is given. All considered methods are evaluated by extensive simulation experiments. As an application, we study the use of the Lysosomal Associated Membrane Protein Family Member 5 (Lamp5) gene expression as biomarker to distinguish among three types of glutamatergic neurons

Eyes on the prize: early economic evaluation to guide translational research:examples from the development of biomarkers for type 2 diabetes

The Center for Translational Molecular Medicine (CTMM) was a large public-private partnership that existed between 2008 and 2016. It was dedicated to the development of new technologies in molecular medicine. The main objectives of the CTMM were to reduce mortality in the Dutch population by 20% and healthcare expenditures by 1 billion Euro’s by 2019. The PREDDICCt project (€18.4 million budget) was one of the projects of the CTMM. This project aimed to develop innovative biomarker-based technologies to identify individuals at high risk of developing type 2 diabetes or its complications. In my doctoral research we evaluated the output of the PREDICCt project. To that end, we developed new methods for the early economic evaluation of biomedical innovation.We assessed in which application new biomarkers for type 2 diabetes would have the largest clinical and economic value and estimated the value of the biomarkers developed in the PREDICCt project. We have concluded that the developed biomarkers, as well as number of the applications for which the project aimed to develop new biomarkers, have very limited value and are unlikely to result in useful new technologies.Based on our research, we concluded that the main objectives of the CTMM have not been achieved. We have, however, demonstrated that using early economic evaluation it is possible to identify the most promising research strategies before the start and during translational research projects. This can prevent the waste of public and private research funding in the future

Incorporating a New Summary Statistic into the Min–Max Approach: A Min–Max–Median, Min–Max–IQR Combination of Biomarkers for Maximising the Youden Index

Linearly combining multiple biomarkers is a common practice that can provide a better diagnostic performance. When the number of biomarkers is sufficiently high, a computational burden problem arises. Liu et al. proposed a distribution-free approach (min–max approach) that linearly combines the minimum and maximum values of the biomarkers, involving only a single coefficient search. However, the combination of minimum and maximum biomarkers alone may not be sufficient in terms of discrimination. In this paper, we propose a new approach that extends that of Liu et al. by incorporating a new summary statistic, specifically, the median or interquartile range (min–max–median and min–max–IQR approaches) in order to find the optimal combination that maximises the Youden index. Although this approach is more computationally intensive than the one proposed by Liu et al, it includes more information and the number of parameters to be estimated remains reasonable. We compare the performance of the proposed approaches (min–max–median and min–max–IQR) with the min–max approach and logistic regression. For this purpose, a wide range of different simulated data scenarios were explored. We also apply the approaches to two real datasets (Duchenne Muscular Dystrophy and Small for Gestational Age)

Statistical Models for the Analysis of Complex Data

This dissertation focuses on developing statistical models to analyze complex data. The motivating applications in this work include infectious disease screening, engineering, and public health problems. Chapters 2 and 3 take a frequentist approach to modeling and parameter estimation while Chapters 4 and 5 proceed with Bayesian methods. Maximum likelihood estimation is implemented in a case of missing data through latent variables (Chapter 2) as well as by embedding a finite element model within the likelihood framework (Chapter 3). Two Markov Chain Monte Carlo (MCMC) algorithms are applied to estimate parameters and fit regression models using data obtained from a coupled system (Chapter 4) and data depending on spatial random effects (Chapter 5). In particular, spike and slab prior distributions, Gibbs steps, and Metropolis-Hastings steps are used to complete estimation. The finite sample performance of our techniques are investigated using extensive numerical simulation studies that are based on the motivating data sets. The methods are then applied to data sets on the Heptatits B infection, spring and mass systems, acceleration data from vehicle-bridge coupled systems, and opioid overdoses in South Carolina

A stepwise algorithm for linearly combining biomakers under Youden Index maximisation

Combining multiple biomarkers to provide predictive models with a greater discriminatory ability is a discipline that has received attention in recent years. Choosing the probability threshold that corresponds to the highest combined marker accuracy is key in disease diagnosis. The Youden index is a statistical metric that provides an appropriate synthetic index for diagnostic accuracy and a good criterion for choosing a cut-off point to dichotomize a biomarker. In this study, we present a new stepwise algorithm for linearly combining continuous biomarkers to maximize the Youden index. To investigate the performance of our algorithm, we analyzed a wide range of simulated scenarios and compared its performance with that of five other linear combination methods in the literature (a stepwise approach introduced by Yin and Tian, the min-max approach, logistic regression, a parametric approach under multivariate normality and a non-parametric kernel smoothing approach). The obtained results show that our proposed stepwise approach showed similar results to other algorithms in normal simulated scenarios and outperforms all other algorithms in non-normal simulated scenarios. In scenarios of biomarkers with the same means and a different covariance matrix for the diseased and non-diseased population, the min-max approach outperforms the rest. The methods were also applied on two real datasets (to discriminate Duchenne muscular dystrophy and prostate cancer), whose results also showed a higher predictive ability in our algorithm in the prostate cancer databas