DiagTest3Grp: An R Package for Analyzing Diagnostic Tests with Three Ordinal Groups

Medical researchers endeavor to identify potentially useful biomarkers to develop markerbased screening assays for disease diagnosis and prevention. Useful summary measures which properly evaluate the discriminative ability of diagnostic markers are critical for this purpose. Literature and existing software, for example, R packages nicely cover summary measures for diagnostic markers used for the binary case (e.g., healthy vs. diseased). An intermediate population at an early disease stage usually exists between the healthy and the fully diseased population in many disease processes. Supporting utilities for threegroup diagnostic tests are highly desired and important for identifying patients at the early disease stage for timely treatments. However, application packages which provide summary measures for three ordinal groups are currently lacking. This paper focuses on two summary measures of diagnostic accuracy—volume under the receiver operating characteristic surface and the extended Youden index, with three diagnostic groups. We provide the R package DiagTest3Grp to estimate, under both parametric and nonparametric assumptions, the two summary measures and the associated variances, as well as the optimal cut-points for disease diagnosis. An omnibus test for multiple markers and a Wald test for two markers, on independent or paired samples, are incorporated to compare diagnostic accuracy across biomarkers. Sample size calculation under the normality assumption can be performed in the R package to design future diagnostic studies. A real world application evaluating the diagnostic accuracy of neuropsychological markers for Alzheimer’s disease is used to guide readers through step-by-step implementation of DiagTest3Grp to demonstrate its utility

Statistical Inferences for the Youden Index

In diagnostic test studies, one crucial task is to evaluate the diagnostic accuracy of a test. Currently, most studies focus on the Receiver Operating Characteristics Curve and the Area Under the Curve. On the other hand, the Youden index, widely applied in practice, is another comprehensive measurement for the performance of a diagnostic test. For a continuous-scale test classifying diseased and non-diseased groups, finding the Youden index of the test is equivalent to maximize the sum of sensitivity and specificity for all the possible values of the cut-point. This dissertation concentrates on statistical inferences for the Youden index. First, an auxiliary tool for the Youden index, called the diagnostic curve, is defined and used to evaluate the diagnostic test. Second, in the paired-design study to assess the diagnostic accuracy of two biomarkers, the difference in paired Youden indices frequently acts as an evaluation standard. We propose an exact confidence interval for the difference in paired Youden indices based on generalized pivotal quantities. A maximum likelihood estimate-based interval and a bootstrap-based interval are also included in the study. Third, for certain diseases, an intermediate level exists between diseased and non-diseased status. With such concern, we define the Youden index for three ordinal groups, propose the empirical estimate of the Youden index, study the asymptotic properties of the empirical Youden index estimate, and construct parametric and nonparametric confidence intervals for the Youden index. Finally, since covariates often affect the accuracy of a diagnostic test, therefore, we propose estimates for the Youden index with a covariate adjustment under heteroscedastic regression models for the test results. Asymptotic properties of the covariate-adjusted Youden index estimators are investigated under normal error and non-normal error assumptions

Generalization of Kullback-Leibler Divergence for Multi-Stage Diseases: Application to Diagnostic Test Accuracy and Optimal Cut-Points Selection Criterion

The Kullback-Leibler divergence (KL), which captures the disparity between two distributions, has been considered as a measure for determining the diagnostic performance of an ordinal diagnostic test. This study applies KL and further generalizes it to comprehensively measure the diagnostic accuracy test for multi-stage (K \u3e 2) diseases, named generalized total Kullback-Leibler divergence (GTKL). Also, GTKL is proposed as an optimal cut-points selection criterion for discriminating subjects among different disease stages. Moreover, the study investigates a variety of applications of GTKL on measuring the rule-in/out potentials in the single-stage and multi-stage levels. Intensive simulation studies are conducted to compare the performance of GTKL and other diagnostic accuracy measures, such as generalized Youden index (GYI), hypervolume under the manifold (HUM), and maximum absolute determinant (MADET). Furthermore, a comprehensive analysis of a real dataset is performed to illustrate the application of the proposed measure

Generalization of Net Benefit of Diagnostic Tests into Multi-stage Clinical Conditions: A Collapsing Approach

Evaluating diagnostic tests based on benefit-risk involves both the tests’ accuracy and the clinical implications of the diagnostic errors. Diagnostic tests are commonly classified into two stages: either positive or negative for a clinical condition (diseased or non-diseased). However, some diseases have more than two stages, such as Alzheimer’s. In diseases with more than two stages, the benefits and risks of the clinical consequences could differ from stage to stage. I could not find any investigations to account for the difference in benefits and risks of tests with more than two stages in the literature. The benefit to cost values for each stage of the disease could be different. This dissertation extends the net benefit approach of evaluating diagnostic tests in binary disease cases to multi-stage clinical conditions. Consequently, I extend the diagnostic yield table to multi-stage clinical conditions. I develop a decision process based on net benefit for evaluating diagnostic tests. The decision process provides additional interpretation for rule-in or rule-out clinical conditions and their adverse consequences from unnecessary workups in multi-stage diseases. Numerical examples, as well as real data, are provided to illustrate the proposed measures

Using the ROC Curve to Measure Association and Evaluate Prediction Accuracy for a Binary Outcome

This review article addresses the ROC curve and its advantage over the odds ratio to measure the association between a continuous variable and a binary outcome. A simple parametric model under the normality assumption and the method of Box-Cox transformation for non-normal data are discussed. Applications of the binormal model and the Box-Cox transformation under both univariate and multivariate inference are illustrated by a comprehensive data analysis tutorial. Finally, a summary and recommendations are given as to the usage of the binormal ROC curve

Optimal cutoff points for classification in diagnostic studies: new contributions and software development

Continuous diagnostic tests (biomarkers or risk markers) are often used to discriminate between healthy and diseased populations. For the clinical application of such tests, the key aspect is how to select an appropriate cutpoint or discrimination value c that defines positive and negative test results. In general, individuals with a diagnostic test value smaller than c are classified as healthy and otherwise as diseased. In the literature, several methods have been proposed to select the threshold value c in terms of different specific criteria of optimality. Among others, one of the methods most used in clinical practice is the Symmetry point that maximizes simultaneously both types of correct classifications. From a graphical viewpoint, the Symmetry point is associated to the operating point on the Receiver Operating Characteristic (ROC) curve that intersects the diagonal line passing through the points (0,1) and (1,0). However, this cutpoint is actually valid only when the error of misclassifying a diseased patient has the same severity than the error of misclassifying a healthy patient. Since this may not be the case in practice, an important issue in order to assess the clinical effectiveness of a biomarker is to take into account the costs associated with the decisions taken when selecting the threshold value. Moreover, to facilitate the task of selecting the optimal cut-off point in clinical practice, it is essential to have software that implements the existing optimal criteria in an user-friendly environment. Another interesting issue appears when the marker shows an irregular distribution, with a dominance of diseased subjects in noncontiguous regions. Using a single cutpoint, as common practice in traditional ROC analysis, would not be appropriate for these scenarios because it would lead to erroneous conclusions, not taking full advantage of the intrinsic classificatory capacity of the marke

Nonparametric predictive inference for diagnostic test thresholds

Measuring the accuracy of diagnostic tests is crucial in many application areas including medicine, machine learning and credit scoring. The receiver operating characteristic (ROC) curve and surface are useful tools to assess the ability of diagnostic tests to discriminate between ordered classes or groups. To define these diagnostic tests, selecting the optimal thresholds that maximize the accuracy of these tests is required. One procedure that is commonly used to find the optimal thresholds is by maximizing what is known as Youden’s index. This article presents nonparametric predictive inference (NPI) for selecting the optimal thresholds of a diagnostic test. NPI is a frequentist statistical method that is explicitly aimed at using few modeling assumptions, enabled through the use of lower and upper probabilities to quantify uncertainty. Based on multiple future observations, the NPI approach is presented for selecting the optimal thresholds for two-group and three-group scenarios. In addition, a pairwise approach has also been presented for the three-group scenario. The article ends with an example to illustrate the proposed methods and a simulation study of the predictive performance of the proposed methods along with some classical methods such as Youden index. The NPI-based methods show some interesting results that overcome some of the issues concerning the predictive performance of Youden’s index