Generalized Confidence Intervals for Partial Youden Index and its Corresponding Optimal Cut-Off Point

In the field of diagnostic test studies, the accuracy of a diagnostic test is essential in evaluating the performance of the test. The receiver operating characteristic (ROC) curve and the area under the curve (AUC) are widely used in such evaluation procedures. Meanwhile, the Youden index is also introduced into practice to measure the accuracy of the diagnostic test from another aspect. The Youden index maximizes the sum of sensitivity and specificity, assuring decent true positive and negative rates. It draws one\u27s attention due to its merit of finding the optimal cut-off points of biomarkers. Similar to Partial ROC, a new index, called Partial Youden index can be defined as an extension of Youden\u27s Index. It is more meaningful than regular Youden index since the regular one is just a special case of the Partial Youden Index. In this thesis, we focus on construction of generalized confidence intervals for the Partial Youden Index and its corresponding optimal cut-off points. Extensive simulation studies are conducted to evaluate the finite sample performances of the new intervals

Statistical Inferences for the Youden Index

In diagnostic test studies, one crucial task is to evaluate the diagnostic accuracy of a test. Currently, most studies focus on the Receiver Operating Characteristics Curve and the Area Under the Curve. On the other hand, the Youden index, widely applied in practice, is another comprehensive measurement for the performance of a diagnostic test. For a continuous-scale test classifying diseased and non-diseased groups, finding the Youden index of the test is equivalent to maximize the sum of sensitivity and specificity for all the possible values of the cut-point. This dissertation concentrates on statistical inferences for the Youden index. First, an auxiliary tool for the Youden index, called the diagnostic curve, is defined and used to evaluate the diagnostic test. Second, in the paired-design study to assess the diagnostic accuracy of two biomarkers, the difference in paired Youden indices frequently acts as an evaluation standard. We propose an exact confidence interval for the difference in paired Youden indices based on generalized pivotal quantities. A maximum likelihood estimate-based interval and a bootstrap-based interval are also included in the study. Third, for certain diseases, an intermediate level exists between diseased and non-diseased status. With such concern, we define the Youden index for three ordinal groups, propose the empirical estimate of the Youden index, study the asymptotic properties of the empirical Youden index estimate, and construct parametric and nonparametric confidence intervals for the Youden index. Finally, since covariates often affect the accuracy of a diagnostic test, therefore, we propose estimates for the Youden index with a covariate adjustment under heteroscedastic regression models for the test results. Asymptotic properties of the covariate-adjusted Youden index estimators are investigated under normal error and non-normal error assumptions

Using the ROC Curve to Measure Association and Evaluate Prediction Accuracy for a Binary Outcome

This review article addresses the ROC curve and its advantage over the odds ratio to measure the association between a continuous variable and a binary outcome. A simple parametric model under the normality assumption and the method of Box-Cox transformation for non-normal data are discussed. Applications of the binormal model and the Box-Cox transformation under both univariate and multivariate inference are illustrated by a comprehensive data analysis tutorial. Finally, a summary and recommendations are given as to the usage of the binormal ROC curve

Some Novel Statistical Inferences

In medical diagnostic studies, the area under the Receiver Operating Characteristic (ROC) curve (AUC) and Youden index are two summary measures widely used in the evaluation of the diagnostic accuracy of a medical test with continuous test results. The first half of this dissertation will highlight ROC analysis including extension of Youden index to the partial Youden index as well as novel confidence interval estimation for AUC and Youden index in the presence of covariates in induced linear regression models. Extensive simulation results show that the proposed methods perform well with small to moderate sized samples. In addition, some real examples will be presented to illustrate the methods. The latter half focuses on the application of empirical likelihood method in economics and finance. Two models draw our attention. The first one is the predictive regression model with independent and identically distributed errors. Some uniform tests have been proposed in the literature without distinguishing whether the predicting variable is stationary or nearly integrated. Here, we extend the empirical likelihood methods in Zhu, Cai and Peng (2014) with independent errors to the case of an AR error process. The proposed new tests do not need to know whether the predicting variable is stationary or nearly integrated, and whether it has a finite variance or an infinite variance. Another model we considered is a GARCH(1,1) sequence or an AR(1) model with ARCH(1) errors. It is known that the observations have a heavy tail and the tail index is determined by an estimating equation. Therefore, one can estimate the tail index by solving the estimating equation with unknown parameters replaced by Quasi Maximum Likelihood Estimation (QMLE), and profile empirical likelihood method can be employed to effectively construct a confidence interval for the tail index. However, this requires that the errors of such a model have at least finite fourth moment to ensure asymptotic normality with n1/2 rate of convergence and Wilk\u27s Theorem. We show that the finite fourth moment can be relaxed by employing some Least Absolute Deviations Estimate (LADE) instead of QMLE for the unknown parameters by noting that the estimating equation for determining the tail index is invariant to a scale transformation of the underlying model. Furthermore, the proposed tail index estimators have a normal limit with n1/2 rate of convergence under minimal moment condition, which may have an infinite fourth moment, and Wilk\u27s theorem holds for the proposed profile empirical likelihood methods. Hence a confidence interval for the tail index can be obtained without estimating any additional quantities such as asymptotic variance

Optimal cutoff points for classification in diagnostic studies: new contributions and software development

Continuous diagnostic tests (biomarkers or risk markers) are often used to discriminate between healthy and diseased populations. For the clinical application of such tests, the key aspect is how to select an appropriate cutpoint or discrimination value c that defines positive and negative test results. In general, individuals with a diagnostic test value smaller than c are classified as healthy and otherwise as diseased. In the literature, several methods have been proposed to select the threshold value c in terms of different specific criteria of optimality. Among others, one of the methods most used in clinical practice is the Symmetry point that maximizes simultaneously both types of correct classifications. From a graphical viewpoint, the Symmetry point is associated to the operating point on the Receiver Operating Characteristic (ROC) curve that intersects the diagonal line passing through the points (0,1) and (1,0). However, this cutpoint is actually valid only when the error of misclassifying a diseased patient has the same severity than the error of misclassifying a healthy patient. Since this may not be the case in practice, an important issue in order to assess the clinical effectiveness of a biomarker is to take into account the costs associated with the decisions taken when selecting the threshold value. Moreover, to facilitate the task of selecting the optimal cut-off point in clinical practice, it is essential to have software that implements the existing optimal criteria in an user-friendly environment. Another interesting issue appears when the marker shows an irregular distribution, with a dominance of diseased subjects in noncontiguous regions. Using a single cutpoint, as common practice in traditional ROC analysis, would not be appropriate for these scenarios because it would lead to erroneous conclusions, not taking full advantage of the intrinsic classificatory capacity of the marke

New Non-Parametric Confidence Interval for the Youden

Youden index, a main summary index for the Receiver Operating Characteristic (ROC) curve, is a comprehensive measurement for the effectiveness of a diagnostic test. For a continuous-scale diagnostic test, the optimal cut-point for the positive of disease is the cut-point leading to the maximization of the sum of sensitivity and specificity. Finding the Youden index of the test is equivalent to maximize the sum of sensitivity and specificity for all the possible values of the cut-point. In this thesis, we propose a new non-parametric confidence interval for the Youden index. Extensive simulation studies are conducted to compare the relative performance of the new interval with the existing intervals for the index. Our simulation results indicate that the newly developed non-parametric method performs as well as the existing parametric method but it has better finite sample performance than the existing non-parametric methods. The new method is flexible and easy to implement in practice. A real example is also used to illustrate the application of the proposed interval

Bayesian nonparametric inference for the three-class Youden index and its associated optimal cut-points

The three-class Youden index serves both as a measure of medical test accuracy and a criterion to choose the optimal pair of cutoff values for classifying subjects into three ordinal disease categories (e.g. no disease, mild disease, advanced disease). We present a Bayesian nonparametric approach for estimating the three-class Youden index and its corresponding optimal cutoff values based on Dirichlet process mixtures, which are robust models that can handle intricate features of distributions for complex data. Results from a simulation study are presented and an application to data from the Trail Making Test to assess cognitive impairment in Parkinson’s disease patients is detailed. </jats:p

GsymPoint: An R Package to Estimate the Generalized Symmetry Point, an Optimal Cut-off Point for Binary Classification in Continuous Diagnostic Tests

In clinical practice, it is very useful to select an optimal cutpoint in the scale of a continuous biomarker or diagnostic test for classifying individuals as healthy or diseased. Several methods for choosing optimal cutpoints have been presented in the literature, depending on the ultimate goal. One of these methods, the generalized symmetry point, recently introduced, generalizes the symmetry point by incorporating the misclassification costs. Two statistical approaches have been proposed in the literature for estimating this optimal cutpoint and its associated sensitivity and specificity measures, a parametric method based on the generalized pivotal quantity and a nonparametric method based on empirical likelihood. In this paper, we introduce GsymPoint, an R package that implements these methods in a user-friendly environment, allowing the end-user to calculate the generalized symmetry point depending on the levels of certain categorical covariates. The practical use of this package is illustrated using three real biomedical datasetsThis research has been supported by several Grants from the Spanish Ministry of Science and Innovation. M. López-Ratón and C. Cadarso-Suárez acknowledge support to MTM2011-15849-E, MTM2011-28285-C02-00, MTM2014-52975-C2-1-R and MTM2015-69068-REDT. E.M. Molanes-López acknowledges support to MTM2011-28285-C02-02, ECO2011-25706, MTM2011-15849-E and MTM2015-69068-REDT. E. Letón acknowledges support to MTM2011-15849-E, MTM2011-28285-C02-02, PI13/02446 and MTM2015-69068-REDTS

Statistical Inference on Optimal Points to Evaluate Multi-State Classification Systems

In decision making, an optimal point represents the settings for which a classification system should be operated to achieve maximum performance. Clearly, these optimal points are of great importance in classification theory. Not only is the selection of the optimal point of interest, but quantifying the uncertainty in the optimal point and its performance is also important. The Youden index is a metric currently employed for selection and performance quantification of optimal points for classification system families. The Youden index quantifies the correct classification rates of a classification system, and its confidence interval quantifies the uncertainty in this measurement. This metric currently focuses on two or three classes, and only allows for the utility of correct classifications and the cost of total misclassifications to be considered. An alternative to this metric for three or more classes is a cost function which considers the sum of incorrect classification rates. This new metric is preferable as it can include class prevalences and costs associated with every classification. In multi-class settings this informs better decisions and inferences on optimal points. The work in this dissertation develops theory and methods for confidence intervals on a metric based on misclassfication rates, Bayes Cost, and where possible, the thresholds found for an optimal point using Bayes Cost. Hypothesis tests for Bayes Cost are also developed to test a classification systems performance or compare systems with an emphasis on classification systems involving three or more classes. Performance of the newly proposed methods is demonstrated with simulation

The Diagnostic Value of Near-Infrared Spectroscopy to Predict Delayed Cerebral Ischemia and Unfavorable Outcome After Subarachnoid Hemorrhage

OBJECTIVE: Near-infrared spectroscopy (NIRS) is a non-invasive tool to monitor cerebral regional oxygen saturation. Impairment of microvascular circulation with subsequent cerebral hypoxia during delayed cerebral ischemia (DCI) is associated with poor functional outcome after subarachnoid hemorrhage (SAH). Therefore, NIRS could be useful to predict the risk for DCI and functional outcome. However, only limited data is available on NIRS regional cerebral tissue oxygen saturation (rSO2) distribution in SAH. The aim of this study was to compare the distribution of NIRS rSO2 values in non-traumatic SAH patients with the occurrence of DCI and functional outcome at two months. In addition, the predictive value of NIRS rSO2 was compared with the previously validated SAFIRE grade (derived from Size of the aneurysm, Age, FIsher grade, world federation of neurosurgical societies after REsuscitation).METHODS: In this study, the rSO2 distribution of patient with and without DCI after SAH are compared. The optimal cutoff points to predict DCI and outcome are assessed, and its predictive value is compared to the SAFIRE grade.RESULTS: Out of 41 patients, 12 developed DCI, and 9 had unfavorable outcome at 60 days. Prediction of DCI with NIRS had an area under the curve (AUC) of 0.77 (95%CI 0.62-0.92; p=0.0028) with an optimal cutoff point of 65% (sensitivity 1.00; specificity 0.45). Prediction of favorable outcome with NIRS had an AUC of 0.86 (95%CI 0.74-0.98; p=0.0003) with an optimal cutoff point of 63% (sensitivity 1.00; specificity 0.63). Regression analysis showed that NIRS rSO2 score is complementary to the SAFIRE grade.CONCLUSION: NIRS rSO2 monitoring in patients with SAH may improve prediction of DCI and clinical outcome after SAH.</p