ROCnReg: An R Package for Receiver Operating Characteristic Curve Inference With and Without Covariates

This paper introduces the package ROCnReg that allows estimating the pooled ROC curve, the covariate-specific ROC curve, and the covariate-adjusted ROC curve by different methods, both from (semi) parametric and nonparametric perspectives and within Bayesian and frequentist paradigms. From the estimated ROC curve (pooled, covariate-specific, or covariate-adjusted), several summary measures of discriminatory accuracy, such as the (partial) area under the ROC curve and the Youden index, can be obtained. The package also provides functions to obtain ROC-based optimal threshold values using several criteria, namely, the Youden index criterion and the criterion that sets a target value for the false positive fraction. For the Bayesian methods, we provide tools for assessing model fit via posterior predictive checks, while the model choice can be carried out via several information criteria. Numerical and graphical outputs are provided for all methods. This is the only package implementing Bayesian procedures for ROC curves

Bayesian nonparametric inference for the covariate-adjusted ROC curve

Accurate diagnosis of disease is of fundamental importance in clinical practice and medical research. Before a medical diagnostic test is routinely used in practice, its ability to distinguish between diseased and nondiseased states must be rigorously assessed through statistical analysis. The receiver operating characteristic (ROC) curve is the most popular used tool for evaluating the discriminatory ability of continuous-outcome diagnostic tests. It has been acknowledged that several factors (e.g., subject-specific characteristics, such as age and/or gender) can affect the test's accuracy beyond disease status. Recently, the covariate-adjusted ROC curve has been proposed and successfully applied as a global summary measure of diagnostic accuracy that takes covariate information into account. We motivate the use of the covariate-adjusted ROC curve and develop a highly robust model based on a combination of B-splines dependent Dirichlet process mixture models and the Bayesian bootstrap. Multiple simulation studies demonstrate the ability of our model to successfully recover the true covariate-adjusted ROC curve and to produce valid inferences in a variety of complex scenarios. Our methods are motivated by and applied to an endocrine study where the main goal is to assess the accuracy of the body mass index, adjusted for age and gender, for predicting clusters of cardiovascular disease risk factors. The R-package AROC, implementing our proposed methods, is provided

Confidence Bands for ROC Curves: Methods and an Empirical Study

In this paper we study techniques for generating and evaluating confidence bands on ROC curves. ROC curve evaluation is rapidly becoming a commonly used evaluation metric in machine learning, although evaluating ROC curves has thus far been limited to studying the area under the curve (AUC) or generation of one-dimensional confidence intervals by freezing one variable—the false-positive rate, or threshold on the classification scoring function. Researchers in the medical field have long been using ROC curves and have many well-studied methods for analyzing such curves, including generating confidence intervals as well as simultaneous confidence bands. In this paper we introduce these techniques to the machine learning community and show their empirical fitness on the Covertype data set—a standard machine learning benchmark from the UCI repository. We show how some of these methods work remarkably well, others are too loose, and that existing machine learning methods for generation of 1-dimensional confidence intervals do not translate well to generation of simultanous bands—their bands are too tight.NYU, Stern School of Business, IOMS Department, Center for Digital Economy Researc

Extreme points of Lorenz and ROC curves with applications to inequality analysis

We find the extreme points of the set of convex functions ℓ : [0,1] → [0,1] with a fixed area and ℓ(0) = 0, ℓ(1) = 1. This collection is formed by Lorenz curves with a given value of their Gini index. The analogous set of concave functions can be viewed as Receiver Operating Characteristic (ROC) curves. These functions are extensively used in economics (inequality and risk analysis) and machine learning (evaluation of the performance of binary classifiers). We also compute the maximal L1-distance between two Lorenz (or ROC) curves with specified Gini coefficients. This result allows us to introduce a bidimensional index to compare two of such curves, in a more informative and insightful manner than with the usual unidimensional measures considered in the literature (Gini index or area under the ROC curve). The analysis of real income microdata illustrates the practical use of this proposed index in statistical inferenceA. Baíllo and J. Cárcamo are supported by the Spanish MCyT grant PID2019-109387GB-I00. C. MoraCorral is supported by the Spanish MCyT grant MTM2017-85934-C3-2-