Flexible combination of multiple diagnostic biomarkers to improve diagnostic accuracy

In medical research, it is common to collect information of multiple continuous biomarkers to improve the accuracy of diagnostic tests. Combining the measurements of these biomarkers into one single score is a popular practice to integrate the collected information, where the accuracy of the resultant diagnostic test is usually improved. To measure the accuracy of a diagnostic test, the Youden index has been widely used in literature. Various parametric and nonparametric methods have been proposed to linearly combine biomarkers so that the corresponding Youden index can be optimized. Yet there seems to be little justification of enforcing such a linear combination. This paper proposes a flexible approach that allows both linear and nonlinear combinations of biomarkers. The proposed approach formulates the problem in a large margin classification framework, where the combination function is embedded in a flexible reproducing kernel Hilbert space. Advantages of the proposed approach are demonstrated in a variety of simulated experiments as well as a real application to a liver disorder study

Sparse Reject Option Classifier Using Successive Linear Programming

In this paper, we propose an approach for learning sparse reject option classifiers using double ramp loss $L_{dr}$. We use DC programming to find the risk minimizer. The algorithm solves a sequence of linear programs to learn the reject option classifier. We show that the loss $L_{dr}$ is Fisher consistent. We also show that the excess risk of loss $L_d$ is upper bounded by the excess risk of $L_{dr}$. We derive the generalization error bounds for the proposed approach. We show the effectiveness of the proposed approach by experimenting it on several real world datasets. The proposed approach not only performs comparable to the state of the art but it also successfully learns sparse classifiers

Making Risk Minimization Tolerant to Label Noise

In many applications, the training data, from which one needs to learn a classifier, is corrupted with label noise. Many standard algorithms such as SVM perform poorly in presence of label noise. In this paper we investigate the robustness of risk minimization to label noise. We prove a sufficient condition on a loss function for the risk minimization under that loss to be tolerant to uniform label noise. We show that the $0-1$ loss, sigmoid loss, ramp loss and probit loss satisfy this condition though none of the standard convex loss functions satisfy it. We also prove that, by choosing a sufficiently large value of a parameter in the loss function, the sigmoid loss, ramp loss and probit loss can be made tolerant to non-uniform label noise also if we can assume the classes to be separable under noise-free data distribution. Through extensive empirical studies, we show that risk minimization under the $0-1$ loss, the sigmoid loss and the ramp loss has much better robustness to label noise when compared to the SVM algorithm

Residual Weighted Learning for Estimating Individualized Treatment Rules

Personalized medicine has received increasing attention among statisticians, computer scientists, and clinical practitioners. A major component of personalized medicine is the estimation of individualized treatment rules (ITRs). Recently, Zhao et al. (2012) proposed outcome weighted learning (OWL) to construct ITRs that directly optimize the clinical outcome. Although OWL opens the door to introducing machine learning techniques to optimal treatment regimes, it still has some problems in performance. In this article, we propose a general framework, called Residual Weighted Learning (RWL), to improve finite sample performance. Unlike OWL which weights misclassification errors by clinical outcomes, RWL weights these errors by residuals of the outcome from a regression fit on clinical covariates excluding treatment assignment. We utilize the smoothed ramp loss function in RWL, and provide a difference of convex (d.c.) algorithm to solve the corresponding non-convex optimization problem. By estimating residuals with linear models or generalized linear models, RWL can effectively deal with different types of outcomes, such as continuous, binary and count outcomes. We also propose variable selection methods for linear and nonlinear rules, respectively, to further improve the performance. We show that the resulting estimator of the treatment rule is consistent. We further obtain a rate of convergence for the difference between the expected outcome using the estimated ITR and that of the optimal treatment rule. The performance of the proposed RWL methods is illustrated in simulation studies and in an analysis of cystic fibrosis clinical trial data.Comment: 48 pages, 3 figure

Solving a Class of Linearly Constrained Indefinite Quadratic Problems by D.C. Algorithms

International audienc

Artificial Mixture Methods for Correlated Nominal Responses and Discrete Failure Time.

Multinomial logit model with random effects is a common choice for modeling correlated nominal responses. But due to the presence of random effects and the complex form of the multinomial probabilities, the computation is often costly. We generalize the artificial mixture method for independent nominal response to correlated nominal responses. Our method transforms the complex multinomial likelihood to Poisson-type likelihoods and hence allows for the estimates to be obtained iteratively solving a set of independent low-dimensional problems. The methodology is applied to real data and studied by simulations. For discrete failure time data in large data sets, there are often many ties and a large number of distinct event time points. This poses a challenge of a high-dimensional model. We explore two ideas with the discrete proportional odds (PO) model due to its methodological and computational convenience. The log-likelihood function of discrete PO model is the difference of two convex functions; hence difference convex algorithm (DCA) carries over and brings computational efficiency. An alternative method proposed is a recursive procedure. As a result of simulation studies, these two methods work better than Quasi-Newton method in terms of both accuracy and computational time. The results from the research on the discrete PO model motivate us to develop artificial mixture methods to discrete failure time data. We consider a general discrete transformation model and mediate the high-dimensional optimization problem by changing the model form at the “complete-data” level (conditional on artificial variables). Two complete data representations are studied: proportional hazards (PH) and PO mixture frameworks. In the PH mixture framework, we reduce the high-dimensional optimization problem to many one-dimensional problems. In the PO mixture framework, both recursive solution and DCA can be synthesized into the M-step of EM algorithm leading to simplification in the optimization. PO mixture method is recommended due to its simplicity. It is applied to real data sets to fit a discrete PH and PHPH models. Simulation study fitting discrete PH model shows that the advocated PO mixture method outperforms Quasi-Newton method in terms of both accuracy and speed.Ph.D.BiostatisticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/91531/1/sfwang_1.pd

Double bundle method for finding clarke stationary points in nonsmooth dc programming

The aim of this paper is to introduce a new proximal double bundle method for unconstrained nonsmooth optimization, where the objective function is presented as a difference of two convex (DC) functions. The novelty in our method is a new escape procedure which enables us to guarantee approximate Clarke stationarity for solutions by utilizing the DC components of the objective function. This optimality condition is stronger than the criticality condition typically used in DC programming. Moreover, if a candidate solution is not approximate Clarke stationary, then the escape procedure returns a descent direction. With this escape procedure, we can avoid some shortcomings encountered when criticality is used. The finite termination of the double bundle method to an approximate Clarke stationary point is proved by assuming that the subdifferentials of DC components are polytopes. Finally, some encouraging numerical results are presented