Kernel Methods for Classification with Irregularly Sampled and Contaminated Data.

Design of a classifier consists of two stages: feature extraction and classifier learning. For a better performance, the nature, characteristics, or underlying structure of data should be taken into account in either of the stages when we design a classifier. In this thesis, we present kernel methods for classification with irregularly sampled and contaminated data. First, we propose a feature extraction method for irregularly sampled data. Irregularly sampled data often arises in medical applications where the vital signs of patients are monitored based on the severity of their condition and the availability of nursing staff. In particular, we consider an ICU (intensive care unit) admission prediction problem for a post-operative patient with possible sepsis. The experimental results show that the proposed features, when paired with kernel methods, have more discriminating power than those used by clinicians. Second, we consider one-class classification problem with contaminated data, where the majority of the data comes from a "nominal" distribution with a small fraction of the data coming from an outlying distribution. We deal with this problem by robustly estimating the nominal density (or a level set thereof) from the contaminated data. Our proposed density estimation achieves robustness by combinining a traditional kernel density estimator (KDE) with ideas from classical M-estimation. The robustness of the density estimator is demonstrated with a representer theorem, the influence function, and experimental results. Third, we propose a kernel classifier that optimizes the L_2 distances between "difference of densities". Like a support vector machine (SVM), the classifier is sparse and results from solving a quadratic program. We also provide statistical performance guarantees for the proposed L_2 kernel classifier in the form of a finite sample oracle inequality, and strong consistency in the sense of both ISE and probability of error.Ph.D.Electrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/89858/1/stannum_1.pd

Discriminative Densities from Maximum Contrast Estimation

Meinicke P, Twellmann T, Ritter H. Discriminative Densities from Maximum Contrast Estimation. In: Becker S, Thrun S, Obermayer K, eds. Advances in Neural Information Processing Systems 15. Proceedings of the 2002 conference. Cambridge, Mass.: MIT-Press; 2003: 1009-1018.We propose a framework for classifier design based on discriminative densities for representation of the differences of the class-conditional distributions in a way that is optimal for classification. The densities are selected from a parametrized set by constrained maximization of some objective function which measures the average (bounded) difference, i.e. the contrast between discriminative densities. We show that maximiza- tion of the contrast is equivalent to minimization of an approximation of the Bayes risk. Therefore using suitable classes of probability density functions, the resulting maximum contrast classifiers(MCCs) can approximate the Bayes rule for the general multiclass case. In particular for a certain parametrization of the density functions we obtain MCCs which have the same functional form as the well-known Support Vector Machines (SVMs). We show that MCC-training in general requires some nonlinear optimization but under certain conditions the problem is concave and can be tackled by a single linear program. We indicate the close relation between SVM- and MCC-training and in particular we show that Linear Programming Machines can be viewed as an approxi- mate realization of MCCs. In the experiments on benchmark data sets, the MCC shows a competitive classification performance