Multiclass Learning with Simplex Coding

In this paper we discuss a novel framework for multiclass learning, defined by a suitable coding/decoding strategy, namely the simplex coding, that allows to generalize to multiple classes a relaxation approach commonly used in binary classification. In this framework, a relaxation error analysis can be developed avoiding constraints on the considered hypotheses class. Moreover, we show that in this setting it is possible to derive the first provably consistent regularized method with training/tuning complexity which is independent to the number of classes. Tools from convex analysis are introduced that can be used beyond the scope of this paper

Flexible margin-based classification techniques

Classification is a very useful statistical tool for information extraction. Among numerous classification methods, margin-based classification techniques have attracted a lot of attention. It can be typically expressed as a general minimization problem in the form of $loss + penalty$, where the loss function controls goodness of fit of the training data and the penalty term enforces smoothness of the model. Since the loss function decides how functional margins affect the resulting margin-based classifier, one can modify the existing loss functions to obtain classifiers with desirable properties. In this research, we design several new margin-based classifiers, via modifying loss functions of two well-known classifiers, Penalized Logistic Regression (PLR) and the Support Vector Machine (SVM). In particular, we propose three new binary classification techniques, Robust Penalized Logistic Regression (RPLR), Bounded Constraint Machine (BCM), and the Balancing Support Vector Machine (BSVM). For multicategory case, we propose the multicagegory Composite Least Squares (CLS) classifier, a new multicategory classifier based on the squared loss function. We study properties of the new methods and provide efficient computational algorithms. Simulated and microarray gene expression data analysis examples are used to demonstrate competitive performance of the proposed methods

From Coherent Structures to Universal Properties

Given a 2-category \twocat{K} admitting a calculus of bimodules, and a 2-monad T on it compatible with such calculus, we construct a 2-category \twocat{L} with a 2-monad S on it such that: (1)S has the adjoint-pseudo-algebra property. (2)The 2-categories of pseudo-algebras of S and T are equivalent. Thus, coherent structures (pseudo-T-algebras) are transformed into universally characterised ones (adjoint-pseudo-S-algebras). The 2-category \twocat{L} consists of lax algebras for the pseudo-monad induced by T on the bicategory of bimodules of \twocat{K}. We give an intrinsic characterisation of pseudo-S-algebras in terms of representability. Two major consequences of the above transformation are the classifications of lax and strong morphisms, with the attendant coherence result for pseudo-algebras. We apply the theory in the context of internal categories and examine monoidal and monoidal globular categories (including their monoid classifiers) as well as pseudo-functors into \Cat.Comment: to appear in Journal of Pure and Applied Algebr

A skew approach to enrichment for Gray-categories

It is well known that the category of Gray-categories does not admit a monoidal biclosed structure that models weak higher-dimensional transformations. In this paper, the first of a series on the topic, we describe several skew monoidal closed structures on the category of Gray-categories, one of which captures higher lax transformations, and another which models higher pseudo-transformations.Comment: Minor updates. Submitted for publicatio

Multicategory Large-Margin Unified Machines

Hard and soft classifiers are two important groups of techniques for classification problems. Logistic regression and Support Vector Machines are typical examples of soft and hard classifiers respectively. The essential difference between these two groups is whether one needs to estimate the class conditional probability for the classification task or not. In particular, soft classifiers predict the label based on the obtained class conditional probabilities, while hard classifiers bypass the estimation of probabilities and focus on the decision boundary. In practice, for the goal of accurate classification, it is unclear which one to use in a given situation. To tackle this problem, the Large-margin Unified Machine (LUM) was recently proposed as a unified family to embrace both groups. The LUM family enables one to study the behavior change from soft to hard binary classifiers. For multicategory cases, however, the concept of soft and hard classification becomes less clear. In that case, class probability estimation becomes more involved as it requires estimation of a probability vector. In this paper, we propose a new Multicategory LUM (MLUM) framework to investigate the behavior of soft versus hard classification under multicategory settings. Our theoretical and numerical results help to shed some light on the nature of multicategory classification and its transition behavior from soft to hard classifiers. The numerical results suggest that the proposed tuned MLUM yields very competitive performance

Flexible Classification Techniques with Biomedical Applications

Classification problems are prevalent in many scientific disciplines, especially in biomedical research. Recently, margin based classifiers have become increasingly popular, partly due to their ability in handling large scale problems with fast computational speed and desirable theoretical properties. Despite the success of margin based classifiers, many challenges remain. For example, in practical problems, it can be desirable to estimate the class conditional probability accurately. For high dimensional classification data, penalized margin based classifiers are commonly used. However, when estimating the class conditional probability, the shrinkage effect from the penalty term in the corresponding optimization is often ignored. This effect can lead to large bias in estimation of the class conditional probability. Another important issue on classification is the comparison between soft and hard classifiers for multicategory problems. Moreover, regular multicategory margin based classifiers can suffer from inefficiency by using too many classification functions. In this dissertation, we propose several new classification techniques to overcome the challenges mentioned above. Comprehensive numerical and theoretical studies are presented to demonstrate the usefulness of our new proposed methodologies.Doctor of Philosoph

ODN: Opening the Deep Network for Open-set Action Recognition

In recent years, the performance of action recognition has been significantly improved with the help of deep neural networks. Most of the existing action recognition works hold the \textit{closed-set} assumption that all action categories are known beforehand while deep networks can be well trained for these categories. However, action recognition in the real world is essentially an \textit{open-set} problem, namely, it is impossible to know all action categories beforehand and consequently infeasible to prepare sufficient training samples for those emerging categories. In this case, applying closed-set recognition methods will definitely lead to unseen-category errors. To address this challenge, we propose the Open Deep Network (ODN) for the open-set action recognition task. Technologically, ODN detects new categories by applying a multi-class triplet thresholding method, and then dynamically reconstructs the classification layer and "opens" the deep network by adding predictors for new categories continually. In order to transfer the learned knowledge to the new category, two novel methods, Emphasis Initialization and Allometry Training, are adopted to initialize and incrementally train the new predictor so that only few samples are needed to fine-tune the model. Extensive experiments show that ODN can effectively detect and recognize new categories with little human intervention, thus applicable to the open-set action recognition tasks in the real world. Moreover, ODN can even achieve comparable performance to some closed-set methods.Comment: 6 pages, 3 figures, ICME 201