2,169 research outputs found
Variable selection for the multicategory SVM via adaptive sup-norm regularization
The Support Vector Machine (SVM) is a popular classification paradigm in
machine learning and has achieved great success in real applications. However,
the standard SVM can not select variables automatically and therefore its
solution typically utilizes all the input variables without discrimination.
This makes it difficult to identify important predictor variables, which is
often one of the primary goals in data analysis. In this paper, we propose two
novel types of regularization in the context of the multicategory SVM (MSVM)
for simultaneous classification and variable selection. The MSVM generally
requires estimation of multiple discriminating functions and applies the argmax
rule for prediction. For each individual variable, we propose to characterize
its importance by the supnorm of its coefficient vector associated with
different functions, and then minimize the MSVM hinge loss function subject to
a penalty on the sum of supnorms. To further improve the supnorm penalty, we
propose the adaptive regularization, which allows different weights imposed on
different variables according to their relative importance. Both types of
regularization automate variable selection in the process of building
classifiers, and lead to sparse multi-classifiers with enhanced
interpretability and improved accuracy, especially for high dimensional low
sample size data. One big advantage of the supnorm penalty is its easy
implementation via standard linear programming. Several simulated examples and
one real gene data analysis demonstrate the outstanding performance of the
adaptive supnorm penalty in various data settings.Comment: Published in at http://dx.doi.org/10.1214/08-EJS122 the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Diffusion is All You Need for Learning on Surfaces
We introduce a new approach to deep learning on 3D surfaces such as meshes or
point clouds. Our key insight is that a simple learned diffusion layer can
spatially share data in a principled manner, replacing operations like
convolution and pooling which are complicated and expensive on surfaces. The
only other ingredients in our network are a spatial gradient operation, which
uses dot-products of derivatives to encode tangent-invariant filters, and a
multi-layer perceptron applied independently at each point. The resulting
architecture, which we call DiffusionNet, is remarkably simple, efficient, and
scalable. Continuously optimizing for spatial support avoids the need to pick
neighborhood sizes or filter widths a priori, or worry about their impact on
network size/training time. Furthermore, the principled, geometric nature of
these networks makes them agnostic to the underlying representation and
insensitive to discretization. In practice, this means significant robustness
to mesh sampling, and even the ability to train on a mesh and evaluate on a
point cloud. Our experiments demonstrate that these networks achieve
state-of-the-art results for a variety of tasks on both meshes and point
clouds, including surface classification, segmentation, and non-rigid
correspondence
Theseus: A Library for Differentiable Nonlinear Optimization
We present Theseus, an efficient application-agnostic open source library for
differentiable nonlinear least squares (DNLS) optimization built on PyTorch,
providing a common framework for end-to-end structured learning in robotics and
vision. Existing DNLS implementations are application specific and do not
always incorporate many ingredients important for efficiency. Theseus is
application-agnostic, as we illustrate with several example applications that
are built using the same underlying differentiable components, such as
second-order optimizers, standard costs functions, and Lie groups. For
efficiency, Theseus incorporates support for sparse solvers, automatic
vectorization, batching, GPU acceleration, and gradient computation with
implicit differentiation and direct loss minimization. We do extensive
performance evaluation in a set of applications, demonstrating significant
efficiency gains and better scalability when these features are incorporated.
Project page: https://sites.google.com/view/theseus-a
Longitudinal tracking of physiological state with electromyographic signals.
Electrophysiological measurements have been used in recent history to classify instantaneous physiological configurations, e.g., hand gestures. This work investigates the feasibility of working with changes in physiological configurations over time (i.e., longitudinally) using a variety of algorithms from the machine learning domain. We demonstrate a high degree of classification accuracy for a binary classification problem derived from electromyography measurements before and after a 35-day bedrest. The problem difficulty is increased with a more dynamic experiment testing for changes in astronaut sensorimotor performance by taking electromyography and force plate measurements before, during, and after a jump from a small platform. A LASSO regularization is performed to observe changes in relationship between electromyography features and force plate outcomes. SVM classifiers are employed to correctly identify the times at which these experiments are performed, which is important as these indicate a trajectory of adaptation
- …