1,401 research outputs found
Model Selection for High Dimensional Quadratic Regression via Regularization
Quadratic regression (QR) models naturally extend linear models by
considering interaction effects between the covariates. To conduct model
selection in QR, it is important to maintain the hierarchical model structure
between main effects and interaction effects. Existing regularization methods
generally achieve this goal by solving complex optimization problems, which
usually demands high computational cost and hence are not feasible for high
dimensional data. This paper focuses on scalable regularization methods for
model selection in high dimensional QR. We first consider two-stage
regularization methods and establish theoretical properties of the two-stage
LASSO. Then, a new regularization method, called Regularization Algorithm under
Marginality Principle (RAMP), is proposed to compute a hierarchy-preserving
regularization solution path efficiently. Both methods are further extended to
solve generalized QR models. Numerical results are also shown to demonstrate
performance of the methods.Comment: 37 pages, 1 figure with supplementary materia
On the adaptive elastic-net with a diverging number of parameters
We consider the problem of model selection and estimation in situations where
the number of parameters diverges with the sample size. When the dimension is
high, an ideal method should have the oracle property [J. Amer. Statist. Assoc.
96 (2001) 1348--1360] and [Ann. Statist. 32 (2004) 928--961] which ensures the
optimal large sample performance. Furthermore, the high-dimensionality often
induces the collinearity problem, which should be properly handled by the ideal
method. Many existing variable selection methods fail to achieve both goals
simultaneously. In this paper, we propose the adaptive elastic-net that
combines the strengths of the quadratic regularization and the adaptively
weighted lasso shrinkage. Under weak regularity conditions, we establish the
oracle property of the adaptive elastic-net. We show by simulations that the
adaptive elastic-net deals with the collinearity problem better than the other
oracle-like methods, thus enjoying much improved finite sample performance.Comment: Published in at http://dx.doi.org/10.1214/08-AOS625 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Component selection and smoothing in multivariate nonparametric regression
We propose a new method for model selection and model fitting in multivariate
nonparametric regression models, in the framework of smoothing spline ANOVA.
The ``COSSO'' is a method of regularization with the penalty functional being
the sum of component norms, instead of the squared norm employed in the
traditional smoothing spline method. The COSSO provides a unified framework for
several recent proposals for model selection in linear models and smoothing
spline ANOVA models. Theoretical properties, such as the existence and the rate
of convergence of the COSSO estimator, are studied. In the special case of a
tensor product design with periodic functions, a detailed analysis reveals that
the COSSO does model selection by applying a novel soft thresholding type
operation to the function components. We give an equivalent formulation of the
COSSO estimator which leads naturally to an iterative algorithm. We compare the
COSSO with MARS, a popular method that builds functional ANOVA models, in
simulations and real examples. The COSSO method can be extended to
classification problems and we compare its performance with those of a number
of machine learning algorithms on real datasets. The COSSO gives very
competitive performance in these studies.Comment: Published at http://dx.doi.org/10.1214/009053606000000722 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Variable selection for the multicategory SVM via adaptive sup-norm regularization
The Support Vector Machine (SVM) is a popular classification paradigm in
machine learning and has achieved great success in real applications. However,
the standard SVM can not select variables automatically and therefore its
solution typically utilizes all the input variables without discrimination.
This makes it difficult to identify important predictor variables, which is
often one of the primary goals in data analysis. In this paper, we propose two
novel types of regularization in the context of the multicategory SVM (MSVM)
for simultaneous classification and variable selection. The MSVM generally
requires estimation of multiple discriminating functions and applies the argmax
rule for prediction. For each individual variable, we propose to characterize
its importance by the supnorm of its coefficient vector associated with
different functions, and then minimize the MSVM hinge loss function subject to
a penalty on the sum of supnorms. To further improve the supnorm penalty, we
propose the adaptive regularization, which allows different weights imposed on
different variables according to their relative importance. Both types of
regularization automate variable selection in the process of building
classifiers, and lead to sparse multi-classifiers with enhanced
interpretability and improved accuracy, especially for high dimensional low
sample size data. One big advantage of the supnorm penalty is its easy
implementation via standard linear programming. Several simulated examples and
one real gene data analysis demonstrate the outstanding performance of the
adaptive supnorm penalty in various data settings.Comment: Published in at http://dx.doi.org/10.1214/08-EJS122 the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Congruences for modular forms and applications to crank functions
In this paper, motivated by the work of Mahlburg, we find congruences for a
large class of modular forms. Moreover, we generalize the generating function
of the Andrews-Garvan-Dyson crank on partition and establish several new
infinite families of congruences. In this framework, we showed that both the
birank of an ordered pair of partitions introduced by Hammond and Lewis, and
-crank of -colored partition introduced by Fu and Tang process the same
as the partition function and crank
Robust Brain MRI Image Classification with SIBOW-SVM
The majority of primary Central Nervous System (CNS) tumors in the brain are
among the most aggressive diseases affecting humans. Early detection of brain
tumor types, whether benign or malignant, glial or non-glial, is critical for
cancer prevention and treatment, ultimately improving human life expectancy.
Magnetic Resonance Imaging (MRI) stands as the most effective technique to
detect brain tumors by generating comprehensive brain images through scans.
However, human examination can be error-prone and inefficient due to the
complexity, size, and location variability of brain tumors. Recently, automated
classification techniques using machine learning (ML) methods, such as
Convolutional Neural Network (CNN), have demonstrated significantly higher
accuracy than manual screening, while maintaining low computational costs.
Nonetheless, deep learning-based image classification methods, including CNN,
face challenges in estimating class probabilities without proper model
calibration. In this paper, we propose a novel brain tumor image classification
method, called SIBOW-SVM, which integrates the Bag-of-Features (BoF) model with
SIFT feature extraction and weighted Support Vector Machines (wSVMs). This new
approach effectively captures hidden image features, enabling the
differentiation of various tumor types and accurate label predictions.
Additionally, the SIBOW-SVM is able to estimate the probabilities of images
belonging to each class, thereby providing high-confidence classification
decisions. We have also developed scalable and parallelable algorithms to
facilitate the practical implementation of SIBOW-SVM for massive images. As a
benchmark, we apply the SIBOW-SVM to a public data set of brain tumor MRI
images containing four classes: glioma, meningioma, pituitary, and normal. Our
results show that the new method outperforms state-of-the-art methods,
including CNN
- …