63,708 research outputs found
Fast Cross-Validation via Sequential Testing
With the increasing size of today's data sets, finding the right parameter
configuration in model selection via cross-validation can be an extremely
time-consuming task. In this paper we propose an improved cross-validation
procedure which uses nonparametric testing coupled with sequential analysis to
determine the best parameter set on linearly increasing subsets of the data. By
eliminating underperforming candidates quickly and keeping promising candidates
as long as possible, the method speeds up the computation while preserving the
capability of the full cross-validation. Theoretical considerations underline
the statistical power of our procedure. The experimental evaluation shows that
our method reduces the computation time by a factor of up to 120 compared to a
full cross-validation with a negligible impact on the accuracy
Inverse Uncertainty Quantification using the Modular Bayesian Approach based on Gaussian Process, Part 2: Application to TRACE
Inverse Uncertainty Quantification (UQ) is a process to quantify the
uncertainties in random input parameters while achieving consistency between
code simulations and physical observations. In this paper, we performed inverse
UQ using an improved modular Bayesian approach based on Gaussian Process (GP)
for TRACE physical model parameters using the BWR Full-size Fine-Mesh Bundle
Tests (BFBT) benchmark steady-state void fraction data. The model discrepancy
is described with a GP emulator. Numerical tests have demonstrated that such
treatment of model discrepancy can avoid over-fitting. Furthermore, we
constructed a fast-running and accurate GP emulator to replace TRACE full model
during Markov Chain Monte Carlo (MCMC) sampling. The computational cost was
demonstrated to be reduced by several orders of magnitude.
A sequential approach was also developed for efficient test source allocation
(TSA) for inverse UQ and validation. This sequential TSA methodology first
selects experimental tests for validation that has a full coverage of the test
domain to avoid extrapolation of model discrepancy term when evaluated at input
setting of tests for inverse UQ. Then it selects tests that tend to reside in
the unfilled zones of the test domain for inverse UQ, so that one can extract
the most information for posterior probability distributions of calibration
parameters using only a relatively small number of tests. This research
addresses the "lack of input uncertainty information" issue for TRACE physical
input parameters, which was usually ignored or described using expert opinion
or user self-assessment in previous work. The resulting posterior probability
distributions of TRACE parameters can be used in future uncertainty,
sensitivity and validation studies of TRACE code for nuclear reactor system
design and safety analysis
Adaptive Low-Complexity Sequential Inference for Dirichlet Process Mixture Models
We develop a sequential low-complexity inference procedure for Dirichlet
process mixtures of Gaussians for online clustering and parameter estimation
when the number of clusters are unknown a-priori. We present an easily
computable, closed form parametric expression for the conditional likelihood,
in which hyperparameters are recursively updated as a function of the streaming
data assuming conjugate priors. Motivated by large-sample asymptotics, we
propose a novel adaptive low-complexity design for the Dirichlet process
concentration parameter and show that the number of classes grow at most at a
logarithmic rate. We further prove that in the large-sample limit, the
conditional likelihood and data predictive distribution become asymptotically
Gaussian. We demonstrate through experiments on synthetic and real data sets
that our approach is superior to other online state-of-the-art methods.Comment: 25 pages, To appear in Advances in Neural Information Processing
Systems (NIPS) 201
Regularized Ordinal Regression and the ordinalNet R Package
Regularization techniques such as the lasso (Tibshirani 1996) and elastic net
(Zou and Hastie 2005) can be used to improve regression model coefficient
estimation and prediction accuracy, as well as to perform variable selection.
Ordinal regression models are widely used in applications where the use of
regularization could be beneficial; however, these models are not included in
many popular software packages for regularized regression. We propose a
coordinate descent algorithm to fit a broad class of ordinal regression models
with an elastic net penalty. Furthermore, we demonstrate that each model in
this class generalizes to a more flexible form, for instance to accommodate
unordered categorical data. We introduce an elastic net penalty class that
applies to both model forms. Additionally, this penalty can be used to shrink a
non-ordinal model toward its ordinal counterpart. Finally, we introduce the R
package ordinalNet, which implements the algorithm for this model class
An ADMM Based Framework for AutoML Pipeline Configuration
We study the AutoML problem of automatically configuring machine learning
pipelines by jointly selecting algorithms and their appropriate
hyper-parameters for all steps in supervised learning pipelines. This black-box
(gradient-free) optimization with mixed integer & continuous variables is a
challenging problem. We propose a novel AutoML scheme by leveraging the
alternating direction method of multipliers (ADMM). The proposed framework is
able to (i) decompose the optimization problem into easier sub-problems that
have a reduced number of variables and circumvent the challenge of mixed
variable categories, and (ii) incorporate black-box constraints along-side the
black-box optimization objective. We empirically evaluate the flexibility (in
utilizing existing AutoML techniques), effectiveness (against open source
AutoML toolkits),and unique capability (of executing AutoML with practically
motivated black-box constraints) of our proposed scheme on a collection of
binary classification data sets from UCI ML& OpenML repositories. We observe
that on an average our framework provides significant gains in comparison to
other AutoML frameworks (Auto-sklearn & TPOT), highlighting the practical
advantages of this framework
- …