11,193 research outputs found
A Tutorial on Estimating Time-Varying Vector Autoregressive Models
Time series of individual subjects have become a common data type in
psychological research. These data allow one to estimate models of
within-subject dynamics, and thereby avoid the notorious problem of making
within-subjects inferences from between-subjects data, and naturally address
heterogeneity between subjects. A popular model for these data is the Vector
Autoregressive (VAR) model, in which each variable is predicted as a linear
function of all variables at previous time points. A key assumption of this
model is that its parameters are constant (or stationary) across time. However,
in many areas of psychological research time-varying parameters are plausible
or even the subject of study. In this tutorial paper, we introduce methods to
estimate time-varying VAR models based on splines and kernel-smoothing
with/without regularization. We use simulations to evaluate the relative
performance of all methods in scenarios typical in applied research, and
discuss their strengths and weaknesses. Finally, we provide a step-by-step
tutorial showing how to apply the discussed methods to an openly available time
series of mood-related measurements
Hyperparameter optimization with approximate gradient
Most models in machine learning contain at least one hyperparameter to
control for model complexity. Choosing an appropriate set of hyperparameters is
both crucial in terms of model accuracy and computationally challenging. In
this work we propose an algorithm for the optimization of continuous
hyperparameters using inexact gradient information. An advantage of this method
is that hyperparameters can be updated before model parameters have fully
converged. We also give sufficient conditions for the global convergence of
this method, based on regularity conditions of the involved functions and
summability of errors. Finally, we validate the empirical performance of this
method on the estimation of regularization constants of L2-regularized logistic
regression and kernel Ridge regression. Empirical benchmarks indicate that our
approach is highly competitive with respect to state of the art methods.Comment: Proceedings of the International conference on Machine Learning
(ICML
Feature Augmentation via Nonparametrics and Selection (FANS) in High Dimensional Classification
We propose a high dimensional classification method that involves
nonparametric feature augmentation. Knowing that marginal density ratios are
the most powerful univariate classifiers, we use the ratio estimates to
transform the original feature measurements. Subsequently, penalized logistic
regression is invoked, taking as input the newly transformed or augmented
features. This procedure trains models equipped with local complexity and
global simplicity, thereby avoiding the curse of dimensionality while creating
a flexible nonlinear decision boundary. The resulting method is called Feature
Augmentation via Nonparametrics and Selection (FANS). We motivate FANS by
generalizing the Naive Bayes model, writing the log ratio of joint densities as
a linear combination of those of marginal densities. It is related to
generalized additive models, but has better interpretability and computability.
Risk bounds are developed for FANS. In numerical analysis, FANS is compared
with competing methods, so as to provide a guideline on its best application
domain. Real data analysis demonstrates that FANS performs very competitively
on benchmark email spam and gene expression data sets. Moreover, FANS is
implemented by an extremely fast algorithm through parallel computing.Comment: 30 pages, 2 figure
- …