203 research outputs found
Efficient unimodality test in clustering by signature testing
This paper provides a new unimodality test with application in hierarchical
clustering methods. The proposed method denoted by signature test (Sigtest),
transforms the data based on its statistics. The transformed data has much
smaller variation compared to the original data and can be evaluated in a
simple proposed unimodality test. Compared with the existing unimodality tests,
Sigtest is more accurate in detecting the overlapped clusters and has a much
less computational complexity. Simulation results demonstrate the efficiency of
this statistic test for both real and synthetic data sets
Nonlinear Models Using Dirichlet Process Mixtures
We introduce a new nonlinear model for classification, in which we model the
joint distribution of response variable, y, and covariates, x,
non-parametrically using Dirichlet process mixtures. We keep the relationship
between y and x linear within each component of the mixture. The overall
relationship becomes nonlinear if the mixture contains more than one component.
We use simulated data to compare the performance of this new approach to a
simple multinomial logit (MNL) model, an MNL model with quadratic terms, and a
decision tree model. We also evaluate our approach on a protein fold
classification problem, and find that our model provides substantial
improvement over previous methods, which were based on Neural Networks (NN) and
Support Vector Machines (SVM). Folding classes of protein have a hierarchical
structure. We extend our method to classification problems where a class
hierarchy is available. We find that using the prior information regarding the
hierarchical structure of protein folds can result in higher predictive
accuracy
Improving Classification When a Class Hierarchy is Available Using a Hierarchy-Based Prior
We introduce a new method for building classification models when we have
prior knowledge of how the classes can be arranged in a hierarchy, based on how
easily they can be distinguished. The new method uses a Bayesian form of the
multinomial logit (MNL, a.k.a. ``softmax'') model, with a prior that introduces
correlations between the parameters for classes that are nearby in the tree. We
compare the performance on simulated data of the new method, the ordinary MNL
model, and a model that uses the hierarchy in different way. We also test the
new method on a document labelling problem, and find that it performs better
than the other methods, particularly when the amount of training data is small
Modeling Binary Time Series Using Gaussian Processes with Application to Predicting Sleep States
Motivated by the problem of predicting sleep states, we develop a mixed
effects model for binary time series with a stochastic component represented by
a Gaussian process. The fixed component captures the effects of covariates on
the binary-valued response. The Gaussian process captures the residual
variations in the binary response that are not explained by covariates and past
realizations. We develop a frequentist modeling framework that provides
efficient inference and more accurate predictions. Results demonstrate the
advantages of improved prediction rates over existing approaches such as
logistic regression, generalized additive mixed model, models for ordinal data,
gradient boosting, decision tree and random forest. Using our proposed model,
we show that previous sleep state and heart rates are significant predictors
for future sleep states. Simulation studies also show that our proposed method
is promising and robust. To handle computational complexity, we utilize Laplace
approximation, golden section search and successive parabolic interpolation.
With this paper, we also submit an R-package (HIBITS) that implements the
proposed procedure.Comment: Journal of Classification (2018
Wormhole Hamiltonian Monte Carlo
In machine learning and statistics, probabilistic inference involving
multimodal distributions is quite difficult. This is especially true in high
dimensional problems, where most existing algorithms cannot easily move from
one mode to another. To address this issue, we propose a novel Bayesian
inference approach based on Markov Chain Monte Carlo. Our method can
effectively sample from multimodal distributions, especially when the dimension
is high and the modes are isolated. To this end, it exploits and modifies the
Riemannian geometric properties of the target distribution to create
\emph{wormholes} connecting modes in order to facilitate moving between them.
Further, our proposed method uses the regeneration technique in order to adapt
the algorithm by identifying new modes and updating the network of wormholes
without affecting the stationary distribution. To find new modes, as opposed to
rediscovering those previously identified, we employ a novel mode searching
algorithm that explores a \emph{residual energy} function obtained by
subtracting an approximate Gaussian mixture density (based on previously
discovered modes) from the target density function
Spherical Hamiltonian Monte Carlo for Constrained Target Distributions
We propose a new Markov Chain Monte Carlo (MCMC) method for constrained
target distributions. Our method first maps the -dimensional constrained
domain of parameters to the unit ball . Then, it augments the
resulting parameter space to the -dimensional sphere, . The
boundary of corresponds to the equator of . This
change of domains enables us to implicitly handle the original constraints
because while the sampler moves freely on the sphere, it proposes states that
are within the constraints imposed on the original parameter space. To improve
the computational efficiency of our algorithm, we split the Lagrangian dynamics
into several parts such that a part of the dynamics can be handled analytically
by finding the geodesic flow on the sphere. We apply our method to several
examples including truncated Gaussian, Bayesian Lasso, Bayesian bridge
regression, and a copula model for identifying synchrony among multiple
neurons. Our results show that the proposed method can provide a natural and
efficient framework for handling several types of constraints on target
distributions
Variational Hamiltonian Monte Carlo via Score Matching
Traditionally, the field of computational Bayesian statistics has been
divided into two main subfields: variational methods and Markov chain Monte
Carlo (MCMC). In recent years, however, several methods have been proposed
based on combining variational Bayesian inference and MCMC simulation in order
to improve their overall accuracy and computational efficiency. This marriage
of fast evaluation and flexible approximation provides a promising means of
designing scalable Bayesian inference methods. In this paper, we explore the
possibility of incorporating variational approximation into a state-of-the-art
MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required gradient
computation in the simulation of Hamiltonian flow, which is the bottleneck for
many applications of HMC in big data problems. To this end, we use a {\it
free-form} approximation induced by a fast and flexible surrogate function
based on single-hidden layer feedforward neural networks. The surrogate
provides sufficiently accurate approximation while allowing for fast
exploration of parameter space, resulting in an efficient approximate inference
algorithm. We demonstrate the advantages of our method on both synthetic and
real data problems
Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases
For big data analysis, high computational cost for Bayesian methods often
limits their applications in practice. In recent years, there have been many
attempts to improve computational efficiency of Bayesian inference. Here we
propose an efficient and scalable computational technique for a
state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian
Monte Carlo (HMC). The key idea is to explore and exploit the structure and
regularity in parameter space for the underlying probabilistic model to
construct an effective approximation of its geometric properties. To this end,
we build a surrogate function to approximate the target distribution using
properly chosen random bases and an efficient optimization process. The
resulting method provides a flexible, scalable, and efficient sampling
algorithm, which converges to the correct target distribution. We show that by
choosing the basis functions and optimization process differently, our method
can be related to other approaches for the construction of surrogate functions
such as generalized additive models or Gaussian process models. Experiments
based on simulated and real data show that our approach leads to substantially
more efficient sampling algorithms compared to existing state-of-the art
methods
- β¦