24,491 research outputs found
Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior
In recent years a number of methods have been developed for automatically
learning the (sparse) connectivity structure of Markov Random Fields. These
methods are mostly based on L1-regularized optimization which has a number of
disadvantages such as the inability to assess model uncertainty and expensive
cross-validation to find the optimal regularization parameter. Moreover, the
model's predictive performance may degrade dramatically with a suboptimal value
of the regularization parameter (which is sometimes desirable to induce
sparseness). We propose a fully Bayesian approach based on a "spike and slab"
prior (similar to L0 regularization) that does not suffer from these
shortcomings. We develop an approximate MCMC method combining Langevin dynamics
and reversible jump MCMC to conduct inference in this model. Experiments show
that the proposed model learns a good combination of the structure and
parameter values without the need for separate hyper-parameter tuning.
Moreover, the model's predictive performance is much more robust than L1-based
methods with hyper-parameter settings that induce highly sparse model
structures.Comment: Accepted in the Conference on Uncertainty in Artificial Intelligence
(UAI), 201
Parallelizable sparse inverse formulation Gaussian processes (SpInGP)
We propose a parallelizable sparse inverse formulation Gaussian process
(SpInGP) for temporal models. It uses a sparse precision GP formulation and
sparse matrix routines to speed up the computations. Due to the state-space
formulation used in the algorithm, the time complexity of the basic SpInGP is
linear, and because all the computations are parallelizable, the parallel form
of the algorithm is sublinear in the number of data points. We provide example
algorithms to implement the sparse matrix routines and experimentally test the
method using both simulated and real data.Comment: Presented at Machine Learning in Signal Processing (MLSP2017
Targeted Undersmoothing
This paper proposes a post-model selection inference procedure, called
targeted undersmoothing, designed to construct uniformly valid confidence sets
for a broad class of functionals of sparse high-dimensional statistical models.
These include dense functionals, which may potentially depend on all elements
of an unknown high-dimensional parameter. The proposed confidence sets are
based on an initially selected model and two additionally selected models, an
upper model and a lower model, which enlarge the initially selected model. We
illustrate application of the procedure in two empirical examples. The first
example considers estimation of heterogeneous treatment effects using data from
the Job Training Partnership Act of 1982, and the second example looks at
estimating profitability from a mailing strategy based on estimated
heterogeneous treatment effects in a direct mail marketing campaign. We also
provide evidence on the finite sample performance of the proposed targeted
undersmoothing procedure through a series of simulation experiments
- …