94,931 research outputs found
Optimal Approximate Sampling from Discrete Probability Distributions
This paper addresses a fundamental problem in random variate generation:
given access to a random source that emits a stream of independent fair bits,
what is the most accurate and entropy-efficient algorithm for sampling from a
discrete probability distribution , where the probabilities
of the output distribution of the sampling
algorithm must be specified using at most bits of precision? We present a
theoretical framework for formulating this problem and provide new techniques
for finding sampling algorithms that are optimal both statistically (in the
sense of sampling accuracy) and information-theoretically (in the sense of
entropy consumption). We leverage these results to build a system that, for a
broad family of measures of statistical accuracy, delivers a sampling algorithm
whose expected entropy usage is minimal among those that induce the same
distribution (i.e., is "entropy-optimal") and whose output distribution
is a closest approximation to the target
distribution among all entropy-optimal sampling algorithms
that operate within the specified -bit precision. This optimal approximate
sampler is also a closer approximation than any (possibly entropy-suboptimal)
sampler that consumes a bounded amount of entropy with the specified precision,
a class which includes floating-point implementations of inversion sampling and
related methods found in many software libraries. We evaluate the accuracy,
entropy consumption, precision requirements, and wall-clock runtime of our
optimal approximate sampling algorithms on a broad set of distributions,
demonstrating the ways that they are superior to existing approximate samplers
and establishing that they often consume significantly fewer resources than are
needed by exact samplers
Estimation in hidden Markov models via efficient importance sampling
Given a sequence of observations from a discrete-time, finite-state hidden
Markov model, we would like to estimate the sampling distribution of a
statistic. The bootstrap method is employed to approximate the confidence
regions of a multi-dimensional parameter. We propose an importance sampling
formula for efficient simulation in this context. Our approach consists of
constructing a locally asymptotically normal (LAN) family of probability
distributions around the default resampling rule and then minimizing the
asymptotic variance within the LAN family. The solution of this minimization
problem characterizes the asymptotically optimal resampling scheme, which is
given by a tilting formula. The implementation of the tilting formula is
facilitated by solving a Poisson equation. A few numerical examples are given
to demonstrate the efficiency of the proposed importance sampling scheme.Comment: Published at http://dx.doi.org/10.3150/07--BEJ5163 in the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Perseus: Randomized Point-based Value Iteration for POMDPs
Partially observable Markov decision processes (POMDPs) form an attractive
and principled framework for agent planning under uncertainty. Point-based
approximate techniques for POMDPs compute a policy based on a finite set of
points collected in advance from the agents belief space. We present a
randomized point-based value iteration algorithm called Perseus. The algorithm
performs approximate value backup stages, ensuring that in each backup stage
the value of each point in the belief set is improved; the key observation is
that a single backup may improve the value of many belief points. Contrary to
other point-based methods, Perseus backs up only a (randomly selected) subset
of points in the belief set, sufficient for improving the value of each belief
point in the set. We show how the same idea can be extended to dealing with
continuous action spaces. Experimental results show the potential of Perseus in
large scale POMDP problems
Approximation Algorithms for Distributionally Robust Stochastic Optimization with Black-Box Distributions
Two-stage stochastic optimization is a framework for modeling uncertainty,
where we have a probability distribution over possible realizations of the
data, called scenarios, and decisions are taken in two stages: we make
first-stage decisions knowing only the underlying distribution and before a
scenario is realized, and may take additional second-stage recourse actions
after a scenario is realized. The goal is typically to minimize the total
expected cost. A criticism of this model is that the underlying probability
distribution is itself often imprecise! To address this, a versatile approach
that has been proposed is the {\em distributionally robust 2-stage model}:
given a collection of probability distributions, our goal now is to minimize
the maximum expected total cost with respect to a distribution in this
collection.
We provide a framework for designing approximation algorithms in such
settings when the collection is a ball around a central distribution and the
central distribution is accessed {\em only via a sampling black box}.
We first show that one can utilize the {\em sample average approximation}
(SAA) method to reduce the problem to the case where the central distribution
has {\em polynomial-size} support. We then show how to approximately solve a
fractional relaxation of the SAA (i.e., polynomial-scenario
central-distribution) problem. By complementing this via LP-rounding algorithms
that provide {\em local} (i.e., per-scenario) approximation guarantees, we
obtain the {\em first} approximation algorithms for the distributionally robust
versions of a variety of discrete-optimization problems including set cover,
vertex cover, edge cover, facility location, and Steiner tree, with guarantees
that are, except for set cover, within -factors of the guarantees known
for the deterministic version of the problem
Monte Carlo Bayesian Reinforcement Learning
Bayesian reinforcement learning (BRL) encodes prior knowledge of the world in
a model and represents uncertainty in model parameters by maintaining a
probability distribution over them. This paper presents Monte Carlo BRL
(MC-BRL), a simple and general approach to BRL. MC-BRL samples a priori a
finite set of hypotheses for the model parameter values and forms a discrete
partially observable Markov decision process (POMDP) whose state space is a
cross product of the state space for the reinforcement learning task and the
sampled model parameter space. The POMDP does not require conjugate
distributions for belief representation, as earlier works do, and can be solved
relatively easily with point-based approximation algorithms. MC-BRL naturally
handles both fully and partially observable worlds. Theoretical and
experimental results show that the discrete POMDP approximates the underlying
BRL task well with guaranteed performance.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
- …