42 research outputs found
Auto-Encoding Sequential Monte Carlo
We build on auto-encoding sequential Monte Carlo (AESMC): a method for model
and proposal learning based on maximizing the lower bound to the log marginal
likelihood in a broad family of structured probabilistic models. Our approach
relies on the efficiency of sequential Monte Carlo (SMC) for performing
inference in structured probabilistic models and the flexibility of deep neural
networks to model complex conditional probability distributions. We develop
additional theoretical insights and introduce a new training procedure which
improves both model and proposal learning. We demonstrate that our approach
provides a fast, easy-to-implement and scalable means for simultaneous model
learning and proposal adaptation in deep generative models
On Nesting Monte Carlo Estimators
Many problems in machine learning and statistics involve nested expectations
and thus do not permit conventional Monte Carlo (MC) estimation. For such
problems, one must nest estimators, such that terms in an outer estimator
themselves involve calculation of a separate, nested, estimation. We
investigate the statistical implications of nesting MC estimators, including
cases of multiple levels of nesting, and establish the conditions under which
they converge. We derive corresponding rates of convergence and provide
empirical evidence that these rates are observed in practice. We further
establish a number of pitfalls that can arise from naive nesting of MC
estimators, provide guidelines about how these can be avoided, and lay out
novel methods for reformulating certain classes of nested expectation problems
into single expectations, leading to improved convergence rates. We demonstrate
the applicability of our work by using our results to develop a new estimator
for discrete Bayesian experimental design problems and derive error bounds for
a class of variational objectives.Comment: To appear at International Conference on Machine Learning 201
Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support
The posterior in probabilistic programs with stochastic support decomposes as
a weighted sum of the local posterior distributions associated with each
possible program path. We show that making predictions with this full posterior
implicitly performs a Bayesian model averaging (BMA) over paths. This is
potentially problematic, as model misspecification can cause the BMA weights to
prematurely collapse onto a single path, leading to sub-optimal predictions in
turn. To remedy this issue, we propose alternative mechanisms for path
weighting: one based on stacking and one based on ideas from PAC-Bayes. We show
how both can be implemented as a cheap post-processing step on top of existing
inference engines. In our experiments, we find them to be more robust and lead
to better predictions compared to the default BMA weights
In-Context Learning in Large Language Models Learns Label Relationships but Is Not Conventional Learning
The performance of Large Language Models (LLMs) on downstream tasks often
improves significantly when including examples of the input-label relationship
in the context. However, there is currently no consensus about how this
in-context learning (ICL) ability of LLMs works: for example, while Xie et al.
(2021) liken ICL to a general-purpose learning algorithm, Min et al. (2022b)
argue ICL does not even learn label relationships from in-context examples. In
this paper, we study (1) how labels of in-context examples affect predictions,
(2) how label relationships learned during pre-training interact with
input-label examples provided in-context, and (3) how ICL aggregates label
information across in-context examples. Our findings suggests LLMs usually
incorporate information from in-context labels, but that pre-training and
in-context label relationships are treated differently, and that the model does
not consider all in-context information equally. Our results give insights into
understanding and aligning LLM behavior
SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning
The recent progress in large language models (LLMs), especially the invention
of chain-of-thoughts (CoT) prompting, makes it possible to solve reasoning
problems. However, even the strongest LLMs are still struggling with more
complicated problems that require non-linear thinking and multi-step reasoning.
In this work, we explore whether LLMs have the ability to recognize their own
errors, without resorting to external resources. In particular, we investigate
whether they can be used to identify individual errors within a step-by-step
reasoning. To this end, we propose a zero-shot verification scheme to recognize
such errors. We then use this verification scheme to improve question-answering
performance, by using it to perform weighted voting on different generated
answers. We test the method on three math datasets-GSM8K, MathQA, and MATH-and
find that it successfully recognizes errors and, in turn, increases final
predictive performance