50 research outputs found
Hierarchical models of goal-directed and automatic actions
Decision-making processes behind instrumental actions can be divided into two categories: goal-directed actions, and automatic actions. The structure of automatic actions, their interaction with goal-directed actions, and their behavioral and computational properties are the topics of the current thesis. We conceptualize the structure of automatic actions as sequences of actions that form a single response unit and are integrated within goal-directed processes in a hierarchical manner. We represent this hypothesis using the computational framework of reinforcement learning and develop a new normative computational model for the acquisition of action sequences, and their hierarchical interaction with goal-directed processes. We develop a neurally plausible hypothesis for the role of neuromodulator dopamine as a teaching signal for the acquisition of action sequences. We further explore the predictions of the proposed model in a two-stage decision-making task in humans and we show that the proposed model has higher explanatory power than its alternatives. Finally, we translate the two-stage decision-making task to an experimental protocol in rats and show that, similar to humans, rats also use action sequences and engage in hierarchical decision-making. The results provide a new theoretical and experimental paradigm for conceptualizing and measuring the operation and interaction of goal-directed and automatic actions
Hierarchical models of goal-directed and automatic actions
Decision-making processes behind instrumental actions can be divided into two categories: goal-directed actions, and automatic actions. The structure of automatic actions, their interaction with goal-directed actions, and their behavioral and computational properties are the topics of the current thesis. We conceptualize the structure of automatic actions as sequences of actions that form a single response unit and are integrated within goal-directed processes in a hierarchical manner. We represent this hypothesis using the computational framework of reinforcement learning and develop a new normative computational model for the acquisition of action sequences, and their hierarchical interaction with goal-directed processes. We develop a neurally plausible hypothesis for the role of neuromodulator dopamine as a teaching signal for the acquisition of action sequences. We further explore the predictions of the proposed model in a two-stage decision-making task in humans and we show that the proposed model has higher explanatory power than its alternatives. Finally, we translate the two-stage decision-making task to an experimental protocol in rats and show that, similar to humans, rats also use action sequences and engage in hierarchical decision-making. The results provide a new theoretical and experimental paradigm for conceptualizing and measuring the operation and interaction of goal-directed and automatic actions
The contextual lasso: Sparse linear models via deep neural networks
Sparse linear models are a gold standard tool for interpretable machine
learning, a field of emerging importance as predictive models permeate
decision-making in many domains. Unfortunately, sparse linear models are far
less flexible as functions of their input features than black-box models like
deep neural networks. With this capability gap in mind, we study a not-uncommon
situation where the input features dichotomize into two groups: explanatory
features, which are candidates for inclusion as variables in an interpretable
model, and contextual features, which select from the candidate variables and
determine their effects. This dichotomy leads us to the contextual lasso, a new
statistical estimator that fits a sparse linear model to the explanatory
features such that the sparsity pattern and coefficients vary as a function of
the contextual features. The fitting process learns this function
nonparametrically via a deep neural network. To attain sparse coefficients, we
train the network with a novel lasso regularizer in the form of a projection
layer that maps the network's output onto the space of -constrained
linear models. An extensive suite of experiments on real and synthetic data
suggests that the learned models, which remain highly transparent, can be
sparser than the regular lasso without sacrificing the predictive power of a
standard deep neural network
Gray-box inference for structured Gaussian process models
We develop an automated variational infer- ence method for Bayesian structured prediction problems with Gaussian process (gp) priors and linear-chain likelihoods. Our approach does not need to know the details of the structured likelihood model and can scale up to a large number of observations. Furthermore, we show that the required expected likelihood term and its gradients in the variational objective (ELBO) can be estimated efficiently by using expectations over very low-dimensional Gaussian distributions. Optimization of the ELBO is fully parallelizable over sequences and amenable to stochastic optimization, which we use along with control variate techniques to make our framework useful in practice. Results on a set of natural language processing tasks show that our method can be as good as (and sometimes better than, in particular with respect to expected log-likelihood) hard-coded approaches including svm-struct and crfs, and overcomes the scalability limitations of previous inference algorithms based on sampling. Overall, this is a fundamental step to developing automated inference methods for Bayesian structured prediction
Cross-Entropy Estimators for Sequential Experiment Design with Reinforcement Learning
Reinforcement learning can effectively learn amortised design policies for
designing sequences of experiments. However, current methods rely on
contrastive estimators of expected information gain, which require an
exponential number of contrastive samples to achieve an unbiased estimation. We
propose an alternative lower bound estimator, based on the cross-entropy of the
joint model distribution and a flexible proposal distribution. This proposal
distribution approximates the true posterior of the model parameters given the
experimental history and the design policy. Our estimator requires no
contrastive samples, can achieve more accurate estimates of high information
gains, allows learning of superior design policies, and is compatible with
implicit probabilistic models. We assess our algorithm's performance in various
tasks, including continuous and discrete designs and explicit and implicit
likelihoods
Transformed Distribution Matching for Missing Value Imputation
We study the problem of imputing missing values in a dataset, which has
important applications in many domains. The key to missing value imputation is
to capture the data distribution with incomplete samples and impute the missing
values accordingly. In this paper, by leveraging the fact that any two batches
of data with missing values come from the same data distribution, we propose to
impute the missing values of two batches of samples by transforming them into a
latent space through deep invertible functions and matching them
distributionally. To learn the transformations and impute the missing values
simultaneously, a simple and well-motivated algorithm is proposed. Our
algorithm has fewer hyperparameters to fine-tune and generates high-quality
imputations regardless of how missing values are generated. Extensive
experiments over a large number of datasets and competing benchmark algorithms
show that our method achieves state-of-the-art performance.Comment: ICML 2023 camera-ready version,
https://openreview.net/forum?id=WBWb1FU8i