9,915 research outputs found
Transferring structural knowledge across cognitive maps in humans and models
Relations between task elements often follow hidden underlying structural forms such as periodicities or hierarchies, whose inferences fosters performance. However, transferring structural knowledge to novel environments requires flexible representations that are generalizable over particularities of the current environment, such as its stimuli and size. We suggest that humans represent structural forms as abstract basis sets and that in novel tasks, the structural form is inferred and the relevant basis set is transferred. Using a computational model, we show that such representation allows inference of the underlying structural form, important task states, effective behavioural policies and the existence of unobserved state-trajectories. In two experiments, participants learned three abstract graphs during two successive days. We tested how structural knowledge acquired on Day-1 affected Day-2 performance. In line with our model, participants who had a correct structural prior were able to infer the existence of unobserved state-trajectories and appropriate behavioural policies
DAC: The Double Actor-Critic Architecture for Learning Options
We reformulate the option framework as two parallel augmented MDPs. Under
this novel formulation, all policy optimization algorithms can be used off the
shelf to learn intra-option policies, option termination conditions, and a
master policy over options. We apply an actor-critic algorithm on each
augmented MDP, yielding the Double Actor-Critic (DAC) architecture.
Furthermore, we show that, when state-value functions are used as critics, one
critic can be expressed in terms of the other, and hence only one critic is
necessary. We conduct an empirical study on challenging robot simulation tasks.
In a transfer learning setting, DAC outperforms both its hierarchy-free
counterpart and previous gradient-based option learning algorithms.Comment: NeurIPS 201
A Biologically-Inspired Dual Stream World Model
The medial temporal lobe (MTL), a brain region containing the hippocampus and
nearby areas, is hypothesized to be an experience-construction system in
mammals, supporting both recall and imagination of temporally-extended
sequences of events. Such capabilities are also core to many recently proposed
``world models" in the field of AI research. Taking inspiration from this
connection, we propose a novel variant, the Dual Stream World Model (DSWM),
which learns from high-dimensional observations and dissociates them into
context and content streams. DSWM can reliably generate imagined trajectories
in novel 2D environments after only a single exposure, outperforming a standard
world model. DSWM also learns latent representations which bear a strong
resemblance to place cells found in the hippocampus. We show that this
representation is useful as a reinforcement learning basis function, and that
the generative model can be used to aid the policy learning process using
Dyna-like updates
- …