1,588 research outputs found
Structure Learning in Motor Control:A Deep Reinforcement Learning Model
Motor adaptation displays a structure-learning effect: adaptation to a new
perturbation occurs more quickly when the subject has prior exposure to
perturbations with related structure. Although this `learning-to-learn' effect
is well documented, its underlying computational mechanisms are poorly
understood. We present a new model of motor structure learning, approaching it
from the point of view of deep reinforcement learning. Previous work outside of
motor control has shown how recurrent neural networks can account for
learning-to-learn effects. We leverage this insight to address motor learning,
by importing it into the setting of model-based reinforcement learning. We
apply the resulting processing architecture to empirical findings from a
landmark study of structure learning in target-directed reaching (Braun et al.,
2009), and discuss its implications for a wider range of learning-to-learn
phenomena.Comment: 39th Annual Meeting of the Cognitive Science Society, to appea
The relationship of myocardial contraction and electrical excitation—the correlation between scintigraphic phase image analysis and electrophysiologic mapping
Phase imaging derived from equilibrium radionuclide angiography presents the ventricular contraction sequence. It has been widely but only indirectly correlated with the sequence of electrical myocardial activation.
We sought to determine the specific relationship between the sequence of phase progression and the sequence of myocardial activation, contraction and conduction, in order to document a noninvasive method that could monitor both.
In 7 normal and 9 infarcted dogs, the sequence of phase angle was correlated with the epicardial activation map in 126 episodes of sinus rhythm and pacing from three ventricular sites.
In each episode, the site of earliest phase angle was identical to the focus of initial epicardial activation. Similarly, the serial contraction pattern by phase image analysis matched the electrical epicardial activation sequence completely or demonstrated good agreement in approximately 85% of pacing episodes, without differences between normal or infarct groups.
A noninvasive method to accurately determine the sequence of contraction may serve as a surrogate for the associated electrical activation sequence or be applied to identify their differences
Recommended from our members
Neural correlates of cognitive dissonance and choice-induced preference change
According to many modern economic theories, actions simply reflect an individual's preferences, whereas a psychological phenomenon called “cognitive dissonance” claims that actions can also create preference. Cognitive dissonance theory states that after making a difficult choice between two equally preferred items, the act of rejecting a favorite item induces an uncomfortable feeling (cognitive dissonance), which in turn motivates individuals to change their preferences to match their prior decision (i.e., reducing preference for rejected items). Recently, however, Chen and Risen [Chen K, Risen J (2010) J Pers Soc Psychol 99:573–594] pointed out a serious methodological problem, which casts a doubt on the very existence of this choice-induced preference change as studied over the past 50 y. Here, using a proper control condition and two measures of preferences (self-report and brain activity), we found that the mere act of making a choice can change self-report preference as well as its neural representation (i.e., striatum activity), thus providing strong evidence for choice-induced preference change. Furthermore, our data indicate that the anterior cingulate cortex and dorsolateral prefrontal cortex tracked the degree of cognitive dissonance on a trial-by-trial basis. Our findings provide important insights into the neural basis of how actions can alter an individual's preferences
A Unified Theory of Dual-Process Control
Dual-process theories play a central role in both psychology and
neuroscience, figuring prominently in fields ranging from executive control to
reward-based learning to judgment and decision making. In each of these
domains, two mechanisms appear to operate concurrently, one relatively high in
computational complexity, the other relatively simple. Why is neural
information processing organized in this way? We propose an answer to this
question based on the notion of compression. The key insight is that
dual-process structure can enhance adaptive behavior by allowing an agent to
minimize the description length of its own behavior. We apply a single model
based on this observation to findings from research on executive control,
reward-based learning, and judgment and decision making, showing that seemingly
diverse dual-process phenomena can be understood as domain-specific
consequences of a single underlying set of computational principles
A deep active inference model of the rubber-hand illusion
Understanding how perception and action deal with sensorimotor conflicts,
such as the rubber-hand illusion (RHI), is essential to understand how the body
adapts to uncertain situations. Recent results in humans have shown that the
RHI not only produces a change in the perceived arm location, but also causes
involuntary forces. Here, we describe a deep active inference agent in a
virtual environment, which we subjected to the RHI, that is able to account for
these results. We show that our model, which deals with visual high-dimensional
inputs, produces similar perceptual and force patterns to those found in
humans.Comment: 8 pages, 3 figures, Accepted in 1st International Workshop on Active
Inference, in Conjunction with European Conference of Machine Learning 2020.
The final authenticated publication is available online at
https://doi.org/10.1007/978-3-030-64919-7_1
Minimum Description Length Control
We propose a novel framework for multitask reinforcement learning based on
the minimum description length (MDL) principle. In this approach, which we term
MDL-control (MDL-C), the agent learns the common structure among the tasks with
which it is faced and then distills it into a simpler representation which
facilitates faster convergence and generalization to new tasks. In doing so,
MDL-C naturally balances adaptation to each task with epistemic uncertainty
about the task distribution. We motivate MDL-C via formal connections between
the MDL principle and Bayesian inference, derive theoretical performance
guarantees, and demonstrate MDL-C's empirical effectiveness on both discrete
and high-dimensional continuous control tasks
Subgoal- and goal-related reward prediction errors in medial prefrontal cortex
A longstanding view of the organization of human and animal behavior holds that behavior is hierarchically organizedin other words, directed toward achieving superordinate goals through the achievement of subordinate goals or subgoals. However, most research in neuroscience has focused on tasks without hierarchical structure. In past work, we have shown that negative reward prediction error (RPE) signals in medial prefrontal cortex (mPFC) can be linked not only to superordinate goals but also to subgoals. This suggests that mPFC tracks impediments in the progression toward subgoals. Using fMRI of human participants engaged in a hierarchical navigation task, here we found that mPFC also processes positive prediction errors at the level of subgoals, indicating that this brain region is sensitive to advances in subgoal completion. However, when subgoal RPEs were elicited alongside with goal-related RPEs, mPFC responses reflected only the goal-related RPEs. These findings suggest that information from different levels of hierarchy is processed selectively, depending on the task context
- …