Motor Planning, Not Execution, Separates Motor Memories

Recent theories of limb control emphasize motor cortex as a dynamical system, with planning setting the initial neural state, and execution arising from the self-limiting evolution of the intrinsic neural dynamics. Therefore, movements that share an initial trajectory but then diverge might have different neural states during the execution of the identical initial trajectories. We hypothesized that motor adaptation maps neural states to changes in motor command. This predicts that two opposing perturbations, which interfere when experienced over the same movement, could be learned if each is associated with a different plan even if not executed. We show that planning, but not executing, different follow-through movements allow opposing perturbations to be learned simultaneously over the same movement. However, no learning occurs if different follow throughs are executed, but not planned prior to movement initiation. Our results suggest neural, rather than physical states, are the critical factor associated with motor adaptation.We thank the Wellcome Trust, Royal Society (Noreen Murray Professorship in Neurobiology to D.M.W.), the Cambridge Commonwealth, European and International Trusts and the Rutherford Foundation Trust

The visual geometry of a tool modulates generalization during adaptation.

Knowledge about a tool's dynamics can be acquired from the visual configuration of the tool and through physical interaction. Here, we examine how visual information affects the generalization of dynamic learning during tool use. Subjects rotated a virtual hammer-like object while we varied the object dynamics separately for two rotational directions. This allowed us to quantify the coupling of adaptation between the directions, that is, how adaptation transferred from one direction to the other. Two groups experienced the same dynamics of the object. For one group, the object's visual configuration was displayed, while for the other, the visual display was uninformative as to the dynamics. We fit a range of context-dependent state-space models to the data, comparing different forms of coupling. We found that when the object's visual configuration was explicitly provided, there was substantial coupling, such that 31% of learning in one direction transferred to the other. In contrast, when the visual configuration was ambiguous, despite experiencing the same dynamics, the coupling was reduced to 12%. Our results suggest that generalization of dynamic learning of a tool relies, not only on its dynamic behaviour, but also on the visual configuration with which the dynamics is associated

Multiple decisions about one object involve parallel sensory acquisition but time-multiplexed evidence incorporation.

The brain is capable of processing several streams of information that bear on different aspects of the same problem. Here, we address the problem of making two decisions about one object, by studying difficult perceptual decisions about the color and motion of a dynamic random dot display. We find that the accuracy of one decision is unaffected by the difficulty of the other decision. However, the response times reveal that the two decisions do not form simultaneously. We show that both stimulus dimensions are acquired in parallel for the initial ∼0.1 s but are then incorporated serially in time-multiplexed bouts. Thus, there is a bottleneck that precludes updating more than one decision at a time, and a buffer that stores samples of evidence while access to the decision is blocked. We suggest that this bottleneck is responsible for the long timescales of many cognitive operations framed as decisions

Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain

How Plastic Can Phenotypic Plasticity Be? The Branching Coral Stylophora pistillata as a Model System

Phenotypic plasticity enables multicellular organisms to adjust morphologies and various life history traits to variable environmental challenges. Here, we elucidate fixed and plastic architectural rules for colony astogeny in multiple types of colonial ramets, propagated by cutting from genets of the branching coral Stylophora pistillata from Eilat, the Red Sea. We examined 16 morphometric parameters on 136 one-year old S. pistillata colonies (of seven genotypes), originating from small fragments belonging, each, to one of three single-branch types (single tips, start-up, and advanced bifurcating tips) or to structural preparative manipulations (representing a single or two growth axes). Experiments were guided by the rationale that in colonial forms, complexity of evolving phenotypic plasticity can be associated with a degree of structural modularity, where shapes are approached by erecting iterative growth patterns at different levels of coral-colony organization. Analyses revealed plastic morphometric characters at branch level, and predetermined morphometric traits at colony level (only single trait exhibited plasticity under extreme manipulation state). Therefore, under the experimental manipulations of this study, phenotypic plasticity in S. pistillata appears to be related to branch level of organization, whereas colony traits are controlled by predetermined genetic architectural rules. Each level of organization undergoes its own mode of astogeny. However, depending on the original ramet structure, the spherical 3-D colonial architecture in this species is orchestrated and assembled by both developmental trajectories at the branch level, and traits at the colony level of organization. In nature, branching colonial forms are often subjected to harsh environmental conditions that cause fragmentation of colony into ramets of different sizes and structures. Developmental traits that are plastic, responding to fragment structure and are not predetermine in controlling astogeny, allow formation of species-specific architecture product through integrated but variable developmental routes. This adaptive plasticity or regeneration is an efficient mechanism by which isolated fragments of branching coral species cope with external environmental forces

Interpretation of the sonic hedgehog morphogen gradient by a temporal adaptation mechanism

Morphogens act in developing tissues to control the spatial arrangement of cellular differentiation(1,2). The activity of a morphogen has generally been viewed as a concentration-dependent response to a diffusible signal, but the duration of morphogen signalling can also affect cellular responses(3). One such example is the morphogen sonic hedgehog (SHH). In the vertebrate central nervous system and limbs, the pattern of cellular differentiation is controlled by both the amount and the time of SHH exposure(4-7). How these two parameters are interpreted at a cellular level has been unclear. Here we provide evidence that changing the concentration or duration of SHH has an equivalent effect on intracellular signalling. Chick neural cells convert different concentrations of SHH into time-limited periods of signal transduction, such that signal duration is proportional to SHH concentration. This depends on the gradual desensitization of cells to ongoing SHH exposure, mediated by the SHH-dependent upregulation of patched 1 (PTC1), a ligand-binding inhibitor of SHH signalling(8). Thus, in addition to its role in shaping the SHH gradient(8-10), PTC1 participates cell autonomously in gradient sensing. Together, the data reveal a novel strategy for morphogen interpretation, in which the temporal adaptation of cells to a morphogen integrates the concentration and duration of a signal to control differential gene expression.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/62511/1/nature06347.pd

Time Scale Hierarchies in the Functional Organization of Complex Behaviors

Traditional approaches to cognitive modelling generally portray cognitive events in terms of ‘discrete’ states (point attractor dynamics) rather than in terms of processes, thereby neglecting the time structure of cognition. In contrast, more recent approaches explicitly address this temporal dimension, but typically provide no entry points into cognitive categorization of events and experiences. With the aim to incorporate both these aspects, we propose a framework for functional architectures. Our approach is grounded in the notion that arbitrary complex (human) behaviour is decomposable into functional modes (elementary units), which we conceptualize as low-dimensional dynamical objects (structured flows on manifolds). The ensemble of modes at an agent’s disposal constitutes his/her functional repertoire. The modes may be subjected to additional dynamics (termed operational signals), in particular, instantaneous inputs, and a mechanism that sequentially selects a mode so that it temporarily dominates the functional dynamics. The inputs and selection mechanisms act on faster and slower time scales then that inherent to the modes, respectively. The dynamics across the three time scales are coupled via feedback, rendering the entire architecture autonomous. We illustrate the functional architecture in the context of serial behaviour, namely cursive handwriting. Subsequently, we investigate the possibility of recovering the contributions of functional modes and operational signals from the output, which appears to be possible only when examining the output phase flow (i.e., not from trajectories in phase space or time)

ART: A machine learning Automated Recommendation Tool for synthetic biology

Biology has changed radically in the last two decades, transitioning from a descriptive science into a design science. Synthetic biology allows us to bioengineer cells to synthesize novel valuable molecules such as renewable biofuels or anticancer drugs. However, traditional synthetic biology approaches involve ad-hoc engineering practices, which lead to long development times. Here, we present the Automated Recommendation Tool (ART), a tool that leverages machine learning and probabilistic modeling techniques to guide synthetic biology in a systematic fashion, without the need for a full mechanistic understanding of the biological system. Using sampling-based optimization, ART provides a set of recommended strains to be built in the next engineering cycle, alongside probabilistic predictions of their production levels. We demonstrate the capabilities of ART on simulated data sets, as well as experimental data from real metabolic engineering projects producing renewable biofuels, hoppy flavored beer without hops, and fatty acids. Finally, we discuss the limitations of this approach, and the practical consequences of the underlying assumptions failing

Learning the Optimal Control of Coordinated Eye and Head Movements

Various optimality principles have been proposed to explain the characteristics of coordinated eye and head movements during visual orienting behavior. At the same time, researchers have suggested several neural models to underly the generation of saccades, but these do not include online learning as a mechanism of optimization. Here, we suggest an open-loop neural controller with a local adaptation mechanism that minimizes a proposed cost function. Simulations show that the characteristics of coordinated eye and head movements generated by this model match the experimental data in many aspects, including the relationship between amplitude, duration and peak velocity in head-restrained and the relative contribution of eye and head to the total gaze shift in head-free conditions. Our model is a first step towards bringing together an optimality principle and an incremental local learning mechanism into a unified control scheme for coordinated eye and head movements

Temporal regularity of the environment drives time perception

It’s reasonable to assume that a regularly paced sequence should be perceived as regular, but here we show that perceived regularity depends on the context in which the sequence is embedded. We presented one group of participants with perceptually regularly paced sequences, and another group of participants with mostly irregularly paced sequences (75% irregular, 25% regular). The timing of the final stimulus in each sequence could be varied. In one experiment, we asked whether the last stimulus was regular or not. We found that participants exposed to an irregular environment frequently reported perfectly regularly paced stimuli to be irregular. In a second experiment, we asked participants to judge whether the final stimulus was presented before or after a flash. In this way, we were able to determine distortions in temporal perception as changes in the timing necessary for the sound and the flash to be perceived synchronous. We found that within a regular context, the perceived timing of deviant last stimuli changed so that the relative anisochrony appeared to be perceptually decreased. In the irregular context, the perceived timing of irregular stimuli following a regular sequence was not affected. These observations suggest that humans use temporal expectations to evaluate the regularity of sequences and that expectations are combined with sensory stimuli to adapt perceived timing to follow the statistics of the environment. Expectations can be seen as a-priori probabilities on which perceived timing of stimuli depend