4,374 research outputs found
Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation
The control of nonlinear dynamical systems remains a major challenge for
autonomous agents. Current trends in reinforcement learning (RL) focus on
complex representations of dynamics and policies, which have yielded impressive
results in solving a variety of hard control tasks. However, this new
sophistication and extremely over-parameterized models have come with the cost
of an overall reduction in our ability to interpret the resulting policies. In
this paper, we take inspiration from the control community and apply the
principles of hybrid switching systems in order to break down complex dynamics
into simpler components. We exploit the rich representational power of
probabilistic graphical models and derive an expectation-maximization (EM)
algorithm for learning a sequence model to capture the temporal structure of
the data and automatically decompose nonlinear dynamics into stochastic
switching linear dynamical systems. Moreover, we show how this framework of
switching models enables extracting hierarchies of Markovian and
auto-regressive locally linear controllers from nonlinear experts in an
imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro
Differential Dynamic Programming for time-delayed systems
Trajectory optimization considers the problem of deciding how to control a
dynamical system to move along a trajectory which minimizes some cost function.
Differential Dynamic Programming (DDP) is an optimal control method which
utilizes a second-order approximation of the problem to find the control. It is
fast enough to allow real-time control and has been shown to work well for
trajectory optimization in robotic systems. Here we extend classic DDP to
systems with multiple time-delays in the state. Being able to find optimal
trajectories for time-delayed systems with DDP opens up the possibility to use
richer models for system identification and control, including recurrent neural
networks with multiple timesteps in the state. We demonstrate the algorithm on
a two-tank continuous stirred tank reactor. We also demonstrate the algorithm
on a recurrent neural network trained to model an inverted pendulum with
position information only.Comment: 7 pages, 6 figures, conference, Decision and Control (CDC), 2016 IEEE
55th Conference o
Closed-form continuous-time neural networks
Continuous-time neural networks are a class of machine learning systems that can tackle representation learning on spatiotemporal decision-making tasks. These models are typically represented by continuous differential equations. However, their expressive power when they are deployed on computers is bottlenecked by numerical differential equation solvers. This limitation has notably slowed down the scaling and understanding of numerous natural physical phenomena such as the dynamics of nervous systems. Ideally, we would circumvent this bottleneck by solving the given dynamical system in closed form. This is known to be intractable in general. Here, we show that it is possible to closely approximate the interaction between neurons and synapses—the building blocks of natural and artificial neural networks—constructed by liquid time-constant networks efficiently in closed form. To this end, we compute a tightly bounded approximation of the solution of an integral appearing in liquid time-constant dynamics that has had no known closed-form solution so far. This closed-form solution impacts the design of continuous-time and continuous-depth neural models. For instance, since time appears explicitly in closed form, the formulation relaxes the need for complex numerical solvers. Consequently, we obtain models that are between one and five orders of magnitude faster in training and inference compared with differential equation-based counterparts. More importantly, in contrast to ordinary differential equation-based continuous networks, closed-form networks can scale remarkably well compared with other deep learning instances. Lastly, as these models are derived from liquid networks, they show good performance in time-series modelling compared with advanced recurrent neural network models
Neutral theory and scale-free neural dynamics
Avalanches of electrochemical activity in brain networks have been
empirically reported to obey scale-invariant behavior --characterized by
power-law distributions up to some upper cut-off-- both in vitro and in vivo.
Elucidating whether such scaling laws stem from the underlying neural dynamics
operating at the edge of a phase transition is a fascinating possibility, as
systems poised at criticality have been argued to exhibit a number of important
functional advantages. Here we employ a well-known model for neural dynamics
with synaptic plasticity, to elucidate an alternative scenario in which
neuronal avalanches can coexist, overlapping in time, but still remaining
scale-free. Remarkably their scale-invariance does not stem from underlying
criticality nor self-organization at the edge of a continuous phase transition.
Instead, it emerges from the fact that perturbations to the system exhibit a
neutral drift --guided by demographic fluctuations-- with respect to endogenous
spontaneous activity. Such a neutral dynamics --similar to the one in neutral
theories of population genetics-- implies marginal propagation of activity,
characterized by power-law distributed causal avalanches. Importantly, our
results underline the importance of considering causal information --on which
neuron triggers the firing of which-- to properly estimate the statistics of
avalanches of neural activity. We discuss the implications of these findings
both in modeling and to elucidate experimental observations, as well as its
possible consequences for actual neural dynamics and information processing in
actual neural networks.Comment: Main text: 8 pages, 3 figures. Supplementary information: 5 pages, 4
figure
- …