3,670 research outputs found
Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation
The control of nonlinear dynamical systems remains a major challenge for
autonomous agents. Current trends in reinforcement learning (RL) focus on
complex representations of dynamics and policies, which have yielded impressive
results in solving a variety of hard control tasks. However, this new
sophistication and extremely over-parameterized models have come with the cost
of an overall reduction in our ability to interpret the resulting policies. In
this paper, we take inspiration from the control community and apply the
principles of hybrid switching systems in order to break down complex dynamics
into simpler components. We exploit the rich representational power of
probabilistic graphical models and derive an expectation-maximization (EM)
algorithm for learning a sequence model to capture the temporal structure of
the data and automatically decompose nonlinear dynamics into stochastic
switching linear dynamical systems. Moreover, we show how this framework of
switching models enables extracting hierarchies of Markovian and
auto-regressive locally linear controllers from nonlinear experts in an
imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro
Sustaining Economic Exploitation of Complex Ecosystems in Computational Models of Coupled Human-Natural Networks
Understanding ecological complexity has stymied scientists for decades. Recent elucidation of the famously coined "devious strategies for stability in enduring natural systems" has opened up a new field of computational analyses of complex ecological networks where the nonlinear dynamics of many interacting species can be more realistically mod-eled and understood. Here, we describe the first extension of this field to include coupled human-natural systems. This extension elucidates new strategies for sustaining extraction of biomass (e.g., fish, forests, fiber) from ecosystems that account for ecological complexity and can pursue multiple goals such as maximizing economic profit, employment and carbon sequestration by ecosystems. Our more realistic modeling of ecosystems helps explain why simpler "maxi-mum sustainable yield" bioeconomic models underpinning much natural resource extraction policy leads to less profit, biomass, and biodiversity than predicted by those simple models. Current research directions of this integrated natu-ral and social science include applying artificial intelligence, cloud computing, and multiplayer online games
- …