Chaotic exploration and learning of locomotion behaviours

We present a general and fully dynamic neural system, which exploits intrinsic chaotic dynamics, for the real-time goal-directed exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modeled as a network of neural oscillators that are initially coupled only through physical embodiment, and goal-directed exploration of coordinated motor patterns is achieved by chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organized dynamics, each of which is a candidate for a locomotion behavior. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states, using its intrinsic chaotic dynamics as a driving force, and stabilizes on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced, which results in an increased diversity of motor outputs, thus achieving multiscale exploration. A rhythmic pattern discovered by this process is memorized and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronization method. Our results show that the novel neurorobotic system is able to create and learn multiple locomotion behaviors for a wide range of body configurations and physical environments and can readapt in realtime after sustaining damage

Extending Feynman's Formalisms for Modelling Human Joint Action Coordination

The recently developed Life-Space-Foam approach to goal-directed human action deals with individual actor dynamics. This paper applies the model to characterize the dynamics of co-action by two or more actors. This dynamics is modelled by: (i) a two-term joint action (including cognitive/motivatonal potential and kinetic energy), and (ii) its associated adaptive path integral, representing an infinite--dimensional neural network. Its feedback adaptation loop has been derived from Bernstein's concepts of sensory corrections loop in human motor control and Brooks' subsumption architectures in robotics. Potential applications of the proposed model in human--robot interaction research are discussed. Keywords: Psycho--physics, human joint action, path integralsComment: 6 pages, Late

Model-based Reinforcement Learning with Parametrized Physical Models and Optimism-Driven Exploration

In this paper, we present a robotic model-based reinforcement learning method that combines ideas from model identification and model predictive control. We use a feature-based representation of the dynamics that allows the dynamics model to be fitted with a simple least squares procedure, and the features are identified from a high-level specification of the robot's morphology, consisting of the number and connectivity structure of its links. Model predictive control is then used to choose the actions under an optimistic model of the dynamics, which produces an efficient and goal-directed exploration strategy. We present real time experimental results on standard benchmark problems involving the pendulum, cartpole, and double pendulum systems. Experiments indicate that our method is able to learn a range of benchmark tasks substantially faster than the previous best methods. To evaluate our approach on a realistic robotic control task, we also demonstrate real time control of a simulated 7 degree of freedom arm.Comment: 8 page