Engineering entrainment and adaptation in limit cycle systems: From biological inspiration to applications in robotics

Periodic behavior is key to life and is observed in multiple instances and at multiple time scales in our metabolism, our natural environment, and our engineered environment. A natural way of modeling or generating periodic behavior is done by using oscillators, i.e., dynamical systems that exhibit limit cycle behavior. While there is extensive literature on methods to analyze such dynamical systems, much less work has been done on methods to synthesize an oscillator to exhibit some specific desired characteristics. The goal of this article is twofold: (1) to provide a framework for characterizing and designing oscillators and (2) to review how classes of well-known oscillators can be understood and related to this framework. The basis of the framework is to characterize oscillators in terms of their fundamental temporal and spatial behavior and in terms of properties that these two behaviors can be designed to exhibit. This focus on fundamental properties is important because it allows us to systematically compare a large variety of oscillators that might at first sight appear very different from each other. We identify several specifications that are useful for design, such as frequency-locking behavior, phase-locking behavior, and specific output signal shape. We also identify two classes of design methods by which these specifications can be met, namely offline methods and online methods. By relating these specifications to our framework and by presenting several examples of how oscillators have been designed in the literature, this article provides a useful methodology and toolbox for designing oscillators for a wide range of purposes. In particular, the focus on synthesis of limit cycle dynamical systems should be useful both for engineering and for computational modeling of physical or biological phenomen

Resonant hopping of a robot controlled by an artificial neural oscillator

"The bouncing gaits of terrestrial animals (hopping, running, trotting) can be modeled as a hybrid dynamic system, with spring-mass dynamics during stance and ballistic motion during the aerial phase. We used a simple hopping robot controlled by an artificial neural oscillator to test the ability of the neural oscillator to adaptively drive this hybrid dynamic system. The robot had a single joint, actuated by an artificial pneumatic muscle in series with a tendon spring. We examined how the oscillator-robot system responded to variation in two neural control parameters: descending neural drive and neuromuscular gain. We also tested the ability of the oscillator-robot system to adapt to variations in mechanical properties by changing the series and parallel spring stiffnesses. Across a 100-fold variation in both supraspinal gain and muscle gain, hopping frequency changed by less than 10%. The neural oscillator consistently drove the system at the resonant half-period for the stance phase, and adapted to a new resonant half-period when the muscle series and parallel stiffnesses were altered. Passive cycling of elastic energy in the tendon accounted for 70-79% of the mechanical work done during each hop cycle. Our results demonstrate that hopping dynamics were largely determined by the intrinsic properties of the mechanical system, not the specific choice of neural oscillator parameters. The findings provide the first evidence that an artificial neural oscillator will drive a hybrid dynamic system at partial resonance."http://deepblue.lib.umich.edu/bitstream/2027.42/64204/1/bb8_2_026001.pd

Dynamical system for learning the waveform and frequency of periodic signals - application to drumming

The paper presents a two-layered system for learning and encoding a periodic signal and its application to a drumming task. The two layers are the dynamical system responsible for extracting the main frequency of the input signal, based on adaptive frequency oscillators, and the dynamical system responsible for learning of the waveform with a built in learning algorithm. By combining the two dynamical systems we can rapidly teach new trajectories to robots. The systems works online for any periodic signal, requires no signal processing and can be applied in parallel to multiple dimensions. Furthermore, it can adapt to changes in frequency and shape, e.g. to non-stationary signals, and is computationally inexpensive. The algorithm is demonstrated in a drumming task using the HOAP-2 humanoid robot

Modeling Brain Resonance Phenomena Using a Neural Mass Model

Stimulation with rhythmic light flicker (photic driving) plays an important role in the diagnosis of schizophrenia, mood disorder, migraine, and epilepsy. In particular, the adjustment of spontaneous brain rhythms to the stimulus frequency (entrainment) is used to assess the functional flexibility of the brain. We aim to gain deeper understanding of the mechanisms underlying this technique and to predict the effects of stimulus frequency and intensity. For this purpose, a modified Jansen and Rit neural mass model (NMM) of a cortical circuit is used. This mean field model has been designed to strike a balance between mathematical simplicity and biological plausibility. We reproduced the entrainment phenomenon observed in EEG during a photic driving experiment. More generally, we demonstrate that such a single area model can already yield very complex dynamics, including chaos, for biologically plausible parameter ranges. We chart the entire parameter space by means of characteristic Lyapunov spectra and Kaplan-Yorke dimension as well as time series and power spectra. Rhythmic and chaotic brain states were found virtually next to each other, such that small parameter changes can give rise to switching from one to another. Strikingly, this characteristic pattern of unpredictability generated by the model was matched to the experimental data with reasonable accuracy. These findings confirm that the NMM is a useful model of brain dynamics during photic driving. In this context, it can be used to study the mechanisms of, for example, perception and epileptic seizure generation. In particular, it enabled us to make predictions regarding the stimulus amplitude in further experiments for improving the entrainment effect

Chaotic exploration and learning of locomotor behaviours

Recent developments in the embodied approach to understanding the generation of adaptive behaviour, suggests that the design of adaptive neural circuits for rhythmic motor patterns should not be done in isolation from an appreciation, and indeed exploitation, of neural-body-environment interactions. Utilising spontaneous mutual entrainment between neural systems and physical bodies provides a useful passage to the regions of phase space which are naturally structured by the neuralbody- environmental interactions. A growing body of work has provided evidence that chaotic dynamics can be useful in allowing embodied systems to spontaneously explore potentially useful motor patterns. However, up until now there has been no general integrated neural system that allows goal-directed, online, realtime exploration and capture of motor patterns without recourse to external monitoring, evaluation or training methods. For the first time, we introduce such a system in the form of a fully dynamic neural system, exploiting intrinsic chaotic dynamics, for the exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modelled as a network of neural oscillators which are coupled only through physical embodiment, and goal directed exploration of coordinated motor patterns is achieved by a chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organised dynamics each of which is a candidate for a locomotion behaviour. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states using its intrinsic chaotic dynamics as a driving force and stabilises the system on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced which results in an increased diversity of motor outputs, thus achieving multi-scale exploration. A rhythmic pattern discovered by this process is memorised and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronisation method. The dynamical nature of the weak coupling through physical embodiment allows this adaptive weight learning to be easily integrated, thus forming a continuous exploration-learning system. Our result shows that the novel neuro-robotic system is able to create and learn a number of emergent locomotion behaviours for a wide range of body configurations and physical environment, and can re-adapt after sustaining damage. The implications and analyses of these results for investigating the generality and limitations of the proposed system are discussed

Emergent coordination between humans and robots

Emergent coordination or movement synchronization is an often observed phenomenon in human behavior. Humans synchronize their gait when walking next to each other, they synchronize their postural sway when standing closely, and they also synchronize their movement behavior in many other situations of daily life. Why humans are doing this is an important question of ongoing research in many disciplines: apparently movement synchronization plays a role in children’s development and learning; it is related to our social and emotional behavior in interaction with others; it is an underlying principle in the organization of communication by means of language and gesture; and finally, models explaining movement synchronization between two individuals can also be extended to group behavior. Overall, one can say that movement synchronization is an important principle of human interaction behavior. Besides interacting with other humans, in recent years humans do more and more interact with technology. This was first expressed in the interaction with machines in industrial settings, was taken further to human-computer interaction and is now facing a new challenge: the interaction with active and autonomous machines, the interaction with robots. If the vision of today’s robot developers comes true, in the near future robots will be fully integrated not only in our workplace, but also in our private lives. They are supposed to support humans in activities of daily living and even care for them. These circumstances however require the development of interactional principles which the robot can apply to the direct interaction with humans. In this dissertation the problem of robots entering the human society will be outlined and the need for the exploration of human interaction principles that are transferable to human-robot interaction will be emphasized. Furthermore, an overview on human movement synchronization as a very important phenomenon in human interaction will be given, ranging from neural correlates to social behavior. The argument of this dissertation is that human movement synchronization is a simple but striking human interaction principle that can be applied in human-robot interaction to support human activity of daily living, demonstrated on the example of pick-and-place tasks. This argument is based on five publications. In the first publication, human movement synchronization is explored in goal-directed tasks which bare similar requirements as pick-and-place tasks in activities of daily living. In order to explore if a merely repetitive action of the robot is sufficient to encourage human movement synchronization, the second publication reports a human-robot interaction study in which a human interacts with a non-adaptive robot. Here however, movement synchronization between human and robot does not emerge, which underlines the need for adaptive mechanisms. Therefore, in the third publication, human adaptive behavior in goal-directed movement synchronization is explored. In order to make the findings from the previous studies applicable to human-robot interaction, in the fourth publication the development of an interaction model based on dynamical systems theory is outlined which is ready for implementation on a robotic platform. Following this, a brief overview on a first human-robot interaction study based on the developed interaction model is provided. The last publication describes an extension of the previous approach which also includes the human tendency to make use of events to adapt their movements to. Here, also a first human-robot interaction study is reported which confirms the applicability of the model. The dissertation concludes with a discussion on the presented findings in the light of human-robot interaction and psychological aspects of joint action research as well as the problem of mutual adaptation.Spontan auftretende Koordination oder Bewegungssynchronisierung ist ein häufig zu beobachtendes Phänomen im Verhalten von Menschen. Menschen synchronisieren ihre Schritte beim nebeneinander hergehen, sie synchronisieren die Schwingbewegung zum Ausgleich der Körperbalance wenn sie nahe beieinander stehen und sie synchronisieren ihr Bewegungsverhalten generell in vielen weiteren Handlungen des täglichen Lebens. Die Frage nach dem warum ist eine Frage mit der sich die Forschung in der Psychologie, Neuro- und Bewegungswissenschaft aber auch in der Sozialwissenschaft nach wie vor beschäftigt: offenbar spielt die Bewegungssynchronisierung eine Rolle in der kindlichen Entwicklung und beim Erlernen von Fähigkeiten und Verhaltensmustern; sie steht in direktem Bezug zu unserem sozialen Verhalten und unserer emotionalen Wahrnehmung in der Interaktion mit Anderen; sie ist ein grundlegendes Prinzip in der Organisation von Kommunikation durch Sprache oder Gesten; außerdem können Modelle, die Bewegungssynchronisierung zwischen zwei Individuen erklären, auch auf das Verhalten innerhalb von Gruppen ausgedehnt werden. Insgesamt kann man also sagen, dass Bewegungssynchronisierung ein wichtiges Prinzip im menschlichen Interaktionsverhalten darstellt. Neben der Interaktion mit anderen Menschen interagieren wir in den letzten Jahren auch zunehmend mit der uns umgebenden Technik. Hier fand zunächst die Interaktion mit Maschinen im industriellen Umfeld Beachtung, später die Mensch-Computer-Interaktion. Seit kurzem sind wir jedoch mit einer neuen Herausforderung konfrontiert: der Interaktion mit aktiven und autonomen Maschinen, Maschinen die sich bewegen und aktiv mit Menschen interagieren, mit Robotern. Sollte die Vision der heutigen Roboterentwickler Wirklichkeit werde, so werden Roboter in der nahen Zukunft nicht nur voll in unser Arbeitsumfeld integriert sein, sondern auch in unser privates Leben. Roboter sollen den Menschen in ihren täglichen Aktivitäten unterstützen und sich sogar um sie kümmern. Diese Umstände erfordern die Entwicklung von neuen Interaktionsprinzipien, welche Roboter in der direkten Koordination mit dem Menschen anwenden können. In dieser Dissertation wird zunächst das Problem umrissen, welches sich daraus ergibt, dass Roboter zunehmend Einzug in die menschliche Gesellschaft finden. Außerdem wird die Notwendigkeit der Untersuchung menschlicher Interaktionsprinzipien, die auf die Mensch-Roboter-Interaktion transferierbar sind, hervorgehoben. Die Argumentation der Dissertation ist, dass die menschliche Bewegungssynchronisierung ein einfaches aber bemerkenswertes menschliches Interaktionsprinzip ist, welches in der Mensch-Roboter-Interaktion angewendet werden kann um menschliche Aktivitäten des täglichen Lebens, z.B. Aufnahme-und-Ablege-Aufgaben (pick-and-place tasks), zu unterstützen. Diese Argumentation wird auf fünf Publikationen gestützt. In der ersten Publikation wird die menschliche Bewegungssynchronisierung in einer zielgerichteten Aufgabe untersucht, welche die gleichen Anforderungen erfüllt wie die Aufnahme- und Ablageaufgaben des täglichen Lebens. Um zu untersuchen ob eine rein repetitive Bewegung des Roboters ausreichend ist um den Menschen zur Etablierung von Bewegungssynchronisierung zu ermutigen, wird in der zweiten Publikation eine Mensch-Roboter-Interaktionsstudie vorgestellt in welcher ein Mensch mit einem nicht-adaptiven Roboter interagiert. In dieser Studie wird jedoch keine Bewegungssynchronisierung zwischen Mensch und Roboter etabliert, was die Notwendigkeit von adaptiven Mechanismen unterstreicht. Daher wird in der dritten Publikation menschliches Adaptationsverhalten in der Bewegungssynchronisierung in zielgerichteten Aufgaben untersucht. Um die so gefundenen Mechanismen für die Mensch-Roboter Interaktion nutzbar zu machen, wird in der vierten Publikation die Entwicklung eines Interaktionsmodells basierend auf Dynamischer Systemtheorie behandelt. Dieses Modell kann direkt in eine Roboterplattform implementiert werden. Anschließend wird kurz auf eine erste Studie zur Mensch- Roboter Interaktion basierend auf dem entwickelten Modell eingegangen. Die letzte Publikation beschreibt eine Weiterentwicklung des bisherigen Vorgehens welche der Tendenz im menschlichen Verhalten Rechnung trägt, die Bewegungen an Ereignissen auszurichten. Hier wird außerdem eine erste Mensch-Roboter- Interaktionsstudie vorgestellt, die die Anwendbarkeit des Modells bestätigt. Die Dissertation wird mit einer Diskussion der präsentierten Ergebnisse im Kontext der Mensch-Roboter-Interaktion und psychologischer Aspekte der Interaktionsforschung sowie der Problematik von beiderseitiger Adaptivität abgeschlossen

Engineering limit cycle systems:adaptive frequency oscillators and applications to adaptive locomotion control of compliant robots

In this thesis, we present a dynamical systems approach to adaptive controllers for locomotion control. The approach is based on a rigorous mathematical framework using nonlinear dynamical systems and is inspired by theories of self-organization. Nonlinear dynamical systems such as coupled oscillators are an interesting approach for the on-line generation of trajectories for robots with many degrees of freedom (e.g. legged locomotion). However, designing a nonlinear dynamical system to satisfy a given specification and goal is not an easy task, and, hitherto no methodology exists to approach this problem in a unified way. Nature presents us with satisfactory solutions for the coordination of many degrees of freedom. One central feature observed in biological subjects is the ability of the neural systems to exploit natural dynamics of the body to achieve efficient locomotion. In order to be able to exploit the body properties, adaptive mechanisms must be at work. Recent work has pointed out the importance of the mechanical system for efficient locomotion. Even more interestingly, such well suited mechanical systems do not need complicated control. Yet, in robotics, in most approaches, adaptive mechanisms are either missing or they are not based on a rigorous framework, i.e. they are based on heuristics and ad-hoc approaches. Over the last three decades there has been enormous progress in describing movement coordination with the help of Synergetic approaches. This has led to the formulation of a theoretical framework: the theory of dynamic patterns. This framework is mathematically rigorous and at the same time fully operational. However, it does not provide any guidelines for synthetic approaches as needed for the engineering of robots with many degrees of freedom, nor does it directly help to explain adaptive systems. We will show how we can extend the theoretical framework to build adaptive systems. For this purpose, we propose the use of multi-scale dynamical systems. The basic idea behind multi-scale dynamical systems is that a given dynamical system gets extended by additional slow dynamics of its parameters, i.e. some of the parameters become state variables. The advantages of the framework of multi-scale dynamical systems for adaptive controllers are 1) fully dynamic description, 2) no separation of learning algorithm and learning substrate, 3) no separation of learning trials or time windows, 4) mathematically rigorous, 5) low dimensional systems. However, in order to fully exploit the framework important questions have to be solved. Most importantly, methodologies for designing the feedback loops have to be found and important theoretical questions about stability and convergence properties of the devised systems have to be answered. In order to tackle this challenge, we first introduce an engineering view on designing nonlinear dynamical systems and especially oscillators. We will highlight the important differences and freedom that this engineering view introduces as opposed to a modeling one. We then apply this approach by first proposing a very simple adaptive toy-system, consisting of a dynamical system coupled to a spring-mass system. Due to its spring-mass dynamics, this system contains clear natural dynamics in the form of resonant frequencies. We propose a prototype adaptive multi-scale system, the adaptive frequency oscillator, which is able to adapt its intrinsic frequency to the resonant frequency of the body dynamics. After a small sidetrack to show that we can use adaptive frequency oscillators also for other applications than for adaptive controllers, namely for frequency analysis, we then come back to further investigation of the adaptive controller. We apply the same controller concept to a simple spring-mass hopper system. The spring-mass system consists of a body with two legs attached by rotational joints. The legs contain spring-damper elements. Finally, we present results of the implementation of the controller on a real robot, the experimental robot PUPPY II. This robot is a under-actuated robot with spring dynamics in the knee joints. It will be shown, that due to the appropriate simplification and concentration on relevant features in the toy-system the controller concepts works without a fundamental change on all systems from the toy system up to the real robot

Um sistema de controle reativo para locomoção de robôs quadrúpedes

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia de Automação e Sistemas, Florianópolis, 2013.A locomoção ágil de robôs com pernas em terrenos irregulares necessita que todas as tarefas - desde a geração/planejamento de trajetórias ao controle do movimento - interajam de maneira harmoniosa. As diferentes tarefas, por exemplo, a geração de trajetórias e as ações de controle, em geral, não podem gerar conflitos com relação ao movimento desejado. Nesta tese, propõe-se uma estrutura de controle reativa para locomoção de robôs quadrúpedes em terrenos irregulares. O objetivo de tal estrutura è fazer frente à problemas relacionados à locomoção em superfícies irregulares, ao erro de rastreamento de trajetória e a imprecisão da estimação de estados. A estrutura compreende a dois módulos principais: um relacionado a geração do movimento, e outro relacionado ao controle do movimento do robô. Para a geração do movimento propõe-se uma abordagem baseada em Geradores de Padrões Centrais, que geram referências no espaço da tarefa e podem ser modulados de acordo com a superfície do terreno. Para o controle do movimento propõe-se uma estratégia de controle baseado em projeção de espaço nulo e uma estratégia de controle para rejeição de distúrbios baseada no conceito de pontos de captura. As principais contribuições teóricas foram validadas em simulação e implementadas em um robô real. Ao final do documento, tais resultados são apresentados para demonstrar a efetividade da estrutura proposta.2014-08-06T17:47:31