1,254 research outputs found
Evolution of central pattern generators for the control of a five-link bipedal walking mechanism
Central pattern generators (CPGs), with a basis is neurophysiological
studies, are a type of neural network for the generation of rhythmic motion.
While CPGs are being increasingly used in robot control, most applications are
hand-tuned for a specific task and it is acknowledged in the field that generic
methods and design principles for creating individual networks for a given task
are lacking. This study presents an approach where the connectivity and
oscillatory parameters of a CPG network are determined by an evolutionary
algorithm with fitness evaluations in a realistic simulation with accurate
physics. We apply this technique to a five-link planar walking mechanism to
demonstrate its feasibility and performance. In addition, to see whether
results from simulation can be acceptably transferred to real robot hardware,
the best evolved CPG network is also tested on a real mechanism. Our results
also confirm that the biologically inspired CPG model is well suited for legged
locomotion, since a diverse manifestation of networks have been observed to
succeed in fitness simulations during evolution.Comment: 11 pages, 9 figures; substantial revision of content, organization,
and quantitative result
Chaotic exploration and learning of locomotion behaviours
We present a general and fully dynamic neural system, which exploits intrinsic chaotic dynamics, for the real-time goal-directed exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modeled as a network of neural oscillators that are initially coupled only through physical embodiment, and goal-directed exploration of coordinated motor patterns is achieved by chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organized dynamics, each of which is a candidate for a locomotion behavior. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states, using its intrinsic chaotic dynamics as a driving force, and stabilizes on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced, which results in an increased diversity of motor outputs, thus achieving multiscale exploration. A rhythmic pattern discovered by this process is memorized and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronization method. Our results show that the novel neurorobotic system is able to create and learn multiple locomotion behaviors for a wide range of body configurations and physical environments and can readapt in realtime after sustaining damage
MOTION CONTROL SIMULATION OF A HEXAPOD ROBOT
This thesis addresses hexapod robot motion control. Insect morphology and locomotion patterns inform the design of a robotic model, and motion control is achieved via trajectory planning and bio-inspired principles. Additionally, deep learning and multi-agent reinforcement learning are employed to train the robot motion control strategy with leg coordination achieves using a multi-agent deep reinforcement learning framework. The thesis makes the following contributions:
First, research on legged robots is synthesized, with a focus on hexapod robot motion control. Insect anatomy analysis informs the hexagonal robot body and three-joint single robotic leg design, which is assembled using SolidWorks. Different gaits are studied and compared, and robot leg kinematics are derived and experimentally verified, culminating in a three-legged gait for motion control.
Second, an animal-inspired approach employs a central pattern generator (CPG) control unit based on the Hopf oscillator, facilitating robot motion control in complex environments such as stable walking and climbing. The robot\u27s motion process is quantitatively evaluated in terms of displacement change and body pitch angle.
Third, a value function decomposition algorithm, QPLEX, is applied to hexapod robot motion control. The QPLEX architecture treats each leg as a separate agent with local control modules, that are trained using reinforcement learning. QPLEX outperforms decentralized approaches, achieving coordinated rhythmic gaits and increased robustness on uneven terrain. The significant of terrain curriculum learning is assessed, with QPLEX demonstrating superior stability and faster consequence.
The foot-end trajectory planning method enables robot motion control through inverse kinematic solutions but has limited generalization capabilities for diverse terrains. The animal-inspired CPG-based method offers a versatile control strategy but is constrained to core aspects. In contrast, the multi-agent deep reinforcement learning-based approach affords adaptable motion strategy adjustments, rendering it a superior control policy. These methods can be combined to develop a customized robot motion control policy for specific scenarios
Motion representation with spiking neural networks for grasping and manipulation
Die Natur bedient sich Millionen von Jahren der Evolution, um adaptive physikalische Systeme mit effizienten Steuerungsstrategien zu erzeugen. Im Gegensatz zur konventionellen Robotik plant der Mensch nicht einfach eine Bewegung und fĂŒhrt sie aus, sondern es gibt eine Kombination aus mehreren Regelkreisen, die zusammenarbeiten, um den Arm zu bewegen und ein Objekt mit der Hand zu greifen. Mit der Forschung an humanoiden und biologisch inspirierten Robotern werden komplexe kinematische Strukturen und komplizierte Aktor- und Sensorsysteme entwickelt. Diese Systeme sind schwierig zu steuern und zu programmieren, und die klassischen Methoden der Robotik können deren StĂ€rken nicht immer optimal ausnutzen. Die neurowissenschaftliche Forschung hat groĂe Fortschritte beim VerstĂ€ndnis der verschiedenen Gehirnregionen und ihrer entsprechenden Funktionen gemacht. Dennoch basieren die meisten Modelle auf groĂ angelegten Simulationen, die sich auf die Reproduktion der KonnektivitĂ€t und der statistischen neuronalen AktivitĂ€t konzentrieren. Dies öffnet eine LĂŒcke bei der Anwendung verschiedener Paradigmen, um Gehirnmechanismen und Lernprinzipien zu validieren und Funktionsmodelle zur Steuerung von Robotern zu entwickeln. Ein vielversprechendes Paradigma ist die ereignis-basierte Berechnung mit SNNs. SNNs fokussieren sich auf die biologischen Aspekte von Neuronen und replizieren deren Arbeitsweise. Sie sind fĂŒr spike- basierte Kommunikation ausgelegt und ermöglichen die Erforschung von Mechanismen des Gehirns fĂŒr das Lernen mittels neuronaler PlastizitĂ€t. Spike-basierte Kommunikation nutzt hoch parallelisierten Hardware-Optimierungen mittels neuromorpher Chips, die einen geringen Energieverbrauch und schnelle lokale Operationen ermöglichen.
In dieser Arbeit werden verschiedene SNNs zur DurchfĂŒhrung von Bewegungss- teuerung fĂŒr Manipulations- und Greifaufgaben mit einem Roboterarm und einer anthropomorphen Hand vorgestellt. Diese basieren auf biologisch inspirierten funktionalen Modellen des menschlichen Gehirns. Ein Motor-Primitiv wird auf parametrische Weise mit einem Aktivierungsparameter und einer Abbildungsfunktion auf die Roboterkinematik ĂŒbertragen. Die Topologie des SNNs spiegelt die kinematische Struktur des Roboters wider. Die Steuerung des Roboters erfolgt ĂŒber das Joint Position Interface. Um komplexe Bewegungen und Verhaltensweisen modellieren zu können, werden die Primitive in verschiedenen Schichten einer Hierarchie angeordnet. Dies ermöglicht die Kombination und Parametrisierung der Primitiven und die Wiederverwendung von einfachen Primitiven fĂŒr verschiedene Bewegungen. Es gibt verschiedene Aktivierungsmechanismen fĂŒr den Parameter, der ein Motorprimitiv steuert â willkĂŒrliche, rhythmische und reflexartige. AuĂerdem bestehen verschiedene Möglichkeiten neue Motorprimitive entweder online oder offline zu lernen. Die Bewegung kann entweder als Funktion modelliert oder durch Imitation der menschlichen AusfĂŒhrung gelernt werden. Die SNNs können in andere Steuerungssysteme integriert oder mit anderen SNNs kombiniert werden. Die Berechnung der inversen Kinematik oder die Validierung von Konfigurationen fĂŒr die Planung ist nicht erforderlich, da der Motorprimitivraum nur durchfĂŒhrbare Bewegungen hat und keine ungĂŒltigen Konfigurationen enthĂ€lt.
FĂŒr die Evaluierung wurden folgende Szenarien betrachtet, das Zeigen auf verschiedene Ziele, das Verfolgen einer Trajektorie, das AusfĂŒhren von rhythmischen oder sich wiederholenden Bewegungen, das AusfĂŒhren von Reflexen und das Greifen von einfachen Objekten. ZusĂ€tzlich werden die Modelle des Arms und der Hand kombiniert und erweitert, um die mehrbeinige Fortbewegung als Anwendungsfall der Steuerungsarchitektur mit Motorprimitiven zu modellieren. Als Anwendungen fĂŒr einen Arm (3 DoFs) wurden die Erzeugung von Zeigebewegungen und das perzeptionsgetriebene Erreichen von Zielen modelliert. Zur Erzeugung von Zeigebewegun- gen wurde ein Basisprimitiv, das auf den Mittelpunkt einer Ebene zeigt, offline mit vier Korrekturprimitiven kombiniert, die eine neue Trajektorie erzeugen. FĂŒr das wahrnehmungsgesteuerte Erreichen eines Ziels werden drei Primitive online kombiniert unter Verwendung eines Zielsignals.
Als Anwendungen fĂŒr eine FĂŒnf-Finger-Hand (9 DoFs) wurden individuelle Finger-aktivierungen und Soft-Grasping mit nachgiebiger Steuerung modelliert. Die Greif- bewegungen werden mit Motor-Primitiven in einer Hierarchie modelliert, wobei die Finger-Primitive die Synergien zwischen den Gelenken und die Hand-Primitive die unterschiedlichen Affordanzen zur Koordination der Finger darstellen. FĂŒr jeden Finger werden zwei Reflexe hinzugefĂŒgt, zum Aktivieren oder Stoppen der Bewegung bei Kontakt und zum Aktivieren der nachgiebigen Steuerung.
Dieser Ansatz bietet enorme FlexibilitĂ€t, da Motorprimitive wiederverwendet, parametrisiert und auf unterschiedliche Weise kombiniert werden können. Neue Primitive können definiert oder gelernt werden. Ein wichtiger Aspekt dieser Arbeit ist, dass im Gegensatz zu Deep Learning und End-to-End-Lernmethoden, keine umfangreichen DatensĂ€tze benötigt werden, um neue Bewegungen zu lernen. Durch die Verwendung von Motorprimitiven kann der gleiche Modellierungsansatz fĂŒr verschiedene Roboter verwendet werden, indem die Abbildung der Primitive auf die Roboterkinematik neu definiert wird. Die Experimente zeigen, dass durch Motor- primitive die Motorsteuerung fĂŒr die Manipulation, das Greifen und die Lokomotion vereinfacht werden kann. SNNs fĂŒr Robotikanwendungen ist immer noch ein Diskussionspunkt. Es gibt keinen State-of-the-Art-Lernalgorithmus, es gibt kein Framework Ă€hnlich dem fĂŒr Deep Learning, und die Parametrisierung von SNNs ist eine Kunst. Nichtsdestotrotz können Robotikanwendungen - wie Manipulation und Greifen - Benchmarks und realistische Szenarien liefern, um neurowissenschaftliche Modelle zu validieren. AuĂerdem kann die Robotik die Möglichkeiten der ereignis- basierten Berechnung mit SNNs und neuromorpher Hardware nutzen. Die physikalis- che Nachbildung eines biologischen Systems, das vollstĂ€ndig mit SNNs implementiert und auf echten Robotern evaluiert wurde, kann neue Erkenntnisse darĂŒber liefern, wie der Mensch die Motorsteuerung und Sensorverarbeitung durchfĂŒhrt und wie diese in der Robotik angewendet werden können. Modellfreie Bewegungssteuerungen, inspiriert von den Mechanismen des menschlichen Gehirns, können die Programmierung von Robotern verbessern, indem sie die Steuerung adaptiver und flexibler machen
Fast biped walking with a neuronal controller and physical computation
Biped walking remains a difficult problem and robot models can
greatly {facilitate} our understanding of the underlying
biomechanical principles as well as their neuronal control. The
goal of this study is to specifically demonstrate that stable
biped walking can be achieved by combining the physical properties
of the walking robot with a small, reflex-based neuronal network,
which is governed mainly by local sensor signals. This study shows
that human-like gaits emerge without {specific} position or
trajectory control and that the walker is able to compensate small
disturbances through its own dynamical properties. The reflexive
controller used here has the following characteristics, which are
different from earlier approaches: (1) Control is mainly local.
Hence, it uses only two signals (AEA=Anterior Extreme Angle and
GC=Ground Contact) which operate at the inter-joint level. All
other signals operate only at single joints. (2) Neither position
control nor trajectory tracking control is used. Instead, the
approximate nature of the local reflexes on each joint allows the
robot mechanics itself (e.g., its passive dynamics) to contribute
substantially to the overall gait trajectory computation. (3) The
motor control scheme used in the local reflexes of our robot is
more straightforward and has more biological plausibility than
that of other robots, because the outputs of the motorneurons in
our reflexive controller are directly driving the motors of the
joints, rather than working as references for position or velocity
control. As a consequence, the neural controller and the robot
mechanics are closely coupled as a neuro-mechanical system and
this study emphasises that dynamically stable biped walking gaits
emerge from the coupling between neural computation and physical
computation. This is demonstrated by different walking
experiments using two real robot as well as by a Poincar\'{e} map
analysis applied on a model of the robot in order to assess its
stability. In addition, this neuronal control structure allows the
use of a policy gradient reinforcement learning algorithm to tune
the parameters of the neurons in real-time, during walking. This
way the robot can reach a record-breaking walking speed of 3.5
leg-lengths per second after only a few minutes of online
learning, which is even comparable to the fastest relative speed
of human walking
Spinal Locomotor Circuits Develop Using Hierarchical Rules Based on Motorneuron Position and Identity
SummaryThe coordination of multi-muscle movements originates in the circuitry that regulates the firing patterns of spinal motorneurons. Sensory neurons rely on the musculotopic organization of motorneurons to establish orderly connections, prompting us to examine whether the intraspinal circuitry that coordinates motor activity likewise uses cell position as an internal wiring reference. We generated a motorneuron-specific GCaMP6f mouse line and employed two-photon imaging to monitor the activity of lumbar motorneurons. We show that the central pattern generator neural network coordinately drives rhythmic columnar-specific motorneuron bursts at distinct phases of the locomotor cycle. Using multiple genetic strategies to perturb the subtype identity and orderly position of motorneurons, we found that neurons retained their rhythmic activityâbut cell position was decoupled from the normal phasing pattern underlying flexion and extension. These findings suggest a hierarchical basis of motor circuit formation that relies on increasingly stringent matching of neuronal identity and position
Chaotic exploration and learning of locomotor behaviours
Recent developments in the embodied approach to understanding the generation of
adaptive behaviour, suggests that the design of adaptive neural circuits for rhythmic
motor patterns should not be done in isolation from an appreciation, and indeed
exploitation, of neural-body-environment interactions. Utilising spontaneous mutual
entrainment between neural systems and physical bodies provides a useful passage
to the regions of phase space which are naturally structured by the neuralbody-
environmental interactions. A growing body of work has provided evidence
that chaotic dynamics can be useful in allowing embodied systems to spontaneously
explore potentially useful motor patterns. However, up until now there has
been no general integrated neural system that allows goal-directed, online, realtime
exploration and capture of motor patterns without recourse to external monitoring,
evaluation or training methods. For the first time, we introduce such a system
in the form of a fully dynamic neural system, exploiting intrinsic chaotic dynamics,
for the exploration and learning of the possible locomotion patterns of an articulated
robot of an arbitrary morphology in an unknown environment. The controller
is modelled as a network of neural oscillators which are coupled only through physical
embodiment, and goal directed exploration of coordinated motor patterns is
achieved by a chaotic search using adaptive bifurcation. The phase space of the
indirectly coupled neural-body-environment system contains multiple transient or
permanent self-organised dynamics each of which is a candidate for a locomotion
behaviour. The adaptive bifurcation enables the system orbit to wander through
various phase-coordinated states using its intrinsic chaotic dynamics as a driving
force and stabilises the system on to one of the states matching the given goal
criteria. In order to improve the sustainability of useful transient patterns, sensory
homeostasis has been introduced which results in an increased diversity of motor outputs,
thus achieving multi-scale exploration. A rhythmic pattern discovered by this
process is memorised and sustained by changing the wiring between initially disconnected
oscillators using an adaptive synchronisation method. The dynamical nature
of the weak coupling through physical embodiment allows this adaptive weight learning
to be easily integrated, thus forming a continuous exploration-learning system.
Our result shows that the novel neuro-robotic system is able to create and learn a
number of emergent locomotion behaviours for a wide range of body configurations
and physical environment, and can re-adapt after sustaining damage. The implications
and analyses of these results for investigating the generality and limitations of
the proposed system are discussed
Reproducing Five Motor Behaviors in a Salamander Robot With Virtual Muscles and a Distributed CPG Controller Regulated by Drive Signals and Proprioceptive Feedback
Diverse locomotor behaviors emerge from the interactions between the spinal central pattern generator (CPG), descending brain signals and sensory feedback. Salamander motor behaviors include swimming, struggling, forward underwater stepping, and forward and backward terrestrial stepping. Electromyographic and kinematic recordings of the trunk show that each of these five behaviors is characterized by specific patterns of muscle activation and body curvature. Electrophysiological recordings in isolated spinal cords show even more diverse patterns of activity. Using numerical modeling and robotics, we explored the mechanisms through which descending brain signals and proprioceptive feedback could take advantage of the flexibility of the spinal CPG to generate different motor patterns. Adapting a previous CPG model based on abstract oscillators, we propose a model that reproduces the features of spinal cord recordings: the diversity of motor patterns, the correlation between phase lags and cycle frequencies, and the spontaneous switches between slow and fast rhythms. The five salamander behaviors were reproduced by connecting the CPG model to a mechanical simulation of the salamander with virtual muscles and local proprioceptive feedback. The main results were validated on a robot. A distributed controller was used to obtain the fast control loops necessary for implementing the virtual muscles. The distributed control is demonstrated in an experiment where the robot splits into multiple functional parts. The five salamander behaviors were emulated by regulating the CPG with two descending drives. Reproducing the kinematics of backward stepping and struggling however required stronger muscle contractions. The passive oscillations observed in the salamander's tail during forward underwater stepping could be reproduced using a third descending drive of zero to the tail oscillators. This reduced the drag on the body in our hydrodynamic simulation. We explored the effect of local proprioceptive feedback during swimming and forward terrestrial stepping. We found that feedback could replace or reduce the need for different drives in both cases. It also reduced the variability of intersegmental phase lags toward values appropriate for locomotion. Our work suggests that different motor behaviors do not require different CPG circuits: a single circuit can produce various behaviors when modulated by descending drive and sensory feedback
Corticospinal Excitability During Locomotion in Humans
The coordination between signals from cortical structures and spinal segmental pathways responsible for the control of locomotion remains a contentious issue in human motor control. The signals are known to be integrated, but the nature of these neural calculations is unknown. To understand these interactions in humans, noninvasive cortical stimulation techniques can be combined with detailed analyses of muscle activity patterns in locomotor tasks.;In this study, I tested the relationship between corticospinal inputs and the forward velocity of each limb. Transcranial magnetic stimulation (TMS) was used to elicit motor evoked potentials (MEPs) in 12 healthy human volunteers during locomotion on a split-belt treadmill, allowing for the evaluation of corticospinal excitability (CSE) throughout 4 gait tasks. Participants were instrumented with electromyography (EMG) sensors to collect activity of representative muscles of both lower limbs. The velocity conditions were limited to two symmetrical tasks, with both belts moving at either 1 or 1.25 m/s, and two asymmetrical tasks, with one belt moving at 1 and the other at 1.25 m/s. During each trial, a double cone coil was used to stimulate the area of the primary motor cortex (M1) associated with voluntary control of the lower limbs, throughout different phases of the step cycle. Real-time stimulation targeting was accomplished using tracking hardware and neuronavigation software, and the relative location of stimulation was automatically saved to ensure consistency between trials. The MEP epochs within the EMG signals were detected using an outgoing TMS synchronization pulse. Individual MEP magnitudes were normalized to the pre-stimulation muscle activity for comparison, and were binned according to step cycle phase.;Statistical tests were conducted using intra- and inter-subject group means from different velocity conditions, and phases of the step cycle. The relative locations of stimulation were also analyzed to describe the accuracy and precision of stimulation across trials. The non-normalized MEP amplitude was correlated with the muscle activity during control steps. The pattern of MEP modulation supported the hypothesis that CSE contains information about limb velocity
- âŠ