Respiratory, postural and spatio-kinetic motor stabilization, internal models, top-down timed motor coordination and expanded cerebello-cerebral circuitry: a review

Human dexterity, bipedality, and song/speech vocalization in Homo are reviewed within a motor evolution perspective in regard to 

(i) brain expansion in cerebello-cerebral circuitry, 
(ii) enhanced predictive internal modeling of body kinematics, body kinetics and action organization, 
(iii) motor mastery due to prolonged practice, 
(iv) task-determined top-down, and accurately timed feedforward motor adjustment of multiple-body/artifact elements, and 
(v) reduction in automatic preflex/spinal reflex mechanisms that would otherwise restrict such top-down processes. 

Dual-task interference and developmental neuroimaging research argues that such internal modeling based motor capabilities are concomitant with the evolution of 
(vi) enhanced attentional, executive function and other high-level cognitive processes, and that 
(vii) these provide dexterity, bipedality and vocalization with effector nonspecific neural resources. 

The possibility is also raised that such neural resources could 
(viii) underlie human internal model based nonmotor cognitions. 
&#xa

Exploiting Multimodal Information in Deep Learning

Humans are good at using multimodal information to perceive and to interact with the world. Such information includes visual, auditory, kinesthetic, etc. Despite the advancement in deep learning using single modality in the past decade, there are relatively fewer works focused on multimodal learning. Even with existing multimodal deep learning works, most of them focus on a small number of modalities. This dissertation will investigate various distinct forms of multi-modal learning: multiple visual modalities as input, audio-visual multimodal input, and visual and proprioceptive (kinesthetic) multimodal input. Specifically, in the first project we investigate synthesizing light fields from a single image and estimated depth. In the second project, we investigate face recognition for unconstrained videos with audio-visual multimodal inputs. Finally, we investigate learning to construct and use tools with visual, proprioceptive and kinesthetic multimodal inputs. In the first task, we investigate synthesizing light fields with a single RGB image and its estimated depth. Synthesizing novel views (light fields) from a single image is very challenging since the depth information is lost, and depth information is crucial for view synthesis. We propose to use a pre-trained model to estimate the depth, and then fuse the depth information together with the RGB image to generate the light fields. Our experiments showed that multimodal input (RGB image and depth) significantly improved the performance over the single image input. In the second task, we focus on learning face recognition for low quality videos. For low quality videos such as low-resolution online videos and surveillance videos, recognizing faces based on video frames alone is very challenging. We propose to use audio information in the video clip to aid in the face recognition task. To achieve this goal, we propose Audio-Visual Aggregation Network (AVAN) to aggregate audio features and visual features using an attention mechanism. Empirical results show that our approach using both visual and audio information significantly improves the face recognition accuracy on unconstrained videos. Finally, in the third task, we propose to use visual, proprioceptive and kinesthetic inputs to learn to construct and use tools. The use of tools in animals indicates high levels of cognitive capability, and, aside from humans, it is observed only in a small number of higher mammals and avian species, and constructing novel tools is an even more challenging task. Learning this task with only visual input is challenging, therefore, we propose to use visual and proprioceptive (kinesthetic) inputs to accelerate the learning. We build a physically simulated environment for tool construction task. We also introduce a hierarchical reinforcement learning approach to learn to construct tools and reach the target, without any prior knowledge. The main contribution of this dissertation is in the investigation of multiple scenarios where multimodal processing leads to enhanced performance. We expect the specific methods developed in this work, such as the extraction of hidden modalities (depth), use of attention, and hierarchical rewards, to help us better understand multimodal processing in deep learning

Evolutionary robotics and neuroscience

No description supplie

On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation

Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas

Chaotic exploration and learning of locomotor behaviours

Recent developments in the embodied approach to understanding the generation of adaptive behaviour, suggests that the design of adaptive neural circuits for rhythmic motor patterns should not be done in isolation from an appreciation, and indeed exploitation, of neural-body-environment interactions. Utilising spontaneous mutual entrainment between neural systems and physical bodies provides a useful passage to the regions of phase space which are naturally structured by the neuralbody- environmental interactions. A growing body of work has provided evidence that chaotic dynamics can be useful in allowing embodied systems to spontaneously explore potentially useful motor patterns. However, up until now there has been no general integrated neural system that allows goal-directed, online, realtime exploration and capture of motor patterns without recourse to external monitoring, evaluation or training methods. For the first time, we introduce such a system in the form of a fully dynamic neural system, exploiting intrinsic chaotic dynamics, for the exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modelled as a network of neural oscillators which are coupled only through physical embodiment, and goal directed exploration of coordinated motor patterns is achieved by a chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organised dynamics each of which is a candidate for a locomotion behaviour. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states using its intrinsic chaotic dynamics as a driving force and stabilises the system on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced which results in an increased diversity of motor outputs, thus achieving multi-scale exploration. A rhythmic pattern discovered by this process is memorised and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronisation method. The dynamical nature of the weak coupling through physical embodiment allows this adaptive weight learning to be easily integrated, thus forming a continuous exploration-learning system. Our result shows that the novel neuro-robotic system is able to create and learn a number of emergent locomotion behaviours for a wide range of body configurations and physical environment, and can re-adapt after sustaining damage. The implications and analyses of these results for investigating the generality and limitations of the proposed system are discussed

Design of artificial neural oscillatory circuits for the control of lamprey- and salamander-like locomotion using evolutionary algorithms

This dissertation investigates the evolutionary design of oscillatory artificial neural networks for the control of animal-like locomotion. It is inspired by the neural organ¬ isation of locomotor circuitries in vertebrates, and explores in particular the control of undulatory swimming and walking. The difficulty with designing such controllers is to find mechanisms which can transform commands concerning the direction and the speed of motion into the multiple rhythmic signals sent to the multiple actuators typically involved in animal-like locomotion. In vertebrates, such control mechanisms are provided by central pattern generators which are neural circuits capable of pro¬ ducing the patterns of oscillations necessary for locomotion without oscillatory input from higher control centres or from sensory feedback. This thesis explores the space of possible neural configurations for the control of undulatory locomotion, and addresses the problem of how biologically plausible neural controllers can be automatically generated.Evolutionary algorithms are used to design connectionist models of central pattern generators for the motion of simulated lampreys and salamanders. This work is inspired by Ekeberg's neuronal and mechanical simulation of the lamprey [Ekeberg 93]. The first part of the thesis consists of developing alternative neural controllers for a similar mechanical simulation. Using a genetic algorithm and an incremental approach, a variety of controllers other than the biological configuration are successfully developed which can control swimming with at least the same efficiency. The same method is then used to generate synaptic weights for a controller which has the observed biological connectivity in order to illustrate how the genetic algorithm could be used for developing neurobiological models. Biologically plausible controllers are evolved which better fit physiological observations than Ekeberg's hand-crafted model. Finally, in collaboration with Jerome Kodjabachian, swimming controllers are designed using a developmental encoding scheme, in which developmental programs are evolved which determine how neurons divide and get connected to each other on a two-dimensional substrate.The second part of this dissertation examines the control of salamander-like swimming and trotting. Salamanders swim like lampreys but, on the ground, they switch to a trotting gait in which the trunk performs a standing wave with the nodes at the girdles. Little is known about the locomotion circuitry of the salamander, but neurobiologists have hypothesised that it is based on a lamprey-like organisation. A mechanical sim¬ ulation of a salamander-like animat is developed, and neural controllers capable of exhibiting the two types of gaits are evolved. The controllers are made of two neural oscillators projecting to the limb motoneurons and to lamprey-like trunk circuitry. By modulating the tonic input applied to the networks, the type of gait, the speed and the direction of motion can be varied.By developing neural controllers for lamprey- and salamander-like locomotion, this thesis provides insights into the biological control of undulatory swimming and walking, and shows how evolutionary algorithms can be used for developing neurobiological models and for generating neural controllers for locomotion. Such a method could potentially be used for designing controllers for swimming or walking robots, for instance

Behavior finding: Morphogenetic Designs Shaped by Function

Evolution has shaped an incredible diversity of multicellular living organisms, whose complex forms are self-made through a robust developmental process. This fundamental combination of biological evolution and development has served as an inspiration for novel engineering design methodologies, with the goal to overcome the scalability problems suffered by classical top-down approaches. Top-down methodologies are based on the manual decomposition of the design into modular, independent subunits. In contrast, recent computational morphogenetic techniques have shown that they were able to automatically generate truly complex innovative designs. Algorithms based on evolutionary computation and artificial development have been proposed to automatically design both the structures, within certain constraints, and the controllers that optimize their function. However, the driving force of biological evolution does not resemble an enumeration of design requirements, but much rather relies on the interaction of organisms within the environment. Similarly, controllers do not evolve nor develop separately, but are woven into the organism’s morphology. In this chapter, we discuss evolutionary morphogenetic algorithms inspired by these important aspects of biological evolution. The proposed methodologies could contribute to the automation of processes that design “organic” structures, whose morphologies and controllers are intended to solve a functional problem. The performance of the algorithms is tested on a class of optimization problems that we call behavior-finding. These challenges are not explicitly based on morphology or controller constraints, but only on the solving abilities and efficacy of the design. Our results show that morphogenetic algorithms are well suited to behavior-finding

On neuromechanical approaches for the study of biological and robotic grasp and manipulation

abstract: Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank and open-minded assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas at the interface of neuromechanics, neuroscience, rehabilitation and robotics.The electronic version of this article is the complete one and can be found online at: https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-017-0305-