3D printed neuromorphic sensing systems

Thanks to the high energy efficiency, neuromorphic devices are spotlighted recently by mimicking the calculation principle of the human brain through the parallel computation and the memory function. Various bio-inspired \u27in-memory computing\u27 (IMC) devices were developed during the past decades, such as synaptic transistors for artificial synapses. By integrating with specific sensors, neuromorphic sensing systems are achievable with the bio-inspired signal perception function. A signal perception process is possible by a combination of stimuli sensing, signal conversion/transmission, and signal processing. However, most neuromorphic sensing systems were demonstrated without signal conversion/transmission functions. Therefore, those cannot fully mimic the function provides by the sensory neuron in the biological system. This thesis aims to design a neuromorphic sensing system with a complete function as biological sensory neurons. To reach such a target, 3D printed sensors, electrical oscillators, and synaptic transistors were developed as functions of artificial receptors, artificial neurons, and artificial synapses, respectively. Moreover, since the 3D printing technology has demonstrated a facile process due to fast prototyping, the proposed 3D neuromorphic sensing system was designed as a 3D integrated structure and fabricated by 3D printing technologies. A novel multi-axis robot 3D printing system was also utilized to increase the fabrication efficiency with the capability of printing on vertical and tilted surfaces seamlessly. Furthermore, the developed 3D neuromorphic system was easily adapted to the application of tactile sensing. A portable neuromorphic system was integrated with a tactile sensing system for the intelligent tactile sensing application of the humanoid robot. Finally, the bio-inspired reflex arc for the unconscious response was also demonstrated by training the neuromorphic tactile sensing system

A Robotic System for Learning Visually-Driven Grasp Planning (Dissertation Proposal)

We use findings in machine learning, developmental psychology, and neurophysiology to guide a robotic learning system\u27s level of representation both for actions and for percepts. Visually-driven grasping is chosen as the experimental task since it has general applicability and it has been extensively researched from several perspectives. An implementation of a robotic system with a gripper, compliant instrumented wrist, arm and vision is used to test these ideas. Several sensorimotor primitives (vision segmentation and manipulatory reflexes) are implemented in this system and may be thought of as the innate perceptual and motor abilities of the system. Applying empirical learning techniques to real situations brings up such important issues as observation sparsity in high-dimensional spaces, arbitrary underlying functional forms of the reinforcement distribution and robustness to noise in exemplars. The well-established technique of non-parametric projection pursuit regression (PPR) is used to accomplish reinforcement learning by searching for projections of high-dimensional data sets that capture task invariants. We also pursue the following problem: how can we use human expertise and insight into grasping to train a system to select both appropriate hand preshapes and approaches for a wide variety of objects, and then have it verify and refine its skills through trial and error. To accomplish this learning we propose a new class of Density Adaptive reinforcement learning algorithms. These algorithms use statistical tests to identify possibly interesting regions of the attribute space in which the dynamics of the task change. They automatically concentrate the building of high resolution descriptions of the reinforcement in those areas, and build low resolution representations in regions that are either not populated in the given task or are highly uniform in outcome. Additionally, the use of any learning process generally implies failures along the way. Therefore, the mechanics of the untrained robotic system must be able to tolerate mistakes during learning and not damage itself. We address this by the use of an instrumented, compliant robot wrist that controls impact forces

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 299)

This bibliography lists 96 reports, articles, and other documents introduced into the NASA scientific and technical information system in June, 1987

Human inspired humanoid robots control architecture

This PhD Thesis tries to present a different point of view when talking about the development of control architectures for humanoid robots. Specifically, this Thesis is focused on studying the human postural control system as well as on the use of this knowledge to develop a novel architecture for postural control in humanoid robots. The research carried on in this thesis shows that there are two types of components for postural control: a reactive one, and other predictive or anticipatory. This work has focused on the development of the second component through the implementation of a predictive system complementing the reactive one. The anticipative control system has been analysed in the human case and it has been extrapolated to the architecture for controlling the humanoid robot TEO. In this way, its different components have been developed based on how humans work without forgetting the tasks it has been designed for. This control system is based on the composition of sensorial perceptions, the evaluation of stimulus through the use of the psychophysics theory of the surprise, and the creation of events that can be used for activating some reaction strategies (synergies) The control system developed in this Thesis, as well as the human being does, processes information coming from different sensorial sources. It also composes the named perceptions, which depend on the type of task the postural control acts over. The value of those perceptions is obtained using bio-inspired evaluation techniques of sensorial inference. Once the sensorial input has been obtained, it is necessary to process it in order to foresee possible disturbances that may provoke an incorrect performance of a task. The system developed in this Thesis evaluates the sensorial information, previously transformed into perceptions, through the use of the “Surprise Theory”, and it generates some events called “surprises” used for predicting the evolution of a task. Finally, the anticipative system for postural control can compose, if necessary, the proper reactions through the use of predefined movement patterns called synergies. Those reactions can complement or substitute completely the normal performance of a task. The performance of the anticipative system for postural control as well as the performance of each one of its components have been tested through simulations and the application of the results in the humanoid robot TEO from the RoboticsLab research group in the Systems Engineering and Automation Department from the Carlos III University of Madrid. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Esta Tesis Doctoral pretende aportar un punto de vista diferente en el desarrollo de arquitecturas de control para robots humanoides. En concreto, esta Tesis se centra en el estudio del sistema de control postural humano y en la aplicación de este conocimiento en el desarrollo de una nueva arquitectura de control postural para robots humanoides. El estudio realizado en esta Tesis pone de manifiesto la existencia de una componente de control postural reactiva y otra predictiva o anticipativa. Este trabajo se ha centrado en el desarrollo de la segunda componente mediante la implementación de un sistema predictivo que complemente al sistema reactivo. El sistema de control anticipativo ha sido estudiado en el caso humano y extrapolado para la arquitectura de control del robot humanoide TEO. De este modo, sus diferentes componentes han sido desarrollados inspirándose en el funcionamiento humano y considerando las tareas para las que dicho robot ha sido concebido. Dicho sistema está basado en la composición de percepciones sensoriales, la evaluación de los estímulos mediante el uso de la teoría psicofísica de la sorpresa y la generación de eventos que sirvan para activar estrategias de reacción (sinergias). El sistema de control desarrollado en esta Tesis, al igual que el ser humano, procesa información de múltiples fuentes sensoriales y compone las denominadas percepciones, que dependen del tipo de tarea sobre la que actúa el control postural. El valor de estas percepciones es obtenido utilizando técnicas de evaluación bioinspiradas de inferencia sensorial. Una vez la entrada sensorial ha sido obtenida, es necesario procesarla para prever posibles perturbaciones que puedan ocasionar una incorrecta realización de una tarea. El sistema desarrollado en esta Tesis evalúa la información sensorial, previamente transformada en percepciones, mediante la ‘Teoría de la Sorpresa’ y genera eventos llamados ‘sorpresas’ que sirven para predecir la evolución de una tarea. Por último, el sistema anticipativo de control postural puede componer, si fuese necesario, las reacciones adecuadas mediante el uso de patrones de movimientos predefinidos llamados sinergias. Dichas reacciones pueden complementar o sustituir por completo la ejecución normal de una tarea. El funcionamiento del sistema anticipativo de control postural y de cada uno de sus componentes ha sido probado tanto por medio de simulaciones como por su aplicación en el robot humanoide TEO del grupo de investigación RoboticsLab en el Departamento de Ingeniería de Sistemas y Automática de la Universidad Carlos III de Madrid

Embodied Models and Neurorobotics

Neuroscience has become a very broad field indeed: each year around 30,000 researchers and students from around the ... We trace a path from neuron to cognition via computational neuroscience, but what is computational neuroscience

Adaptive control of compliant robots with Reservoir Computing

In modern society, robots are increasingly used to handle dangerous, repetitive and/or heavy tasks with high precision. Because of the nature of the tasks, either being dangerous, high precision or simply repetitive, robots are usually constructed with high torque motors and sturdy materials, that makes them dangerous for humans to handle. In a car-manufacturing company, for example, a large cage is placed around the robot’s workspace that prevents humans from entering its vicinity. In the last few decades, efforts have been made to improve human-robot interaction. Often the movement of robots is characterized as not being smooth and clearly dividable into sub-movements. This makes their movement rather unpredictable for humans. So, there exists an opportunity to improve the motion generation of robots to enhance human-robot interaction. One interesting research direction is that of imitation learning. Here, human motions are recorded and demonstrated to the robot. Although the robot is able to reproduce such movements, it cannot be generalized to other situations. Therefore, a dynamical system approach is proposed where the recorded motions are embedded into the dynamics of the system. Shaping these nonlinear dynamics, according to recorded motions, allows for dynamical system to generalize beyond demonstration. As a result, the robot can generate motions of other situations not included in the recorded human demonstrations. In this dissertation, a Reservoir Computing approach is used to create a dynamical system in which such demonstrations are embedded. Reservoir Computing systems are Recurrent Neural Network-based approaches that are efficiently trained by considering only the training of the readout connections and retaining all other connections of such a network unchanged given their initial randomly chosen values. Although they have been used to embed periodic motions before, they were extended to embed discrete motions, or both. This work describes how such a motion pattern-generating system is built, investigates the nature of the underlying dynamics and evaluates their robustness in the face of perturbations. Additionally, a dynamical system approach to obstacle avoidance is proposed that is based on vector fields in the presence of repellers. This technique can be used to extend the motion abilities of the robot without need for changing the trained Motion Pattern Generator (MPG). Therefore, this approach can be applied in real-time on any system that generates a certain movement trajectory. Assume that the MPG system is implemented on an industrial robotic arm, similar to the ones used in a car factory. Even though the obstacle avoidance strategy presented is able to modify the generated motion of the robot’s gripper in such a way that it avoids obstacles, it does not guarantee that other parts of the robot cannot collide with a human. To prevent this, engineers have started to use advanced control algorithms that measure the amount of torque that is applied on the robot. This allows the robot to be aware of external perturbations. However, it turns out that, even with fast control loops, the adaptation to compensate for a sudden perturbation, is too slow to prevent high interaction forces. To reduce such forces, researchers started to use mechanical elements that are passively compliant (e.g., springs) and light-weight flexible materials to construct robots. Although such compliant robots are much safer and inherently energy efficient to use, their control becomes much harder. Most control approaches use model information about the robot (e.g., weight distribution and shape). However, when constructing a compliant robot it is hard to determine the dynamics of these materials. Therefore, a model-free adaptive control framework is proposed that assumes no prior knowledge about the robot. By interacting with the robot it learns an inverse robot model that is used as controller. The more it interacts, the better the control be- comes. Appropriately, this framework is called Inverse Modeling Adaptive (IMA) control framework. I have evaluated the IMA controller’s tracking ability on sev- eral tasks, investigating its model independence and stability. Furthermore, I have shown its fast learning ability and comparable performance to taskspecific designed controllers. Given both the MPG and IMA controllers, it is possible to improve the inter- actability of a compliant robot in a human-friendly environment. When the robot is to perform human-like motions for a large set of tasks, we need to demonstrate motion examples of all these tasks. However, biological research concerning the motion generation of animals and humans revealed that a limited set of motion patterns, called motion primitives, are modulated and combined to generate advanced motor/motion skills that humans and animals exhibit. Inspired by these interesting findings, I investigate if a single motion primitive indeed can be modulated to achieve a desired motion behavior. By some elementary experiments, where an MPG is controlled by an IMA controller, a proof of concept is presented. Furthermore, a general hierarchy is introduced that describes how a robot can be controlled in a biology-inspired manner. I also investigated how motion primitives can be combined to produce a desired motion. However, I was unable to get more advanced implementations to work. The results of some simple experiments are presented in the appendix. Another approach I investigated assumes that the primitives themselves are undefined. Instead, only a high-level description is given, which describes that every primitive on average should contribute equally, while still allowing for a single primitive to specialize in a part of the motion generation. Without defining the behavior of a primitive, only a set of untrained IMA controllers is used of which each will represent a single primitive. As a result of the high-level heuristic description, the task space is tiled into sub-regions in an unsupervised manner. Resulting in controllers that indeed represent a part of the motion generation. I have applied this Modular Architecture with Control Primitives (MACOP) on an inverse kinematic learning task and investigated the emerged primitives. Thanks to the tiling of the task space, it becomes possible to control redundant systems, because redundant solutions can be spread over several control primitives. Within each sub region of the task space, a specific control primitive is more accurate than in other regions allowing for the task complexity to be distributed over several less complex tasks. Finally, I extend the use of an IMA-controller, which is tracking controller, to the control of under-actuated systems. By using a sample-based planning algorithm it becomes possible to explore the system dynamics in which a path to a desired state can be planned. Afterwards, MACOP is used to incorporate feedback and to learn the necessary control commands corresponding to the planned state space trajectory, even if it contains errors. As a result, the under-actuated control of a cart pole system was achieved. Furthermore, I presented the concept of a simulation based control framework that allows the learning of the system dynamics, planning and feedback control iteratively and simultaneously

Models for reinforcement learning and design of a soft robot inspired by Drosophila larvae

Designs for robots are often inspired by animals, as they are designed mimicking animals’ mechanics, motions, behaviours and learning. The Drosophila, known as the fruit fly, is a well-studied model animal. In this thesis, the Drosophila larva is studied and the results are applied to robots. More specifically: a part of the Drosophila larva’s neural circuit for operant learning is modelled, based on which a synaptic plasticity model and a neural circuit model for operant learning, as well as a dynamic neural network for robot reinforcement learning, are developed; then Drosophila larva’s motor system for locomotion is studied, and based on it a soft robot system is designed. Operant learning is a concept similar to reinforcement learning in computer science, i.e. learning by reward or punishment for behaviour. Experiments have shown that a wide range of animals is capable of operant learning, including animal with only a few neurons, such as Drosophila. The fact implies that operant learning can establish without a large number of neurons. With it as an assumption, the structure and dynamics of synapses are investigated, and a synaptic plasticity model is proposed. The model includes nonlinear dynamics of synapses, especially receptor trafficking which affects synaptic strength. Tests of this model show it can enable operant learning at the neuron level and apply to a broad range of NNs, including feedforward, recurrent and spiking NNs. The mushroom body is a learning centre of the insect brain known and modelled for associative learning, but not yet for operant learning. To investigate whether it participates in operant learning, Drosophila larvae are studied with a transgenic tool by my collaborators. Based on the experiment and the results, a mushroom body model capable of operant learning is modelled. The proposed neural circuit model can reproduce the operant learning of the turning behaviour of Drosophila larvae. Then the synaptic plasticity model is simplified for robot learning. With the simplified model, a recurrent neural network with internal neural dynamics can learn to control a planar bipedal robot in a benchmark reinforcement learning task which is called bipedal walker by OpenAI. Benefiting efficiency in parameter space exploration instead of action space exploration, it is the first known solution to the task with reinforcement learning approaches. Although existing pneumatic soft robots can have multiple muscles embedded in a component, it is far less than the muscles in the Drosophila larva, which are well-organised in a tiny space. A soft robot system is developed based on the muscle pattern of the Drosophila larva, to explore the possibility to embed a high density of muscles in a limited space. Three versions of the body wall with pneumatic muscles mimicking the muscle pattern are designed. A pneumatic control system and embedded control system are also developed for controlling the robot. With a bioinspired body wall will a large number of muscles, the robot performs lifelike motions in experiments