Locomotion training of legged robots using hybrid machine learning techniques

In this study artificial neural networks and fuzzy logic are used to control the jumping behavior of a three-link uniped robot. The biped locomotion control problem is an increment of the uniped locomotion control. Study of legged locomotion dynamics indicates that a hierarchical controller is required to control the behavior of a legged robot. A structured control strategy is suggested which includes navigator, motion planner, biped coordinator and uniped controllers. A three-link uniped robot simulation is developed to be used as the plant. Neurocontrollers were trained both online and offline. In the case of on-line training, a reinforcement learning technique was used to train the neurocontroller to make the robot jump to a specified height. After several hundred iterations of training, the plant output achieved an accuracy of 7.4%. However, when jump distance and body angular momentum were also included in the control objectives, training time became impractically long. In the case of off-line training, a three-layered backpropagation (BP) network was first used with three inputs, three outputs and 15 to 40 hidden nodes. Pre-generated data were presented to the network with a learning rate as low as 0.003 in order to reach convergence. The low learning rate required for convergence resulted in a very slow training process which took weeks to learn 460 examples. After training, performance of the neurocontroller was rather poor. Consequently, the BP network was replaced by a Cerebeller Model Articulation Controller (CMAC) network. Subsequent experiments described in this document show that the CMAC network is more suitable to the solution of uniped locomotion control problems in terms of both learning efficiency and performance. A new approach is introduced in this report, viz., a self-organizing multiagent cerebeller model for fuzzy-neural control of uniped locomotion is suggested to improve training efficiency. This is currently being evaluated for a possible patent by NASA, Johnson Space Center. An alternative modular approach is also developed which uses separate controllers for each stage of the running stride. A self-organizing fuzzy-neural controller controls the height, distance and angular momentum of the stride. A CMAC-based controller controls the movement of the leg from the time the foot leaves the ground to the time of landing. Because the leg joints are controlled at each time step during flight, movement is smooth and obstacles can be avoided. Initial results indicate that this approach can yield fast, accurate results

Adaptive Control of Arm Movement based on Cerebellar Model

This study is an attempt to take advantage of a cerebellar model to control a biomimetic arm. Aware that a variety of cerebellar models with different levels of details has been developed, we focused on a high-level model called MOSAIC. This model is thought to be able to describe the cerebellar functionality without getting into the details of the neural circuitry. To understand where this model exactly fits, we glanced over the biology of the cerebellum and a few alternative models. Certainly, the arm control loop is composed of other components. We reviewed those elements with emphasis on modeling for our simulation. Among these models, the arm and the muscle system received the most attention. The musculoskeletal model tested independently and by means of optimization techniques, a human-like control of arm through muscle activations achieved. We have discussed how MOSAIC can solve a control problem and what drawbacks it has. Consequently, toward making a practical use of MOSAIC model, several ideas developed and tested. In this process, we borrowed concepts and methods from the control theory. Specifically, known schemes of adaptive control of a manipulator, linearization and approximation were utilized. Our final experiment dealt with a modified/adjusted MOSAIC model to adaptively control the arm. We call this model ORF-MOSAIC (Organized by Receptive Fields MOdular Selection And Identification for Control). With as few as 16 modules, we were able to control the arm in a workspace of 30 x 30 cm. The system was able to adapt to an external field as well as handling new objects despite delays. The discussion section suggests that there are similarities between microzones in the cerebellum and the modules of this new model

Adaptive control of compliant robots with Reservoir Computing

In modern society, robots are increasingly used to handle dangerous, repetitive and/or heavy tasks with high precision. Because of the nature of the tasks, either being dangerous, high precision or simply repetitive, robots are usually constructed with high torque motors and sturdy materials, that makes them dangerous for humans to handle. In a car-manufacturing company, for example, a large cage is placed around the robot’s workspace that prevents humans from entering its vicinity. In the last few decades, efforts have been made to improve human-robot interaction. Often the movement of robots is characterized as not being smooth and clearly dividable into sub-movements. This makes their movement rather unpredictable for humans. So, there exists an opportunity to improve the motion generation of robots to enhance human-robot interaction. One interesting research direction is that of imitation learning. Here, human motions are recorded and demonstrated to the robot. Although the robot is able to reproduce such movements, it cannot be generalized to other situations. Therefore, a dynamical system approach is proposed where the recorded motions are embedded into the dynamics of the system. Shaping these nonlinear dynamics, according to recorded motions, allows for dynamical system to generalize beyond demonstration. As a result, the robot can generate motions of other situations not included in the recorded human demonstrations. In this dissertation, a Reservoir Computing approach is used to create a dynamical system in which such demonstrations are embedded. Reservoir Computing systems are Recurrent Neural Network-based approaches that are efficiently trained by considering only the training of the readout connections and retaining all other connections of such a network unchanged given their initial randomly chosen values. Although they have been used to embed periodic motions before, they were extended to embed discrete motions, or both. This work describes how such a motion pattern-generating system is built, investigates the nature of the underlying dynamics and evaluates their robustness in the face of perturbations. Additionally, a dynamical system approach to obstacle avoidance is proposed that is based on vector fields in the presence of repellers. This technique can be used to extend the motion abilities of the robot without need for changing the trained Motion Pattern Generator (MPG). Therefore, this approach can be applied in real-time on any system that generates a certain movement trajectory. Assume that the MPG system is implemented on an industrial robotic arm, similar to the ones used in a car factory. Even though the obstacle avoidance strategy presented is able to modify the generated motion of the robot’s gripper in such a way that it avoids obstacles, it does not guarantee that other parts of the robot cannot collide with a human. To prevent this, engineers have started to use advanced control algorithms that measure the amount of torque that is applied on the robot. This allows the robot to be aware of external perturbations. However, it turns out that, even with fast control loops, the adaptation to compensate for a sudden perturbation, is too slow to prevent high interaction forces. To reduce such forces, researchers started to use mechanical elements that are passively compliant (e.g., springs) and light-weight flexible materials to construct robots. Although such compliant robots are much safer and inherently energy efficient to use, their control becomes much harder. Most control approaches use model information about the robot (e.g., weight distribution and shape). However, when constructing a compliant robot it is hard to determine the dynamics of these materials. Therefore, a model-free adaptive control framework is proposed that assumes no prior knowledge about the robot. By interacting with the robot it learns an inverse robot model that is used as controller. The more it interacts, the better the control be- comes. Appropriately, this framework is called Inverse Modeling Adaptive (IMA) control framework. I have evaluated the IMA controller’s tracking ability on sev- eral tasks, investigating its model independence and stability. Furthermore, I have shown its fast learning ability and comparable performance to taskspecific designed controllers. Given both the MPG and IMA controllers, it is possible to improve the inter- actability of a compliant robot in a human-friendly environment. When the robot is to perform human-like motions for a large set of tasks, we need to demonstrate motion examples of all these tasks. However, biological research concerning the motion generation of animals and humans revealed that a limited set of motion patterns, called motion primitives, are modulated and combined to generate advanced motor/motion skills that humans and animals exhibit. Inspired by these interesting findings, I investigate if a single motion primitive indeed can be modulated to achieve a desired motion behavior. By some elementary experiments, where an MPG is controlled by an IMA controller, a proof of concept is presented. Furthermore, a general hierarchy is introduced that describes how a robot can be controlled in a biology-inspired manner. I also investigated how motion primitives can be combined to produce a desired motion. However, I was unable to get more advanced implementations to work. The results of some simple experiments are presented in the appendix. Another approach I investigated assumes that the primitives themselves are undefined. Instead, only a high-level description is given, which describes that every primitive on average should contribute equally, while still allowing for a single primitive to specialize in a part of the motion generation. Without defining the behavior of a primitive, only a set of untrained IMA controllers is used of which each will represent a single primitive. As a result of the high-level heuristic description, the task space is tiled into sub-regions in an unsupervised manner. Resulting in controllers that indeed represent a part of the motion generation. I have applied this Modular Architecture with Control Primitives (MACOP) on an inverse kinematic learning task and investigated the emerged primitives. Thanks to the tiling of the task space, it becomes possible to control redundant systems, because redundant solutions can be spread over several control primitives. Within each sub region of the task space, a specific control primitive is more accurate than in other regions allowing for the task complexity to be distributed over several less complex tasks. Finally, I extend the use of an IMA-controller, which is tracking controller, to the control of under-actuated systems. By using a sample-based planning algorithm it becomes possible to explore the system dynamics in which a path to a desired state can be planned. Afterwards, MACOP is used to incorporate feedback and to learn the necessary control commands corresponding to the planned state space trajectory, even if it contains errors. As a result, the under-actuated control of a cart pole system was achieved. Furthermore, I presented the concept of a simulation based control framework that allows the learning of the system dynamics, planning and feedback control iteratively and simultaneously

A Cognitive Model for Generalization during Sequential Learning

Traditional artificial neural network models of learning suffer from catastrophic interference. They are commonly trained to perform only one specific task, and, when trained on a new task, they forget the original task completely. It has been shown that the foundational neurocomputational principles embodied by the Leabra cognitive modeling framework, specifically fast lateral inhibition and a local synaptic plasticity model that incorporates both correlational and error-based components, are sufficient to largely overcome this limitation during the sequential learning of multiple motor skills. Evidence has also provided that Leabra is able to generalize the subsequences of motor skills, when doing so is appropriate. In this paper, we provide a detailed analysis of the extent of generalization possible with Leabra during sequential learning of multiple tasks. For comparison, we measure the generalization exhibited by the backpropagation of error learning algorithm. Furthermore, we demonstrate the applicability of sequential learning to a pair of movement tasks using a simulated robotic arm

Trajectory Generation for a Multibody Robotic System: Modern Methods Based on Product of Exponentials

This work presents several trajectory generation algorithms for multibody robotic systems based on the Product of Exponentials (PoE) formulation, also known as screw theory. A PoE formulation is first developed to model the kinematics and dynamics of a multibody robotic manipulator (Sawyer Robot) with 7 revolute joints and an end-effector. In the first method, an Inverse Kinematics (IK) algorithm based on the Newton-Raphson iterative method is applied to generate constrained joint-space trajectories corresponding to straight-line and curvilinear motions of the end effector in Cartesian space with finite jerk. The second approach describes Constant Screw Axis (CSA) trajectories which are generated using Machine Learning (ML) and Artificial Neural Networks (ANNs) techniques. The CSA method smooths the trajectory in the Special Euclidean (SE(3)) space. In the third approach, a multi-objective Swarm Intelligence (SI) trajectory generation algorithm is developed, where the IK problem is tackled using a combined SI-PoE ML technique resulting in a joint trajectory that avoids obstacles in the workspace, and satisfies the finite jerk constraint on end-effector while minimizing the torque profiles. The final method is a different approach to solving the IK problem using the Deep Q-Learning (DQN) Reinforcement Learning (RL) algorithm which can generate different joint space trajectories given the Cartesian end-effector path. For all methods above, the Newton-Euler recursive algorithm is implemented to compute the inverse dynamics, which generates the joint torques profiles. The simulated torque profiles are experimentally validated by feeding the generated joint trajectories to the Sawyer robotic arm through the developed Robot Operating System (ROS) - Python environment in the Software Development Kit (SDK) mode. The developed algorithms can be used to generate various trajectories for robotic arms (e.g. spacecraft servicing missions)

Advances in Intelligent Robotics and Collaborative Automation

This book provides an overview of a series of advanced research lines in robotics as well as of design and development methodologies for intelligent robots and their intelligent components. It represents a selection of extended versions of the best papers presented at the Seventh IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications IDAACS 2013 that were related to these topics. Its contents integrate state of the art computational intelligence based techniques for automatic robot control to novel distributed sensing and data integration methodologies that can be applied to intelligent robotics and automation systems. The objective of the text was to provide an overview of some of the problems in the field of robotic systems and intelligent automation and the approaches and techniques that relevant research groups within this area are employing to try to solve them.The contributions of the different authors have been grouped into four main sections:• Robots• Control and Intelligence• Sensing• Collaborative automationThe chapters have been structured to provide an easy to follow introduction to the topics that are addressed, including the most relevant references, so that anyone interested in this field can get started in the area