Real-time biped character stepping

PhD ThesisA rudimentary biped activity that is essential in interactive evirtual worlds, such as video-games and training simulations, is stepping. For example, stepping is fundamental in everyday terrestrial activities that include walking and balance recovery. Therefore an effective 3D stepping control algorithm that is computationally fast and easy to implement is extremely valuable and important to character animation research. This thesis focuses on generating real-time controllable stepping motions on-the-fly without key-framed data that are responsive and robust (e.g.,can remain upright and balanced under a variety of conditions, such as pushes and dynami- cally changing terrain). In our approach, we control the character’s direction and speed by means of varying the stepposition and duration. Our lightweight stepping model is used to create coordinated full-body motions, which produce directable steps to guide the character with specific goals (e.g., following a particular path while placing feet at viable locations). We also create protective steps in response to random disturbances (e.g., pushes). Whereby, the system automatically calculates where and when to place the foot to remedy the disruption. In conclusion, the inverted pendulum has a number of limitations that we address and resolve to produce an improved lightweight technique that provides better control and stability using approximate feature enhancements, for instance, ankle-torque and elongated-body

Push Recovery for Humanoid Robot in Dynamic Environment and Classifying the Data Using K-Mean

Push recovery is prime ability that is essential to be incorporated in the process of developing a robust humanoid robot to support bipedalism. In real environment it is very essential for humanoid robot to maintain balance. In this paper we are generating a control system and push recovery controller for humanoid robot walking. We apply different kind of pushes to humanoid robot and the algorithm that can bring a change in the walking stage to sustain walking. The simulation is done in 3D environment using Webots. This paper describes techniques for feature selection to foreshow push recovery for hip, ankle and knee joint. We train the system by K-Mean algorithm and testing is done on crouch data and tested results are reported. Random push data of humanoid robot is collected and classified to see whether push lie in safer region and then tested on given proposed system

Understanding motor control in humans to improve rehabilitation robots

Recent reviews highlighted the limited results of robotic rehabilitation and the low quality of evidences in this field. Despite the worldwide presence of several robotic infrastructures, there is still a lack of knowledge about the capabilities of robotic training effect on the neural control of movement. To fill this gap, a step back to motor neuroscience is needed: the understanding how the brain works in the generation of movements, how it adapts to changes and how it acquires new motor skills is fundamental. This is the rationale behind my PhD project and the contents of this thesis: all the studies included in fact examined changes in motor control due to different destabilizing conditions, ranging from external perturbations, to self-generated disturbances, to pathological conditions. Data on healthy and impaired adults have been collected and quantitative and objective information about kinematics, dynamics, performance and learning were obtained for the investigation of motor control and skill learning. Results on subjects with cervical dystonia show how important assessment is: possibly adequate treatments are missing because the physiological and pathological mechanisms underlying sensorimotor control are not routinely addressed in clinical practice. These results showed how sensory function is crucial for motor control. The relevance of proprioception in motor control and learning is evident also in a second study. This study, performed on healthy subjects, showed that stiffness control is associated with worse robustness to external perturbations and worse learning, which can be attributed to the lower sensitiveness while moving or co-activating. On the other hand, we found that the combination of higher reliance on proprioception with \u201cdisturbance training\u201d is able to lead to a better learning and better robustness. This is in line with recent findings showing that variability may facilitate learning and thus can be exploited for sensorimotor recovery. Based on these results, in a third study, we asked participants to use the more robust and efficient strategy in order to investigate the control policies used to reject disturbances. We found that control is non-linear and we associated this non-linearity with intermittent control. As the name says, intermittent control is characterized by open loop intervals, in which movements are not actively controlled. We exploited the intermittent control paradigm for other two modeling studies. In these studies we have shown how robust is this model, evaluating it in two complex situations, the coordination of two joints for postural balance and the coordination of two different balancing tasks. It is an intriguing issue, to be addressed in future studies, to consider how learning affects intermittency and how this can be exploited to enhance learning or recovery. The approach, that can exploit the results of this thesis, is the computational neurorehabilitation, which mathematically models the mechanisms underlying the rehabilitation process, with the aim of optimizing the individual treatment of patients. Integrating models of sensorimotor control during robotic neurorehabilitation, might lead to robots that are fully adaptable to the level of impairment of the patient and able to change their behavior accordingly to the patient\u2019s intention. This is one of the goals for the development of rehabilitation robotics and in particular of Wristbot, our robot for wrist rehabilitation: combining proper assessment and training protocols, based on motor control paradigms, will maximize robotic rehabilitation effects

Adaptive control of compliant robots with Reservoir Computing

In modern society, robots are increasingly used to handle dangerous, repetitive and/or heavy tasks with high precision. Because of the nature of the tasks, either being dangerous, high precision or simply repetitive, robots are usually constructed with high torque motors and sturdy materials, that makes them dangerous for humans to handle. In a car-manufacturing company, for example, a large cage is placed around the robot’s workspace that prevents humans from entering its vicinity. In the last few decades, efforts have been made to improve human-robot interaction. Often the movement of robots is characterized as not being smooth and clearly dividable into sub-movements. This makes their movement rather unpredictable for humans. So, there exists an opportunity to improve the motion generation of robots to enhance human-robot interaction. One interesting research direction is that of imitation learning. Here, human motions are recorded and demonstrated to the robot. Although the robot is able to reproduce such movements, it cannot be generalized to other situations. Therefore, a dynamical system approach is proposed where the recorded motions are embedded into the dynamics of the system. Shaping these nonlinear dynamics, according to recorded motions, allows for dynamical system to generalize beyond demonstration. As a result, the robot can generate motions of other situations not included in the recorded human demonstrations. In this dissertation, a Reservoir Computing approach is used to create a dynamical system in which such demonstrations are embedded. Reservoir Computing systems are Recurrent Neural Network-based approaches that are efficiently trained by considering only the training of the readout connections and retaining all other connections of such a network unchanged given their initial randomly chosen values. Although they have been used to embed periodic motions before, they were extended to embed discrete motions, or both. This work describes how such a motion pattern-generating system is built, investigates the nature of the underlying dynamics and evaluates their robustness in the face of perturbations. Additionally, a dynamical system approach to obstacle avoidance is proposed that is based on vector fields in the presence of repellers. This technique can be used to extend the motion abilities of the robot without need for changing the trained Motion Pattern Generator (MPG). Therefore, this approach can be applied in real-time on any system that generates a certain movement trajectory. Assume that the MPG system is implemented on an industrial robotic arm, similar to the ones used in a car factory. Even though the obstacle avoidance strategy presented is able to modify the generated motion of the robot’s gripper in such a way that it avoids obstacles, it does not guarantee that other parts of the robot cannot collide with a human. To prevent this, engineers have started to use advanced control algorithms that measure the amount of torque that is applied on the robot. This allows the robot to be aware of external perturbations. However, it turns out that, even with fast control loops, the adaptation to compensate for a sudden perturbation, is too slow to prevent high interaction forces. To reduce such forces, researchers started to use mechanical elements that are passively compliant (e.g., springs) and light-weight flexible materials to construct robots. Although such compliant robots are much safer and inherently energy efficient to use, their control becomes much harder. Most control approaches use model information about the robot (e.g., weight distribution and shape). However, when constructing a compliant robot it is hard to determine the dynamics of these materials. Therefore, a model-free adaptive control framework is proposed that assumes no prior knowledge about the robot. By interacting with the robot it learns an inverse robot model that is used as controller. The more it interacts, the better the control be- comes. Appropriately, this framework is called Inverse Modeling Adaptive (IMA) control framework. I have evaluated the IMA controller’s tracking ability on sev- eral tasks, investigating its model independence and stability. Furthermore, I have shown its fast learning ability and comparable performance to taskspecific designed controllers. Given both the MPG and IMA controllers, it is possible to improve the inter- actability of a compliant robot in a human-friendly environment. When the robot is to perform human-like motions for a large set of tasks, we need to demonstrate motion examples of all these tasks. However, biological research concerning the motion generation of animals and humans revealed that a limited set of motion patterns, called motion primitives, are modulated and combined to generate advanced motor/motion skills that humans and animals exhibit. Inspired by these interesting findings, I investigate if a single motion primitive indeed can be modulated to achieve a desired motion behavior. By some elementary experiments, where an MPG is controlled by an IMA controller, a proof of concept is presented. Furthermore, a general hierarchy is introduced that describes how a robot can be controlled in a biology-inspired manner. I also investigated how motion primitives can be combined to produce a desired motion. However, I was unable to get more advanced implementations to work. The results of some simple experiments are presented in the appendix. Another approach I investigated assumes that the primitives themselves are undefined. Instead, only a high-level description is given, which describes that every primitive on average should contribute equally, while still allowing for a single primitive to specialize in a part of the motion generation. Without defining the behavior of a primitive, only a set of untrained IMA controllers is used of which each will represent a single primitive. As a result of the high-level heuristic description, the task space is tiled into sub-regions in an unsupervised manner. Resulting in controllers that indeed represent a part of the motion generation. I have applied this Modular Architecture with Control Primitives (MACOP) on an inverse kinematic learning task and investigated the emerged primitives. Thanks to the tiling of the task space, it becomes possible to control redundant systems, because redundant solutions can be spread over several control primitives. Within each sub region of the task space, a specific control primitive is more accurate than in other regions allowing for the task complexity to be distributed over several less complex tasks. Finally, I extend the use of an IMA-controller, which is tracking controller, to the control of under-actuated systems. By using a sample-based planning algorithm it becomes possible to explore the system dynamics in which a path to a desired state can be planned. Afterwards, MACOP is used to incorporate feedback and to learn the necessary control commands corresponding to the planned state space trajectory, even if it contains errors. As a result, the under-actuated control of a cart pole system was achieved. Furthermore, I presented the concept of a simulation based control framework that allows the learning of the system dynamics, planning and feedback control iteratively and simultaneously

Analytic and Learned Footstep Control for Robust Bipedal Walking

Bipedal walking is a complex, balance-critical whole-body motion with inherently unstable inverted pendulum-like dynamics. Strong disturbances must be quickly responded to by altering the walking motion and placing the next step in the right place at the right time. Unfortunately, the high number of degrees of freedom of the humanoid body makes the fast computation of well-placed steps a particularly challenging task. Sensor noise, imprecise actuation, and latency in the sensomotoric feedback loop impose further challenges when controlling real hardware. This dissertation addresses these challenges and describes a method of generating a robust walking motion for bipedal robots. Fast modification of footstep placement and timing allows agile control of the walking velocity and the absorption of strong disturbances. In a divide and conquer manner, the concepts of motion and balance are solved separately from each other, and consolidated in a way that a low-dimensional balance controller controls the timing and the footstep locations of a high-dimensional motion generator. Central pattern generated oscillatory motion signals are used for the synthesis of an open-loop stable walk on flat ground, which lacks the ability to respond to disturbances due to the absence of feedback. The Central Pattern Generator exhibits a low-dimensional parameter set to influence the timing and the landing coordinates of the swing foot. For balance control, a simple inverted pendulum-based physical model is used to represent the principal dynamics of walking. The model is robust to disturbances in a way that it returns to an ideal trajectory from a wide range of initial conditions by employing a combination of Zero Moment Point control, step timing, and foot placement strategies. The simulation of the model and its controller output are computed efficiently in closed form, supporting high-frequency balance control at the cost of an insignificant computational load. Additionally, the sagittal step size produced by the controller can be trained online during walking with a novel, gradient descent-based machine learning method. While the analytic controller forms the core of reliable walking, the trained sagittal step size complements the analytic controller in order to improve the overall walking performance. The balanced whole-body walking motion arises by using the footstep coordinates and the step timing predicted by the low-dimensional model as control input for the Central Pattern Generator. Real robot experiments are presented as evidence for disturbance-resistant, omnidirectional gait control, with arguably the strongest push-recovery capabilities to date

Learning-based methods for planning and control of humanoid robots

Nowadays, humans and robots are more and more likely to coexist as time goes by. The anthropomorphic nature of humanoid robots facilitates physical human-robot interaction, and makes social human-robot interaction more natural. Moreover, it makes humanoids ideal candidates for many applications related to tasks and environments designed for humans. No matter the application, an ubiquitous requirement for the humanoid is to possess proper locomotion skills. Despite long-lasting research, humanoid locomotion is still far from being a trivial task. A common approach to address humanoid locomotion consists in decomposing its complexity by means of a model-based hierarchical control architecture. To cope with computational constraints, simplified models for the humanoid are employed in some of the architectural layers. At the same time, the redundancy of the humanoid with respect to the locomotion task as well as the closeness of such a task to human locomotion suggest a data-driven approach to learn it directly from experience. This thesis investigates the application of learning-based techniques to planning and control of humanoid locomotion. In particular, both deep reinforcement learning and deep supervised learning are considered to address humanoid locomotion tasks in a crescendo of complexity. First, we employ deep reinforcement learning to study the spontaneous emergence of balancing and push recovery strategies for the humanoid, which represent essential prerequisites for more complex locomotion tasks. Then, by making use of motion capture data collected from human subjects, we employ deep supervised learning to shape the robot walking trajectories towards an improved human-likeness. The proposed approaches are validated on real and simulated humanoid robots. Specifically, on two versions of the iCub humanoid: iCub v2.7 and iCub v3

Interactive simulation of stylized human locomotion

Animating natural human motion in dynamic environments is difficult because of complex geometric and physical interactions. Simulation provides an automatic solution to parts of this problem, but it needs control systems to produce lifelike motions. This paper describes the systematic computation of controllers that can reproduce a range of locomotion styles in interactive simulations. Given a reference motion that describes the desired style, a derived control system can reproduce that style in simulation and in new environments. Because it produces high-quality motions that are both geometrically and physically consistent with simulated surroundings, interactive animation systems could begin to use this approach along with more established kinematic methods.Singapore-MIT GAMBIT Game LabNational Science Foundation (U.S.) (Fellowship 2007043041)Pixar (Firm

Motion Planning and Control for the Locomotion of Humanoid Robot

This thesis aims to contribute on the motion planning and control problem of the locomotion of humanoid robots. For the motion planning, various methods were proposed in different levels of model dependence. First, a model free approach was proposed which utilizes linear regression to estimate the relationship between foot placement and moving velocity. The data-based feature makes it quite robust to handle modeling error and external disturbance. As a generic control philosophy, it can be applied to various robots with different gaits. To reduce the risk of collecting experimental data of model-free method, based on the simplified linear inverted pendulum model, the classic planning method of model predictive control was explored to optimize CoM trajectory with predefined foot placements or optimize them two together with respect to the ZMP constraint. Along with elaborately designed re-planning algorithm and sparse discretization of trajectories, it is fast enough to run in real time and robust enough to resist external disturbance. Thereafter, nonlinear models are utilized for motion planning by performing forward simulation iteratively following the multiple shooting method. A walking pattern is predefined to fix most of the degrees of the robot, and only one decision variable, foot placement, is left in one motion plane and therefore able to be solved in milliseconds which is sufficient to run in real time. In order to track the planned trajectories and prevent the robot from falling over, diverse control strategies were proposed according to the types of joint actuators. CoM stabilizer was designed for the robots with position-controlled joints while quasi-static Cartesian impedance control and optimization-based full body torque control were implemented for the robots with torque-controlled joints. Various scenarios were set up to demonstrate the feasibility and robustness of the proposed approaches, like walking on uneven terrain, walking with narrow feet or straight leg, push recovery and so on