A Developmental Organization for Robot Behavior

This paper focuses on exploring how learning and development can be structured in synthetic (robot) systems. We present a developmental assembler for constructing reusable and temporally extended actions in a sequence. The discussion adopts the traditions of dynamic pattern theory in which behavior is an artifact of coupled dynamical systems with a number of controllable degrees of freedom. In our model, the events that delineate control decisions are derived from the pattern of (dis)equilibria on a working subset of sensorimotor policies. We show how this architecture can be used to accomplish sequential knowledge gathering and representation tasks and provide examples of the kind of developmental milestones that this approach has already produced in our lab

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment's transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots. Videos of the real-hand demonstrations can be found on the project website: https://sbrl.cs.columbia.edu/Comment: 10 pages, 6 figures, submitted to Robotics Science & Systems 202

Humanoid Mobile Manipulation Using Controller Refinement

An important class of mobile manipulation problems are move-to-grasp problems where a mobile robot must navigate to and pick up an object. One of the distinguishing features of this class of tasks is its coarse-to-fine structure. Near the beginning of the task, the robot can only sense the target object coarsely or indirectly and make gross motion toward the object. However, after the robot has located and approached the object, the robot must finely control its grasping contacts using precise visual and haptic feedback. In this paper, it is proposed that move-to-grasp problems are naturally solved by a sequence of controllers that iteratively refines what ultimately becomes the final solution. This paper introduces the notion of a refining sequence of controllers and characterizes this type of solution. The approach is demonstrated in a move-to-grasp task where Robonaut, the NASA/JSC dexterous humanoid, is mounted on a mobile base and navigates to and picks up a geological sample box. In a series of tests, it is shown that a refining sequence of controllers decreases variance in robot configuration relative to the sample box until a successful grasp has been achieved

Description of motor control using inverse models

Humans can perform complicated movements like writing or running without giving them much thought. The scientific understanding of principles guiding the generation of these movements is incomplete. How the nervous system ensures stability or compensates for injury and constraints – are among the unanswered questions today. Furthermore, only through movement can a human impose their will and interact with the world around them. Damage to a part of the motor control system can lower a person’s quality of life. Understanding how the central nervous system (CNS) forms control signals and executes them helps with the construction of devices and rehabilitation techniques. This allows the user, at least in part, to bypass the damaged area or replace its function, thereby improving their quality of life. CNS forms motor commands, for example a locomotor velocity or another movement task. These commands are thought to be processed through an internal model of the body to produce patterns of motor unit activity. An example of one such network in the spinal cord is a central pattern generator (CPG) that controls the rhythmic activation of synergistic muscle groups for overground locomotion. The descending drive from the brainstem and sensory feedback pathways initiate and modify the activity of the CPG. The interactions between its inputs and internal dynamics are still under debate in experimental and modelling studies. Even more complex neuromechanical mechanisms are responsible for some non-periodic voluntary movements. Most of the complexity stems from internalization of the body musculoskeletal (MS) system, which is comprised of hundreds of joints and muscles wrapping around each other in a sophisticated manner. Understanding their control signals requires a deep understanding of their dynamics and principles, both of which remain open problems. This dissertation is organized into three research chapters with a bottom-up investigation of motor control, plus an introduction and a discussion chapter. Each of the three research chapters are organized as stand-alone articles either published or in preparation for submission to peer-reviewed journals. Chapter two introduces a description of the MS kinematic variables of a human hand. In an effort to simulate human hand motor control, an algorithm was defined that approximated the moment arms and lengths of 33 musculotendon actuators spanning 18 degrees of freedom. The resulting model could be evaluated within 10 microseconds and required less than 100 KB of memory. The structure of the approximating functions embedded anatomical and functional features of the modelled muscles, providing a meaningful description of the system. The third chapter used the developments in musculotendon modelling to obtain muscle activity profiles controlling hand movements and postures. The agonist-antagonist coactivation mechanism was responsible for producing joint stability for most degrees of freedom, similar to experimental observations. Computed muscle excitations were used in an offline control of a myoelectric prosthesis for a single subject. To investigate the higher-order generation of control signals, the fourth chapter describes an analytical model of CPG. Its parameter space was investigated to produce forward locomotion when controlled with a desired speed. The model parameters were varied to produce asymmetric locomotion, and several control strategies were identified. Throughout the dissertation the balance between analytical, simulation, and phenomenological modelling for the description of simple and complex behavior is a recurrent theme of discussion

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion

Improving Soft Pneumatic Actuator Fingers through Integration of Soft Sensors, Position and Force Control, and Rigid Fingernails

We seek to overcome two key limitations which limit the abilities of Soft Pneumatic Actuators (SPAs) to grasp and manipulate objects: 1) Current SPAs lack position or force sensor feedback, which prevents controlling them precisely, and 2) the tip of the SPA is compliant and has high friction against common surfaces, causing the SPA to stick against surfaces when grasping objects from above. Our experiments suggest that we can achieve low steady-state error and overshoot in position and force using feed-forward models that relate pressure, force, and curvature along with a PID controller. We also compare several fingernail designs and show that the best-performing design significantly outperforms having no fingernails when grasping a set of common objects from a table