Learning and Adapting Agile Locomotion Skills by Transferring Experience

Legged robots have enormous potential in their range of capabilities, from navigating unstructured terrains to high-speed running. However, designing robust controllers for highly agile dynamic motions remains a substantial challenge for roboticists. Reinforcement learning (RL) offers a promising data-driven approach for automatically training such controllers. However, exploration in these high-dimensional, underactuated systems remains a significant hurdle for enabling legged robots to learn performant, naturalistic, and versatile agility skills. We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks. To leverage controllers we can acquire in practice, we design this framework to be flexible in terms of their source -- that is, the controllers may have been optimized for a different objective under different dynamics, or may require different knowledge of the surroundings -- and thus may be highly suboptimal for the target task. We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments. We also demonstrate that the agile behaviors learned in this way are graceful and safe enough to deploy in the real world.Comment: Project website: https://sites.google.com/berkeley.edu/twir

Joint friction estimation and slip prediction of biped walking robots

Friction is a nonlinear and complex phenomenon. It is unwanted at the biped joints since it deteriorates the robot’s walking performance in terms of speed and dynamic behavior. On the other hand, it is desired and required between the biped feet and the walking surface to facilitate locomotion. Further, friction forces between the feet and the ground determine the maximum acceleration and deceleration that the robot can afford without foot slip. Although several friction models are developed, there is no exact model that represents the friction behavior. This is why online friction estimation and compensation enter the picture. However, when online model-free estimation is difficult, a model-based method of online identification can prove useful. This thesis proposes a new approach for the joint friction estimation and slip prediction of walking biped robots. The joint friction estimation approach is based on the combination of a measurementbased strategy and a model-based method. The former is used to estimate the joint friction online when the foot is in contact with the ground, it utilizes the force and acceleration measurements in a reduced dynamical model of the biped. The latter adopts a friction model to represent the joint friction when the leg is swinging. The model parameters are identified adaptively using the estimated online friction whenever the foot is in contact. Then the estimated joint friction contributes to joint torque control signals to improve the control performance. The slip prediction is a model-free friction-behavior-inspired approach. A measurement-based online algorithm is designed to estimate the Coulomb friction which is regarded as a slip threshold. To predict the slip, a safety margin is introduced in the negative vicinity of the estimated Coulomb friction. The estimation algorithm concludes that if the applied force is outside the safety margin, then the foot tends to slip. The proposed estimation approaches are validated by experiments on SURALP (Sabanci University Robotics Research Laboratory Platform) and simulations on its model. The results demonstrate the effectiveness of these methods

Instantaneous Momentum-Based Control of Floating Base Systems

In the last two decades a growing number of robotic applications such as autonomous drones, wheeled robots and industrial manipulators started to be employed in several human environments. However, these machines often possess limited locomotion and/or manipulation capabilities, thus reducing the number of achievable tasks and increasing the complexity of robot-environment interaction. Augmenting robots locomotion and manipulation abilities is a fundamental research topic, with a view to enhance robots participation in complex tasks involving safe interaction and cooperation with humans. To this purpose, humanoid robots, aerial manipulators and the novel design of flying humanoid robots are among the most promising platforms researchers are studying in the attempt to remove the existing technological barriers. These robots are often modeled as floating base systems, and have lost the assumption -- typical of fixed base robots -- of having one link always attached to the ground. From the robot control side, contact forces regulation revealed to be fundamental for the execution of interaction tasks. Contact forces can be influenced by directly controlling the robot's momentum rate of change, and this fact gives rise to several momentum-based control strategies. Nevertheless, effective design of force and torque controllers still remains a complex challenge. The variability of sensor load during interaction, the inaccuracy of the force/torque sensing technology and the inherent nonlinearities of robot models are only a few complexities impairing efficient robot force control. This research project focuses on the design of balancing and flight controllers for floating base robots interacting with the surrounding environment. More specifically, the research is built upon the state-of-the-art of momentum-based controllers and applied to three robotic platforms: the humanoid robot iCub, the aerial manipulator OTHex and the jet-powered humanoid robot iRonCub. The project enforces the existing literature with both theoretical and experimental results, aimed at achieving high robot performances and improved stability and robustness, in presence of different physical robot-environment interactions

BIPED GAIT GENERATION FOR HUMANOID DYNAMIC WALKING

Ph.DDOCTOR OF PHILOSOPH

Modeling of human movement for the generation of humanoid robot motion

La robotique humanoïde arrive a maturité avec des robots plus rapides et plus précis. Pour faire face à la complexité mécanique, la recherche a commencé à regarder au-delà du cadre habituel de la robotique, vers les sciences de la vie, afin de mieux organiser le contrôle du mouvement. Cette thèse explore le lien entre mouvement humain et le contrôle des systèmes anthropomorphes tels que les robots humanoïdes. Tout d’abord, en utilisant des méthodes classiques de la robotique, telles que l’optimisation, nous étudions les principes qui sont à la base de mouvements répétitifs humains, tels que ceux effectués lorsqu’on joue au yoyo. Nous nous concentrons ensuite sur la locomotion en nous inspirant de résultats en neurosciences qui mettent en évidence le rôle de la tête dans la marche humaine. En développant une interface permettant à un utilisateur de commander la tête du robot, nous proposons une méthode de contrôle du mouvement corps-complet d’un robot humanoïde, incluant la production de pas et permettant au corps de suivre le mouvement de la tête. Cette idée est poursuivie dans l’étude finale dans laquelle nous analysons la locomotion de sujets humains, dirigée vers une cible, afin d’extraire des caractéristiques du mouvement sous forme invariants. En faisant le lien entre la notion “d’invariant” en neurosciences et celle de “tâche cinématique” en robotique humanoïde, nous développons une méthode pour produire une locomotion réaliste pour d’autres systèmes anthropomorphes. Dans ce cas, les résultats sont illustrés sur le robot humanoïde HRP2 du LAAS-CNRS. La contribution générale de cette thèse est de montrer que, bien que la planification de mouvement pour les robots humanoïdes peut être traitée par des méthodes classiques de robotique, la production de mouvements réalistes nécessite de combiner ces méthodes à l’observation systématique et formelle du comportement humain. ABSTRACT : Humanoid robotics is coming of age with faster and more agile robots. To compliment the physical complexity of humanoid robots, the robotics algorithms being developed to derive their motion have also become progressively complex. The work in this thesis spans across two research fields, human neuroscience and humanoid robotics, and brings some ideas from the former to aid the latter. By exploring the anthropological link between the structure of a human and that of a humanoid robot we aim to guide conventional robotics methods like local optimization and task-based inverse kinematics towards more realistic human-like solutions. First, we look at dynamic manipulation of human hand trajectories while playing with a yoyo. By recording human yoyo playing, we identify the control scheme used as well as a detailed dynamic model of the hand-yoyo system. Using optimization this model is then used to implement stable yoyo-playing within the kinematic and dynamic limits of the humanoid HRP-2. The thesis then extends its focus to human and humanoid locomotion. We take inspiration from human neuroscience research on the role of the head in human walking and implement a humanoid robotics analogy to this. By allowing a user to steer the head of a humanoid, we develop a control method to generate deliberative whole-body humanoid motion including stepping, purely as a consequence of the head movement. This idea of understanding locomotion as a consequence of reaching a goal is extended in the final study where we look at human motion in more detail. Here, we aim to draw to a link between “invariants” in neuroscience and “kinematic tasks” in humanoid robotics. We record and extract stereotypical characteristics of human movements during a walking and grasping task. These results are then normalized and generalized such that they can be regenerated for other anthropomorphic figures with different kinematic limits than that of humans. The final experiments show a generalized stack of tasks that can generate realistic walking and grasping motion for the humanoid HRP-2. The general contribution of this thesis is in showing that while motion planning for humanoid robots can be tackled by classical methods of robotics, the production of realistic movements necessitate the combination of these methods with the systematic and formal observation of human behavior

Unmute This: Circulation, Sociality, and Sound in Viral Media

Cats at keyboards. Dancing hamsters. Giggling babies and dancing flashmobs. A bi-colored dress. Psy’s “Gangnam Style” music video. Over the final decade of the twentieth century and the first decades of the twenty-first, these and countless other examples of digital audiovisual phenomena have been collectively adjectivally described through a biological metaphor that suggests the speed and ubiquity of their circulation—“viral.” This circulation has been facilitated by the internet, and has often been understood as a product of the web’s celebrated capacities for democratic amateur creation, its facilitation of unmediated connection and sharing practices. In this dissertation, I suggest that participation in such phenomena—the production, watching, listening to, circulation, or “sharing” of such objects—has constituted a significant site of twenty-first-century musical practice. Borrowing and adapting Christopher Small’s influential 1998 coinage, I theorize these strands of practice as viral musicking. While scholarship on viral media has tended to center on visual parameters, rendering such phenomena silent, the term “viral musicking” seeks to draw media theory metaphors of voice and listening into dialogue with musicology, precisely at the intersection of audiovisual objects which are played, heard, listened to. The project’s methodology comprises a sonically attuned media archeology, grounded in close readings of internet artifacts and practices; this sonic attunement is afforded through musicological methods, including analyses of genre, aesthetics, and style, discourse analysis, and twenty-first-century reception (micro)histories across a dynamic media assemblage. By analyzing particular ecosystems of platforms, behavior, and devices across the first decades of the twenty-first century, I chart a trajectory in which unpredictable virtual landscapes were tamed into entrenched channels and pathways, enabling a capacious “virality” comprising disparate phenomena from simple looping animations to the surprise release of Beyoncé’s 2013 album. Alongside this narrative, I challenge utopian claims of Web 2.0’s digital democratization by explicating the iterative processes through which material, work, and labor were co-opted from amateur content creators and leveraged for the profit of established media and corporate entities. “Unmute This” articulates two main arguments. First, that virality reified as a concept and set of dynamic-but-predictable processes over the course of the first decades of the twenty-first century; this dissertation charts a cartography of chaos to control, a heterogeneous digital landscape funneled into predictable channels and pathways etched ever more firmly and deeply across the 2010s. Second, that analyzing the musicality of viral objects, attending to the musical and sonic parameters of virally-circulating phenomena, and thinking of viral participation as an extension of musical behavior provide a productive framework for understanding the affective, generic, and social aspects of twenty-first-century virality. The five chapters of the dissertation present analyses of a series of viral objects, arranged roughly chronologically from the turn of the twenty-first century to the middle of the 2010s. The first chapter examines the loops of animated phenomena from The Dancing Baby to Hampster Dance and the Badgers animation; the second moves from loops to musicalization, considering remixing approaches to the so-called “Bus Uncle” and “Bed Intruder” videos. The third chapter also deals with viral remixing, centering around Rebecca Black’s “Friday” video, while the fourth chapter analyzes “unmute this” video posts in the context of the mid-2010s social media platform assemblage. The final chapter presents the 2013 surprise release of Beyoncé’s self-titled visual album as an apotheosis to the viral narratives that precede it—a claim that is briefly interrogated in the dissertation’s epilogue

Maritime expressions:a corpus based exploration of maritime metaphors

This study uses a purpose-built corpus to explore the linguistic legacy of Britain’s maritime history found in the form of hundreds of specialised ‘Maritime Expressions’ (MEs), such as TAKEN ABACK, ANCHOR and ALOOF, that permeate modern English. Selecting just those expressions commencing with ’A’, it analyses 61 MEs in detail and describes the processes by which these technical expressions, from a highly specialised occupational discourse community, have made their way into modern English. The Maritime Text Corpus (MTC) comprises 8.8 million words, encompassing a range of text types and registers, selected to provide a cross-section of ‘maritime’ writing. It is analysed using WordSmith analytical software (Scott, 2010), with the 100 million-word British National Corpus (BNC) as a reference corpus. Using the MTC, a list of keywords of specific salience within the maritime discourse has been compiled and, using frequency data, concordances and collocations, these MEs are described in detail and their use and form in the MTC and the BNC is compared. The study examines the transformation from ME to figurative use in the general discourse, in terms of form and metaphoricity. MEs are classified according to their metaphorical strength and their transference from maritime usage into new registers and domains such as those of business, politics, sports and reportage etc. A revised model of metaphoricity is developed and a new category of figurative expression, the ‘resonator’, is proposed. Additionally, developing the work of Lakov and Johnson, Kovesces and others on Conceptual Metaphor Theory (CMT), a number of Maritime Conceptual Metaphors are identified and their cultural significance is discussed