Adaptive quadruped locomotion: learning to detect and avoid an obstacle

Dissertação de mestrado em Engenharia de InformáticaAutonomy and adaptability are key features in the design and construction of a robotic system capable of carrying out tasks in an unstructured and not predefined environment. Such features are generally observed in animals, biological systems that usually serve as an inspiration models to the design of robotic systems. The autonomy and adaptability of these biological systems partially arises from their ability to learn. Animals learn to move and control their own body when young, they learn to survive, to hunt and avoid undesirable situations, from their progenitors. There has been an increasing interest in defining a way to endow these abilities into the design and creation of robotic systems. This dissertation proposes a mechanism that seeks to create a learning module to a quadruped robot controller that enables it to both, detect and avoid an obstacle in its path. The detection is based on a Forward Internal Model (FIM) trained online to create expectations about the robot’s perceptive information. This information is acquired by a set of range sensors that scan the ground in front of the robot in order to detect the obstacle. In order to avoid stepping on the obstacle, the obstacle detections are used to create a map of responses that will change the locomotion according to what is necessary. The map is built and tuned every time the robot fails to step over the obstacle and defines how the robot should act to avoid these situations in the future. Both learning tasks are carried out online and kept active after the robot has learned, enabling the robot to adapt to possible new situations. The proposed architecture was inspired on [14, 17], but applied here to a quadruped robot with different sensors and specific sensor configuration. Also, the mechanism is coupled with the robot’s locomotion generator based in Central Pattern Generators (CPG)s presented in [22]. In order to achieve its goal, the controller send commands to the CPG so that the necessary changes in the locomotion are applied. Results showed the success in both learning tasks. The robot was able to detect the obstacle, and change its locomotion with the acquired information at collision time.Autonomia e capacidade de adaptação são características chave na criação de sistemas robóticos capazes de levar a cabo diversas tarefas em ambientes não especificamente preparados nem configurados para tal. Estas características são comuns nos animais, sistemas biológicos que muitas vezes servem de modelo e inspiração para desenhar e construir sistemas robóticos. A autonomia e adaptabilidade destes sistemas advém parcialmente da sua capacidade de aprender. Quando ainda jovens, os animais aprendem a controlar o seu corpo e a movimentar-se, muitos mamíferos aprendem a caçar e a sobreviver com os seus progenitores. Ultimamente tem havido um crescente interesse em como aplicar estas características no desenho e criação de sistemas robóticos. Nesta dissertação é proposto um mecanismo que permita que um robô quadrúpede seja capaz de detectar e evitar um obstáculo no seu caminho. A detecção é baseada num Forward Internal Model (FIM) que aprende a prever os valores dos sensores de percepção do robô, os quais procuram detectar obstáculos no seu caminho. Por forma a evitar os obstáculos, os sinais de detecção dos obstáculos são usados na criação de um mapa que permitirá ao robô alterar a sua locomoção mediante o que é necessário. Este mapa é construído à medida que o robô falha e tropeça no obstáculo. Nesse momento, o mapa é definido para que tal situação seja evitada no futuro. Ambos os processos de aprendizagem são levados a cabo em tempo de execução e mantêm esse processo activo por forma a possibilitar a readaptação a eventuais novas situações. Este mecanismo foi inspirado nos trabalhos [14, 17], mas aplicados aqui a um quadrúpede com uma configuração de sensores diferente e específica. O mecanismo será interligado ao gerador da locomoção, baseado em Control Pattern Generator (CPG) proposto em [22]. Por forma a atingir o objectivo final, o controlador irá enviar comandos para o CPG a fim da locomoção ser alterada como necessário. Os resultados obtidos mostram o sucesso em ambos os processos de aprendizagem. O robô é capaz de detectar o obstáculo e alterar a sua locomção de acordo com a informação adquirida nos momentos de colisão com o mesmo, conseguindo efectivamente passar por cima do obstáculo sem nenhum tipo de colisão

Adaptive cancelation of self-generated sensory signals in a whisking robot

Sensory signals are often caused by one's own active movements. This raises a problem of discriminating between self-generated sensory signals and signals generated by the external world. Such discrimination is of general importance for robotic systems, where operational robustness is dependent on the correct interpretation of sensory signals. Here, we investigate this problem in the context of a whiskered robot. The whisker sensory signal comprises two components: one due to contact with an object (externally generated) and another due to active movement of the whisker (self-generated). We propose a solution to this discrimination problem based on adaptive noise cancelation, where the robot learns to predict the sensory consequences of its own movements using an adaptive filter. The filter inputs (copy of motor commands) are transformed by Laguerre functions instead of the often-used tapped-delay line, which reduces model order and, therefore, computational complexity. Results from a contact-detection task demonstrate that false positives are significantly reduced using the proposed scheme

Benchmarking Cerebellar Control

Cerebellar models have long been advocated as viable models for robot dynamics control. Building on an increasing insight in and knowledge of the biological cerebellum, many models have been greatly refined, of which some computational models have emerged with useful properties with respect to robot dynamics control. Looking at the application side, however, there is a totally different picture. Not only is there not one robot on the market which uses anything remotely connected with cerebellar control, but even in research labs most testbeds for cerebellar models are restricted to toy problems. Such applications hardly ever exceed the complexity of a 2 DoF simulated robot arm; a task which is hardly representative for the field of robotics, or relates to realistic applications. In order to bring the amalgamation of the two fields forwards, we advocate the use of a set of robotics benchmarks, on which existing and new computational cerebellar models can be comparatively tested. It is clear that the traditional approach to solve robotics dynamics loses ground with the advancing complexity of robotic structures; there is a desire for adaptive methods which can compete as traditional control methods do for traditional robots. In this paper we try to lay down the successes and problems in the fields of cerebellar modelling as well as robot dynamics control. By analyzing the common ground, a set of benchmarks is suggested which may serve as typical robot applications for cerebellar models

Behavior-specific proprioception models for robotic force estimation: a machine learning approach

Robots that support humans in physically demanding tasks require accurate force sensing capabilities. A common way to achieve this is by monitoring the interaction with the environment directly with dedicated force sensors. Major drawbacks of such special purpose sensors are the increased costs and the reduced payload of the robot platform. Instead, this thesis investigates how the functionality of such sensors can be approximated by utilizing force estimation approaches. Most of today’s robots are equipped with rich proprioceptive sensing capabilities where even a robotic arm, e.g., the UR5, provides access to more than hundred sensor readings. Following this trend, it is getting feasible to utilize a wide variety of sensors for force estimation purposes. Human proprioception allows estimating forces such as the weight of an object by prior experience about sensory-motor patterns. Applying a similar approach to robots enables them to learn from previous demonstrations without the need of dedicated force sensors. This thesis introduces Behavior-Specific Proprioception Models (BSPMs), a novel concept for enhancing robotic behavior with estimates of the expected proprioceptive feedback. A main methodological contribution is the operationalization of the BSPM approach using data-driven machine learning techniques. During a training phase, the behavior is continuously executed while recording proprioceptive sensor readings. The training data acquired from these demonstrations represents ground truth about behavior-specific sensory-motor experiences, i.e., the influence of performed actions and environmental conditions on the proprioceptive feedback. This data acquisition procedure does not require expert knowledge about the particular robot platform, e.g., kinematic chains or mass distribution, which is a major advantage over analytical approaches. The training data is then used to learn BSPMs, e.g. using lazy learning techniques or artificial neural networks. At runtime, the BSPMs provide estimates of the proprioceptive feedback that can be compared to actual sensations. The BSPM approach thus extends classical programming by demonstrations methods where only movement data is learned and enables robots to accurately estimate forces during behavior execution

Optimality, Objectives, and Trade-Offs in Motor Control under Uncertainty

Biological motor control involves multiple objectives and constraints. In this thesis, I investigated the influence of uncertainty on biological sensorimotor control and decision-making, considering various objectives. In the first study, I used a simple biped walking model simulation to study the control of a rhythmic movement under uncertainty. Uncertainty necessitates a more sophisticated form of motor control involving internal model and sensing, and their effective integration. The optimality of the neural pattern generator incorporating sensory information was shown to be dependent on the relative amount of physical disturbance and sensor noise. When the controller was optimized for state estimation, other objectives of improved energy efficiency, reduced variability, and reduced number of falls were also satisfied. In the second study, human participants performed regression and classification tasks on visually presented scatterplot data. The tasks involved a trade-off between acting on small but prevalent errors and acting on big but scarce errors. We used inverse optimization to characterize the loss function used by humans in these regression and classification tasks, and found that these loss functions change systematically as the data sparsity changed. Despite being highly variable, there were overall shifts towards compensating for prevalent small errors more when the sparsity of the visual data decreased. In the third study, I extended the pattern recognition tasks to include visually mediated force tracking. When participants tracked force targets with visual noise, we observed a slight yet consistent force tracking bias. This bias, which increased with noise, was not explained by commonly hypothesized objectives such as a tendency to reduce effort while regulating error. Additional experiments revealed that a model balancing error reduction and transition reduction tendencies effectively explained and predicted experimental data. Transition reduction tendency was further separated into recency bias and central tendency bias. Notably, this bias disappeared when the task became purely visual, suggesting that such biases could be task-dependent. These findings across the three studies provide useful insights into understanding how uncertainty changes objectives and their trade-offs in biological motor control, and in turn, results in a different control strategy and behaviors