Robot learning of everyday object manipulations via human demonstration

We deal with the problem of teaching a robot to manipulate everyday objects through human demonstration. We first design a task descriptor which encapsulates important elements of a task. The design originates from observations that manipulations involved in many everyday object tasks can be considered as a series of sequential rotations and translations, which we call manipulation primitives. We then propose a method that enables a robot to decompose a demonstrated task into sequential manipulation primitives and construct a task descriptor. We also show how to transfer a task descriptor learned from one object to similar objects. In the end, we argue that this framework is highly generic. Particularly, it can be used to construct a robot task database that serves as a manipulation knowledge base for a robot to succeed in manipulating everyday objects

Task-oriented planning for manipulating articulated mechanisms under model uncertainty

Abstract — Personal robots need to manipulate a variety of articulated mechanisms as part of day-to-day tasks. These tasks are often specific, goal-driven, and permit very little bootstrap time for learning the articulation type. In this work, we ad-dress the problem of purposefully manipulating an articulated object, with uncertainty in the type of articulation. To this end, we provide two primary contributions: first, an efficient planning algorithm that, given a set of candidate articulation models, is able to correctly identify the underlying model and simultaneously complete a task; and second, a representation for articulated objects called the Generalized Kinematic Graph (GK-Graph), that allows for modeling complex mechanisms whose articulation varies as a function of the state space. Fi-nally, we provide a practical method to auto-generate candidate articulation models from RGB-D data and present extensive results on the PR2 robot to demonstrate the utility of our representation and the efficiency of our planner. I

Entropy-based strategies for physical exploration of the environment's degrees of freedom

Abstract — Physical exploration refers to the challenge of autonomously discovering and learning how to manipulate the environment’s degrees of freedom (DOF)—by identifying promising points of interaction and pushing or pulling object parts to reveal DOF and their properties. Recent existing work focused on sub-problems like estimating DOF parameters from given data. Here, we address the integrated problem, focusing on the higher-level strategy to iteratively decide on the next exploration point before applying motion generation methods to execute the explorative action and data analysis methods to interpret the feedback. We propose to decide on exploration points based on the expected information gain, or change in entropy in the robot’s current belief (uncertain knowledge) about the DOF. To this end, we first define how we represent such a belief. This requires dealing with the fact that the robot initially does not know which random variables (which DOF, and depending on their type, which DOF properties) actually exist. We then propose methods to estimate the expected information gain for an exploratory action. We analyze these strategies in simple environments and evaluate them in combination with full motion planning and data analysis in a physical simulation environment. I

Physics-, social-, and capability- based reasoning for robotic manipulation

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 124-128).Robots that can function in human-centric domains have the potential to help humans with the chores of everyday life. Moreover, dexterous robots with the ability to reason about the maneuvers they execute for manipulation tasks can function more autonomously and intelligently. This thesis outlines the development of a reasoning architecture that uses physics-, social-, and agent capability-based knowledge to generate manipulation strategies that a dexterous robot can implement in the physical world. The reasoning system learns object affordances through a combination of observations from human interactions, explicit rules and constraints imposed on the system, and hardcoded physics-based logic. Observations from humans performing manipulation tasks are also used to develop a unique manipulation repertoire suitable for the robot. The system then uses Bayesian Networks to probabilistically determine the best manipulation strategies for the robot to execute on new objects. The robot leverages this knowledge during experimental trials where manipulation strategies suggested by the reasoning architecture are shown to perform well in new manipulation environments.by Kenton J. Williams.S.M

Learning relational models with human interaction for planning in robotics

Automated planning has proven to be useful to solve problems where an agent has to maximize a reward function by executing actions. As planners have been improved to salve more expressive and difficult problems, there is an increasing interest in using planning to improve efficiency in robotic tasks. However, planners rely on a domain model, which has to be either handcrafted or learned. Although learning domain models can be very costly, recent approaches provide generalization capabilities and integrate human feedback to reduce the amount of experiences required to learn. In this thesis we propase new methods that allow an agent with no previous knowledge to solve certain problems more efficiently by using task planning. First, we show how to apply probabilistic planning to improve robot performance in manipulation tasks (such as cleaning the dirt or clearing the tableware on a table). Planners obtain sequences of actions that get the best result in the long term, beating reactive strategies. Second, we introduce new reinforcement learning algorithms where the agent can actively request demonstrations from a teacher to learn new actions and speed up the learning process. In particular, we propase an algorithm that allows the user to set the mínimum quality to be achieved, where a better quality also implies that a larger number of demonstrations will be requested . Moreover, the learned model is analyzed to extract the unlearned or problematic parts of the model. This information allow the agent to provide guidance to the teacher when a demonstration is requested, and to avoid irrecoverable errors. Finally, a new domain model learner is introduced that, in addition to relational probabilistic action models, can also learn exogenous effects. This learner can be integrated with existing planners and reinforcement learning algorithms to salve a wide range of problems. In summary, we improve the use of learning and task planning to salve unknown tasks. The improvements allow an agent to obtain a larger benefit from planners, learn faster, balance the number of action executions and teacher demonstrations, avoid irrecoverable errors, interact with a teacher to solve difficult problems, and adapt to the behavior of other agents by learning their dynamics. All the proposed methods were compared with state-of-the-art approaches, and were also demonstrated in different scenarios, including challenging robotic tasks.La planificación automática ha probado ser de gran utilidad para resolver problemas en los que un agente tiene que ejecutar acciones para maximizar una función de recompensa. A medida que los planificadores han sido capaces de resolver problemas cada vez más complejos, ha habido un creciente interés por utilizar dichos planificadores para mejorar la eficiencia de tareas robóticas. Sin embargo, los planificadores requieren un modelo del dominio, el cual puede ser creado a mano o aprendido. Aunque aprender modelos automáticamente puede ser costoso, recientemente han aparecido métodos que permiten la interacción persona-máquina y generalizan el conocimiento para reducir la cantidad de experiencias requeridas para aprender. En esta tesis proponemos nuevos métodos que permiten a un agente sin conocimiento previo de la tarea resolver problemas de forma más eficiente mediante el uso de planificación automática. Comenzaremos mostrando cómo aplicar planificación probabilística para mejorar la eficiencia de robots en tareas de manipulación (como limpiar suciedad o recoger una mesa). Los planificadores son capaces de obtener las secuencias de acciones que producen los mejores resultados a largo plazo, superando a las estrategias reactivas. Por otro lado, presentamos nuevos algoritmos de aprendizaje por refuerzo en los que el agente puede solicitar demostraciones a un profesor. Dichas demostraciones permiten al agente acelerar el aprendizaje o aprender nuevas acciones. En particular, proponemos un algoritmo que permite al usuario establecer la mínima suma de recompensas que es aceptable obtener, donde una recompensa más alta implica que se requerirán más demostraciones. Además, el modelo aprendido será analizado para identificar qué partes están incompletas o son problemáticas. Esta información permitirá al agente evitar errores irrecuperables y también guiar al profesor cuando se solicite una demostración. Finalmente, se ha introducido un nuevo método de aprendizaje para modelos de dominios que, además de obtener modelos relacionales de acciones probabilísticas, también puede aprender efectos exógenos. Mostraremos cómo integrar este método en algoritmos de aprendizaje por refuerzo para poder abordar una mayor cantidad de problemas. En resumen, hemos mejorado el uso de técnicas de aprendizaje y planificación para resolver tareas desconocidas a priori. Estas mejoras permiten a un agente aprovechar mejor los planificadores, aprender más rápido, elegir entre reducir el número de acciones ejecutadas o el número de demostraciones solicitadas, evitar errores irrecuperables, interactuar con un profesor para resolver problemas complejos, y adaptarse al comportamiento de otros agentes aprendiendo sus dinámicas. Todos los métodos propuestos han sido comparados con trabajos del estado del arte, y han sido evaluados en distintos escenarios, incluyendo tareas robóticas

Golf ball picker robot: path generation in unstructured environments towards multiple targets

Tese de doutoramento Engineering Design and Advanced Manufacturing Leaders for Technical IndustriesThe new TWIN-RRT* algorithm solves a motion planning problem in which an agent has multiple possible targets where none of them is compulsory, and retrieves feasible, ―low cost‖, asymptotically optimal and probabilistically complete paths. The TWIN-RRT* algorithm solves path planning problems for both holonomic and non-holonomic robots with or without kinodynamic constraints in a 2D environment, but it was designed to work as well with higher DOF agents and different applications. The new algorithm provides a practical implementation of feasible and fast planning especially where a closed loop is required. Initial and final configurations are allowed to be exactly the same. The TWIN-RRT* algorithm computes an efficient path for one sole agent towards multiple targets where none of them is mandatory. It inherits the low computational cost, probabilistic completeness and asymptotical optimality from RRT*. It uses efficiency as cost function, which can be adapted depending on the application. The TWIN-RRT* complies both with kinodynamic constraints and different cost functions. It was developed to solve a real problem where a robot has to collect golf balls in a driving range, where thousands of balls accumulate every day. This thesis is part of a bigger project, Golfminho, to develop an autonomous robot capable of efficiently collecting balls in a golf practice field.O novo algoritmo TWIN-RRT* resolve problemas de planeamento de trajetórias em que um agente tem múltiplos alvos, onde nenhum deles é obrigatório, e produz um plano exequível, de "baixo custo" computacional, assintoticamente ótimo e probabilisticamente completo. O TWINRRT* resolve problemas de planeamento de trajetórias tanto para robôs holonómicos como não holonómicos com ou sem restrições cinemáticas e/ou dinâmicas num ambiente 2D, mas foi projetado para funcionar também com agentes com maiores graus de liberdade e em diferentes aplicações. O novo algoritmo fornece uma implementação prática de um planeamento viável e rápido, especialmente quando é necessário produzir uma trajetória fechada. As configurações iniciais e finais podem ser exatamente iguais. O algoritmo TWIN-RRT* calcula um caminho eficiente para um agente único em direção a múltiplos alvos, onde nenhum deles é obrigatório. Herda o baixo custo computacional, integralidade probabilística e otimização assintótica do RRT*. Usa a eficiência como função de custo, que pode ser adaptada em função das diferentes aplicações. Para além de diferentes funções de custo, o TWIN-RRT* também mostra conformidade com restrições cinemáticas. Foi desenvolvido para resolver um problema real em que um robô tem que recolher bolas de golfe num Driving Range, onde se acumulam milhares de bolas de golfe por dia. Esta tese é parte integrante do projeto Golfminho, para o desenvolvimento de um robô autónomo capaz de recolher eficientemente bolas num campo de práticas de golfe.Fundação para a Ciência e Tecnologia (FCT) for the PhD grant nº. SFRH/BD/43008/2008