Grasp plannind under task-specific contact constraints

Several aspects have to be addressed before realizing the dream of a robotic hand-arm system with human-like capabilities, ranging from the consolidation of a proper mechatronic design, to the development of precise, lightweight sensors and actuators, to the efficient planning and control of the articular forces and motions required for interaction with the environment. This thesis provides solution algorithms for a main problem within the latter aspect, known as the {\em grasp planning} problem: Given a robotic system formed by a multifinger hand attached to an arm, and an object to be grasped, both with a known geometry and location in 3-space, determine how the hand-arm system should be moved without colliding with itself or with the environment, in order to firmly grasp the object in a suitable way. Central to our algorithms is the explicit consideration of a given set of hand-object contact constraints to be satisfied in the final grasp configuration, imposed by the particular manipulation task to be performed with the object. This is a distinguishing feature from other grasp planning algorithms given in the literature, where a means of ensuring precise hand-object contact locations in the resulting grasp is usually not provided. These conventional algorithms are fast, and nicely suited for planning grasps for pick-an-place operations with the object, but not for planning grasps required for a specific manipulation of the object, like those necessary for holding a pen, a pair of scissors, or a jeweler's screwdriver, for instance, when writing, cutting a paper, or turning a screw, respectively. To be able to generate such highly-selective grasps, we assume that a number of surface regions on the hand are to be placed in contact with a number of corresponding regions on the object, and enforce the fulfilment of such constraints on the obtained solutions from the very beginning, in addition to the usual constraints of grasp restrainability, manipulability and collision avoidance. The proposed algorithms can be applied to robotic hands of arbitrary structure, possibly considering compliance in the joints and the contacts if desired, and they can accommodate general patch-patch contact constraints, instead of more restrictive contact types occasionally considered in the literature. It is worth noting, also, that while common force-closure or manipulability indices are used to asses the quality of grasps, no particular assumption is made on the mathematical properties of the quality index to be used, so that any quality criterion can be accommodated in principle. The algorithms have been tested and validated on numerous situations involving real mechanical hands and typical objects, and find applications in classical or emerging contexts like service robotics, telemedicine, space exploration, prosthetics, manipulation in hazardous environments, or human-robot interaction in general

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

On CAD Informed Adaptive Robotic Assembly

We introduce a robotic assembly system that streamlines the design-to-make workflow for going from a CAD model of a product assembly to a fully programmed and adaptive assembly process. Our system captures (in the CAD tool) the intent of the assembly process for a specific robotic workcell and generates a recipe of task-level instructions. By integrating visual sensing with deep-learned perception models, the robots infer the necessary actions to assemble the design from the generated recipe. The perception models are trained directly from simulation, allowing the system to identify various parts based on CAD information. We demonstrate the system with a workcell of two robots to assemble interlocking 3D part designs. We first build and tune the assembly process in simulation, verifying the generated recipe. Finally, the real robotic workcell assembles the design using the same behavior

Computational Methods for Cognitive and Cooperative Robotics

In the last decades design methods in control engineering made substantial progress in the areas of robotics and computer animation. Nowadays these methods incorporate the newest developments in machine learning and artificial intelligence. But the problems of flexible and online-adaptive combinations of motor behaviors remain challenging for human-like animations and for humanoid robotics. In this context, biologically-motivated methods for the analysis and re-synthesis of human motor programs provide new insights in and models for the anticipatory motion synthesis. This thesis presents the author’s achievements in the areas of cognitive and developmental robotics, cooperative and humanoid robotics and intelligent and machine learning methods in computer graphics. The first part of the thesis in the chapter “Goal-directed Imitation for Robots” considers imitation learning in cognitive and developmental robotics. The work presented here details the author’s progress in the development of hierarchical motion recognition and planning inspired by recent discoveries of the functions of mirror-neuron cortical circuits in primates. The overall architecture is capable of ‘learning for imitation’ and ‘learning by imitation’. The complete system includes a low-level real-time capable path planning subsystem for obstacle avoidance during arm reaching. The learning-based path planning subsystem is universal for all types of anthropomorphic robot arms, and is capable of knowledge transfer at the level of individual motor acts. Next, the problems of learning and synthesis of motor synergies, the spatial and spatio-temporal combinations of motor features in sequential multi-action behavior, and the problems of task-related action transitions are considered in the second part of the thesis “Kinematic Motion Synthesis for Computer Graphics and Robotics”. In this part, a new approach of modeling complex full-body human actions by mixtures of time-shift invariant motor primitives in presented. The online-capable full-body motion generation architecture based on dynamic movement primitives driving the time-shift invariant motor synergies was implemented as an online-reactive adaptive motion synthesis for computer graphics and robotics applications. The last chapter of the thesis entitled “Contraction Theory and Self-organized Scenarios in Computer Graphics and Robotics” is dedicated to optimal control strategies in multi-agent scenarios of large crowds of agents expressing highly nonlinear behaviors. This last part presents new mathematical tools for stability analysis and synthesis of multi-agent cooperative scenarios.In den letzten Jahrzehnten hat die Forschung in den Bereichen der Steuerung und Regelung komplexer Systeme erhebliche Fortschritte gemacht, insbesondere in den Bereichen Robotik und Computeranimation. Die Entwicklung solcher Systeme verwendet heutzutage neueste Methoden und Entwicklungen im Bereich des maschinellen Lernens und der künstlichen Intelligenz. Die flexible und echtzeitfähige Kombination von motorischen Verhaltensweisen ist eine wesentliche Herausforderung für die Generierung menschenähnlicher Animationen und in der humanoiden Robotik. In diesem Zusammenhang liefern biologisch motivierte Methoden zur Analyse und Resynthese menschlicher motorischer Programme neue Erkenntnisse und Modelle für die antizipatorische Bewegungssynthese. Diese Dissertation präsentiert die Ergebnisse der Arbeiten des Autors im Gebiet der kognitiven und Entwicklungsrobotik, kooperativer und humanoider Robotersysteme sowie intelligenter und maschineller Lernmethoden in der Computergrafik. Der erste Teil der Dissertation im Kapitel “Zielgerichtete Nachahmung für Roboter” behandelt das Imitationslernen in der kognitiven und Entwicklungsrobotik. Die vorgestellten Arbeiten beschreiben neue Methoden für die hierarchische Bewegungserkennung und -planung, die durch Erkenntnisse zur Funktion der kortikalen Spiegelneuronen-Schaltkreise bei Primaten inspiriert wurden. Die entwickelte Architektur ist in der Lage, ‘durch Imitation zu lernen’ und ‘zu lernen zu imitieren’. Das komplette entwickelte System enthält ein echtzeitfähiges Pfadplanungssubsystem zur Hindernisvermeidung während der Durchführung von Armbewegungen. Das lernbasierte Pfadplanungssubsystem ist universell und für alle Arten von anthropomorphen Roboterarmen in der Lage, Wissen auf der Ebene einzelner motorischer Handlungen zu übertragen. Im zweiten Teil der Arbeit “Kinematische Bewegungssynthese für Computergrafik und Robotik” werden die Probleme des Lernens und der Synthese motorischer Synergien, d.h. von räumlichen und räumlich-zeitlichen Kombinationen motorischer Bewegungselemente bei Bewegungssequenzen und bei aufgabenbezogenen Handlungs übergängen behandelt. Es wird ein neuer Ansatz zur Modellierung komplexer menschlicher Ganzkörperaktionen durch Mischungen von zeitverschiebungsinvarianten Motorprimitiven vorgestellt. Zudem wurde ein online-fähiger Synthesealgorithmus für Ganzköperbewegungen entwickelt, der auf dynamischen Bewegungsprimitiven basiert, die wiederum auf der Basis der gelernten verschiebungsinvarianten Primitive konstruiert werden. Dieser Algorithmus wurde für verschiedene Probleme der Bewegungssynthese für die Computergrafik- und Roboteranwendungen implementiert. Das letzte Kapitel der Dissertation mit dem Titel “Kontraktionstheorie und selbstorganisierte Szenarien in der Computergrafik und Robotik” widmet sich optimalen Kontrollstrategien in Multi-Agenten-Szenarien, wobei die Agenten durch eine hochgradig nichtlineare Kinematik gekennzeichnet sind. Dieser letzte Teil präsentiert neue mathematische Werkzeuge für die Stabilitätsanalyse und Synthese von kooperativen Multi-Agenten-Szenarien

Motion planning using synergies : application to anthropomorphic dual-arm robots

Motion planning is a traditional field in robotics, but new problems are nevertheless incessantly appearing, due to continuous advances in the robot developments. In order to solve these new problems, as well as to improve the existing solutions to classical problems, new approaches are being proposed. A paradigmatic case is the humanoid robotics, since the advances done in this field require motion planners not only to look efficiently for an optimal solution in the classic way, i.e. optimizing consumed energy or time in the plan execution, but also looking for human-like solutions, i.e. requiring the robot movements to be similar to those of the human beings. This anthropomorphism in the robot motion is desired not only for aesthetical reasons, but it is also needed to allow a better and safer human-robot collaboration: humans can predict more easily anthropomorphic robot motions thus avoiding collisions and enhancing the collaboration with the robot. Nevertheless, obtaining a satisfactory performance of these anthropomorphic robotic systems requires the automatic planning of the movements, which is still an arduous and non-evident task since the complexity of the planning problem increases exponentially with the number of degrees of freedom of the robotic system. This doctoral thesis tackles the problem of planning the motions of dual-arm anthropomorphic robots (optionally with mobile base). The main objective is twofold: obtaining robot motions both in an efficient and in a human-like fashion at the same time. Trying to mimic the human movements while reducing the complexity of the search space for planning purposes leads to the concept of synergies, which could be conceptually defined as correlations (in the joint configuration space as well as in the joint velocity space) between the degrees of freedom of the system. This work proposes new sampling-based motion-planning procedures that exploit the concept of synergies, both in the configuration and velocity space, coordinating the movements of the arms, the hands and the mobile base of mobile anthropomorphic dual-arm robots.La planificación de movimientos es un campo tradicional de la robótica, sin embargo aparecen incesantemente nuevos problemas debido a los continuos avances en el desarrollo de los robots. Para resolver esos nuevos problemas, así como para mejorar las soluciones existentes a los problemas clásicos, se están proponiendo nuevos enfoques. Un caso paradigmático es la robótica humanoide, ya que los avances realizados en este campo requieren que los algoritmos planificadores de movimientos no sólo encuentren eficientemente una solución óptima en el sentido clásico, es decir, optimizar el consumo de energía o el tiempo de ejecución de la trayectoria; sino que también busquen soluciones con apariencia humana, es decir, que el movimiento del robot sea similar al del ser humano. Este antropomorfismo en el movimiento del robot se busca no sólo por razones estéticas, sino porque también es necesario para permitir una colaboración mejor y más segura entre el robot y el operario: el ser humano puede predecir con mayor facilidad los movimientos del robot si éstos son antropomórficos, evitando así las colisiones y mejorando la colaboración humano robot. Sin embargo, para obtener un desempeño satisfactorio de estos sistemas robóticos antropomórficos se requiere una planificación automática de sus movimientos, lo que sigue siendo una tarea ardua y poco evidente, ya que la complejidad del problema aumenta exponencialmente con el número de grados de libertad del sistema robótico. Esta tesis doctoral aborda el problema de la planificación de movimientos en robots antropomorfos bibrazo (opcionalmente con base móvil). El objetivo aquí es doble: obtener movimientos robóticos de forma eficiente y, a la vez, que tengan apariencia humana. Intentar imitar los movimientos humanos mientras a la vez se reduce la complejidad del espacio de búsqueda conduce al concepto de sinergias, que podrían definirse conceptualmente como correlaciones (tanto en el espacio de configuraciones como en el espacio de velocidades de las articulaciones) entre los distintos grados de libertad del sistema. Este trabajo propone nuevos procedimientos de planificación de movimientos que explotan el concepto de sinergias, tanto en el espacio de configuraciones como en el espacio de velocidades, coordinando así los movimientos de los brazos, las manos y la base móvil de robots móviles, bibrazo y antropomórficos.Postprint (published version

Integrated Grasp and Motion Planning using Independent Contact Regions

Traditionally, grasp and arm motion planning are considered as separate tasks. This might lead to problems such as limitations in the grasp possibilities, or unnecessary long times in the solution of the planning problem. This thesis presents an integrated approach that only requires the initial con guration of the robotic arm and the pose of the target object to simultaneously plan a good hand pose and arm trajectory to grasp the object. The planner exploits the concept of independent contact regions to look for the best possible grasp. In this document, two di erent methods have been considered to search for good end-e ector poses. One biases a sampling approach towards favorable regions using principal component analysis, and the other one considers the capabilities of the robotic arm to decide the most promising hand poses. The performance of the methods is evaluated using di erent scenarios for the humanoid robot SpaceJustin. In order to validate the paths, some scenarios were replicated in the laboratory and the generated paths were executed on the real robot

Adhesion State Estimation for Electrostatic Gripper Based on Online Capacitance Measure

Electroadhesion is a suitable technology for developing grippers for applications where fragile, compliant or variable shape objects need to be grabbed and where a retention action is typically preferred to a compression force. This article presents a self-sensing technique for electroadhesive devices (EAD) based on the capacitance measure. Specifically, we demonstrate that measuring the variation of the capacitance between electrodes of an EAD during the adhesion can provide useful information to automatically detect the successful grip of an object and the possible loss of adhesion during manipulation. To this aim, a dedicated electronic circuit is developed that is able to measure capacitance variations while the high voltage required for the adhesion is activated. A test bench characterization is presented to evaluate the self-sensing of capacitance during different states: (1) the EAD is far away from the object to be grasped; (2) the EAD is in contact with the object, but the voltage is not active (i.e., no adhesion); and (3) the EAD is activated and attached to the object. Correlation between the applied voltage, object material and shape and capacitance is made. The self-sensing EAD is then demonstrated in a closed-loop robotic application that employs a robot manipulator arm to pick and place objects of different kinds

Towards Autonomous Selective Harvesting: A Review of Robot Perception, Robot Design, Motion Planning and Control

This paper provides an overview of the current state-of-the-art in selective harvesting robots (SHRs) and their potential for addressing the challenges of global food production. SHRs have the potential to increase productivity, reduce labour costs, and minimise food waste by selectively harvesting only ripe fruits and vegetables. The paper discusses the main components of SHRs, including perception, grasping, cutting, motion planning, and control. It also highlights the challenges in developing SHR technologies, particularly in the areas of robot design, motion planning and control. The paper also discusses the potential benefits of integrating AI and soft robots and data-driven methods to enhance the performance and robustness of SHR systems. Finally, the paper identifies several open research questions in the field and highlights the need for further research and development efforts to advance SHR technologies to meet the challenges of global food production. Overall, this paper provides a starting point for researchers and practitioners interested in developing SHRs and highlights the need for more research in this field.Comment: Preprint: to be appeared in Journal of Field Robotic

Using AI and Robotics for EV battery cable detection.: Development and implementation of end-to-end model-free 3D instance segmentation for industrial purposes

Master's thesis in Information- and communication technology (IKT590)This thesis describes a novel method for capturing point clouds and segmenting instances of cabling found on electric vehicle battery packs. The use of cutting-edge perception algorithm architectures, such as graph-based and voxel-based convolution, in industrial autonomous lithium-ion battery pack disassembly is being investigated. The thesis focuses on the challenge of getting a desirable representation of any battery pack using an ABB robot in conjunction with a high-end structured light camera, with "end-to-end" and "model-free" as design constraints. The thesis employs self-captured datasets comprised of several battery packs that have been captured and labeled. Following that, the datasets are used to create a perception system. This thesis recommends using HDR functionality in an industrial application to capture the full dynamic range of the battery packs. To adequately depict 3D features, a three-point-of-view capture sequence is deemed necessary. A general capture process for an entire battery pack is also presented, but a next-best-scan algorithm is likely required to ensure a "close to complete" representation. Graph-based deep-learning algorithms have been shown to be capable of being scaled up to50,000inputs while still exhibiting strong performance in terms of accuracy and processing time. The results show that an instance segmenting system can be implemented in less than two seconds. Using off-the-shelf hardware, demonstrate that a 3D perception system is industrially viable and competitive with a 2D perception system