Cognitive Reasoning for Compliant Robot Manipulation

Physically compliant contact is a major element for many tasks in everyday environments. A universal service robot that is utilized to collect leaves in a park, polish a workpiece, or clean solar panels requires the cognition and manipulation capabilities to facilitate such compliant interaction. Evolution equipped humans with advanced mental abilities to envision physical contact situations and their resulting outcome, dexterous motor skills to perform the actions accordingly, as well as a sense of quality to rate the outcome of the task. In order to achieve human-like performance, a robot must provide the necessary methods to represent, plan, execute, and interpret compliant manipulation tasks. This dissertation covers those four steps of reasoning in the concept of intelligent physical compliance. The contributions advance the capabilities of service robots by combining artificial intelligence reasoning methods and control strategies for compliant manipulation. A classification of manipulation tasks is conducted to identify the central research questions of the addressed topic. Novel representations are derived to describe the properties of physical interaction. Special attention is given to wiping tasks which are predominant in everyday environments. It is investigated how symbolic task descriptions can be translated into meaningful robot commands. A particle distribution model is used to plan goal-oriented wiping actions and predict the quality according to the anticipated result. The planned tool motions are converted into the joint space of the humanoid robot Rollin' Justin to perform the tasks in the real world. In order to execute the motions in a physically compliant fashion, a hierarchical whole-body impedance controller is integrated into the framework. The controller is automatically parameterized with respect to the requirements of the particular task. Haptic feedback is utilized to infer contact and interpret the performance semantically. Finally, the robot is able to compensate for possible disturbances as it plans additional recovery motions while effectively closing the cognitive control loop. Among others, the developed concept is applied in an actual space robotics mission, in which an astronaut aboard the International Space Station (ISS) commands Rollin' Justin to maintain a Martian solar panel farm in a mock-up environment. This application demonstrates the far-reaching impact of the proposed approach and the associated opportunities that emerge with the availability of cognition-enabled service robots

Cognition-enabled robotic wiping: Representation, planning, execution, and interpretation

Advanced cognitive capabilities enable humans to solve even complex tasks by representing and processing internal models of manipulation actions and their effects. Consequently, humans are able to plan the effect of their motions before execution and validate the performance afterwards. In this work, we derive an analog approach for robotic wiping actions which are fundamental for some of the most frequent household chores including vacuuming the floor, sweeping dust, and cleaning windows. We describe wiping actions and their effects based on a qualitative particle distribution model. This representation enables a robot to plan goal-oriented wiping motions for the prototypical wiping actions of absorbing, collecting and skimming. The particle representation is utilized to simulate the task outcome before execution and infer the real performance afterwards based on haptic perception. This way, the robot is able to estimate the task performance and schedule additional motions if necessary. We evaluate our methods in simulated scenarios, as well as in real experiments with the humanoid service robot Rollin’ Justin

Human-friendly robotic manipulators: safety and performance issues in controller design

Recent advances in robotics have spurred its adoption into new application areas such as medical, rescue, transportation, logistics, personal care and entertainment. In the personal care domain, robots are expected to operate in human-present environments and provide non-critical assistance. Successful and flourishing deployment of such robots present different opportunities as well as challenges. Under a national research project, Bobbie, this dissertation analyzes challenges associated with these robots and proposes solutions for identified problems. The thesis begins by highlighting the important safety concern and presenting a comprehensive overview of safety issues in a typical domestic robot system. By using functional safety concept, the overall safety of the complex robotic system was analyzed through subsystem level safety issues. Safety regions in the world model of the perception subsystem, dependable understanding of the unstructured environment via fusion of sensory subsystems, lightweight and compliant design of mechanical components, passivity based control system and quantitative metrics used to assert safety are some important points discussed in the safety review. The main research focus of this work is on controller design of robotic manipulators against two conflicting requirements: motion performance and safety. Human-friendly manipulators used on domestic robots exhibit a lightweight design and demand a stable operation with a compliant behavior injected via a passivity based impedance controller. Effective motion based manipulation using such a controller requires a highly stiff behavior while important safety requirements are achieved with compliant behaviors. On the basis of this intuitive observation, this research identifies suitable metrics to identify the appropriate impedance for a given performance and safety requirement. This thesis also introduces a domestic robot design that adopts a modular design approach to minimize complexity, cost and development time. On the basis of functional modularity concept where each module has a unique functional contribution in the system, the robot “Bobbie-UT‿ is built as an interconnection of interchangeable mobile platform, torso, robotic arm and humanoid head components. Implementation of necessary functional and safety requirements, design of interfaces and development of suitable software architecture are also discussed with the design

Technology 2001: The Second National Technology Transfer Conference and Exposition, volume 2

Proceedings of the workshop are presented. The mission of the conference was to transfer advanced technologies developed by the Federal government, its contractors, and other high-tech organizations to U.S. industries for their use in developing new or improved products and processes. Volume two presents papers on the following topics: materials science, robotics, test and measurement, advanced manufacturing, artificial intelligence, biotechnology, electronics, and software engineering

A robotic engine assembly pick-place system based on machine learning

Industrial revolution brought humans and machines together in building a better future. Where in one hand there is need to replace the repetitive jobs with machines to increase efficiency and volume of production, on the other hand intelligent and autonomous machines have still a long way to go to achieve dexterity of a human. The current scenario requires a system which can utilise best of both the human and the machine. This thesis studies a industrial use case scenario where human-machine combine their skills to build an autonomous pick place system. This study takes a small step towards the human-robot consortium primarily focusing on developing a vision based system for object detection followed by a manipulator pick place operation. This thesis can be divided into two parts : 1. Scene analysis, where a Convolutional Neural Network (CNN) is used for object detection followed by generation of grasping points using object edge image and an algorithm developed during this thesis. 2. Implementation, it focuses on motion generation while taking care of external disturbances to perform successful pick-place operation. In addition human involvement is required which includes teaching trajectory points for the robot to follow. This trajectory is used to generate image data-set for a new object type and thereafter generating new object detection model. The author primarily focuses on building a system framework where the complexities related to robot programming such as generating trajectory points and informing grasping position is not required. The system automatically detects object and performs a pick place operation, resulting in relieving user from robot programming. The system is composed of a depth camera and a manipulator. Camera is the only sensor available for scene analysis and the action is performed using a Franka manipulator. The two components work in request-response mode over ROS. This thesis introduces a newer approaches such as, dividing an workspace image into its constituent object images and performing object detection, creating training data, generating grasp points based on object shape along length of an object. The thesis also presents a case study where three different objects are chosen as test objects. The experiments are a demonstration of the methods applied and efficiency attained. The case study also provides a glimpse of the future research and development areas

A review on the prospects of mobile manipulators for smart maintenance of railway track

Inspection and repair interventions play vital roles in the asset management of railways. Autonomous mobile manipulators possess considerable potential to replace humans in many hazardous railway track maintenance tasks with high efficiency. This paper investigates the prospects of the use of mobile manipulators in track maintenance tasks. The current state of railway track inspection and repair technologies is initially reviewed, revealing that very few mobile manipulators are in the railways. Of note, the technologies are analytically scrutinized to ascertain advantages, unique capabilities, and potential use in the deployment of mobile manipulators for inspection and repair tasks across various industries. Most mobile manipulators in maintenance use ground robots, while other applications use aerial, underwater, or space robots. Power transmission lines, the nuclear industry, and space are the most extensive application areas. Clearly, the railways infrastructure managers can benefit from the adaptation of best practices from these diversified designs and their broad deployment, leading to enhanced human safety and optimized asset digitalization. A case study is presented to show the potential use of mobile manipulators in railway track maintenance tasks. Moreover, the benefits of the mobile manipulator are discussed based on previous research. Finally, challenges and requirements are reviewed to provide insights into future research

Human Robot Collaboration in Assembly Processes

Human robot collaboration (HRC) in assembly processes, is a concept which aims to integrate the human workforce with the robots in a shared workspace. This is carried out to complement the human workforce to achieve key manufacturing metrics e.g. efficiency, ergonomics, etc. Assembly of the switch module (DIN rail components’ assembly) was taken to be the product under investigation. Design of human robot collaborative assembly process was explored while considering the following objectives and knowledge areas: Design for Assembly (DFA) methodology, rules for task assignment between the human worker and the robot, Identification and mitigation of process hazards, and trainable collaborative setups. A total of five DFA axioms were found to be relevant for the design of human robot collaborative assembly process. Task Assignments between the human and the robot was explored while considering the following objectives: Optimal assembly cycle times and workspace ergonomics. Three out of four phases of assembly process offered room for collaborative endeavors. The task assignments during these assembly phases ensured optimal cycle times and work space ergonomics. Risk assessment along with risk mitigation strategies were evaluated on the basis of ISO 12100. Hand guided frame teaching methodology was used to train the collaborative setup. This enabled an operator friendly/oriented approach towards the design and implementation of collaborative assembly tasks. The obtained results ensured a reliable and industry scalable implementation of human robot collaborative assembly process

Compliant aerial manipulation.

The aerial manipulation is a research field which proposes the integration of robotic manipulators in aerial platforms, typically multirotors – widely known as “drones” – or autonomous helicopters. The development of this technology is motivated by the convenience to reduce the time, cost and risk associated to the execution of certain operations or tasks in high altitude areas or difficult access workspaces. Some illustrative application examples are the detection and insulation of leaks in pipe structures in chemical plants, repairing the corrosion in the blades of wind turbines, the maintenance of power lines, or the installation and retrieval of sensor devices in polluted areas. Although nowadays it is possible to find a wide variety of commercial multirotor platforms with payloads from a few gramps up to several kilograms, and flight times around thirty minutes, the development of an aerial manipulator is still a technological challenge due to the strong requirements relative to the design of the manipulator in terms of very low weight, low inertia, dexterity, mechanical robustness and control. The main contribution of this thesis is the design, development and experimental validation of several prototypes of lightweight (<2 kg) and compliant manipulators to be integrated in multirotor platforms, including human-size dual arm systems, compliant joint arms equipped with human-like finger modules for grasping, and long reach aerial manipulators. Since it is expected that the aerial manipulator is capable to execute inspection and maintenance tasks in a similar way a human operator would do, this thesis proposes a bioinspired design approach, trying to replicate the human arm in terms of size, kinematics, mass distribution, and compliance. This last feature is actually one of the key concepts developed and exploited in this work. Introducing a flexible element such as springs or elastomers between the servos and the links extends the capabilities of the manipulator, allowing the estimation and control of the torque/force, the detection of impacts and overloads, or the localization of obstacles by contact. It also improves safety and efficiency of the manipulator, especially during the operation on flight or in grabbing situations, where the impacts and contact forces may damage the manipulator or destabilize the aerial platform. Unlike most industrial manipulators, where force-torque control is possible at control rates above 1 kHz, the servo actuators typically employed in the development of aerial manipulators present important technological limitations: no torque feedback nor control, only position (and in some models, speed) references, low update rates (<100 Hz), and communication delays. However, these devices are still the best solution due to their high torque to weight ratio, low cost, compact design, and easy assembly and integration. In order to cope with these limitations, the compliant joint arms presented here estimate and control the wrenches from the deflection of the spring-lever transmission mechanism introduced in the joints, measured at joint level with encoders or potentiometers, or in the Cartesian space employing vision sensors. Note that in the developed prototypes, the maximum joint deflection is around 25 degrees, which corresponds to a deviation in the position of the end effector around 20 cm for a human-size arm. The capabilities and functionalities of the manipulators have been evaluated in fixed base test-bench firstly, and then in outdoor flight tests, integrating the arms in different commercial hexarotor platforms. Frequency characterization, position/force/impedance control, bimanual grasping, arm teleoperation, payload mass estimation, or contact-based obstacle localization are some of the experiments presented in this thesis that validate the developed prototypes.La manipulación aérea es un campo de investigación que propone la integración de manipuladores robóticos in plataformas aéreas, típicamente multirotores – comúnmente conocidos como “drones” – o helicópteros autónomos. El desarrollo de esta tecnología está motivada por la conveniencia de reducir el tiempo, coste y riesgo asociado a la ejecución de ciertas operaciones o tareas en áreas de gran altura o espacios de trabajo de difícil acceso. Algunos ejemplos ilustrativos de aplicaciones son la detección y aislamiento de fugas en estructura de tuberías en plantas químicas, la reparación de la corrosión en las palas de aerogeneradores, el mantenimiento de líneas eléctricas, o la instalación y recuperación de sensores en zonas contaminadas. Aunque hoy en día es posible encontrar una amplia variedad de plataformas multirotor comerciales con cargas de pago desde unos pocos gramos hasta varios kilogramos, y tiempo de vuelo entorno a treinta minutos, el desarrollo de los manipuladores aéreos es todavía un desafío tecnológico debido a los exigentes requisitos relativos al diseño del manipulador en términos de muy bajo peso, baja inercia, destreza, robustez mecánica y control. La contribución principal de esta tesis es el diseño, desarrollo y validación experimental de varios prototipos de manipuladores de bajo peso (<2 kg) con capacidad de acomodación (“compliant”) para su integración en plataformas aéreas multirotor, incluyendo sistemas bi-brazo de tamaño humano, brazos robóticos de articulaciones flexibles con dedos antropomórficos para agarre, y manipuladores aéreos de largo alcance. Puesto que se prevé que el manipulador aéreo sea capaz de ejecutar tareas de inspección y mantenimiento de forma similar a como lo haría un operador humano, esta tesis propone un enfoque de diseño bio-inspirado, tratando de replicar el brazo humano en cuanto a tamaño, cinemática, distribución de masas y flexibilidad. Esta característica es de hecho uno de los conceptos clave desarrollados y utilizados en este trabajo. Al introducir un elemento elástico como los muelles o elastómeros entre el los actuadores y los enlaces se aumenta las capacidades del manipulador, permitiendo la estimación y control de las fuerzas y pares, la detección de impactos y sobrecargas, o la localización de obstáculos por contacto. Además mejora la seguridad y eficiencia del manipulador, especialmente durante las operaciones en vuelo, donde los impactos y fuerzas de contacto pueden dañar el manipulador o desestabilizar la plataforma aérea. A diferencia de la mayoría de manipuladores industriales, donde el control de fuerzas y pares es posible a tasas por encima de 1 kHz, los servo motores típicamente utilizados en el desarrollo de manipuladores aéreos presentan importantes limitaciones tecnológicas: no hay realimentación ni control de torque, sólo admiten referencias de posición (o bien de velocidad), y presentan retrasos de comunicación. Sin embargo, estos dispositivos son todavía la mejor solución debido al alto ratio de torque a peso, por su bajo peso, diseño compacto y facilidad de ensamblado e integración. Para suplir estas limitaciones, los brazos robóticos flexibles presentados aquí permiten estimar y controlar las fuerzas a partir de la deflexión del mecanismo de muelle-palanca introducido en las articulaciones, medida a nivel articular mediante potenciómetros o codificadores, o en espacio Cartesiano mediante sensores de visión. Tómese como referencia que en los prototipos desarrollados la máxima deflexión articular es de unos 25 grados, lo que corresponde a una desviación de posición en torno a 20 cm en el efector final para un brazo de tamaño humano. Las capacidades y funcionalidades de estos manipuladores se han evaluado en base fija primero, y luego en vuelos en exteriores, integrando los brazos en diferentes plataformas hexartor comerciales. Caracterización frecuencial, control de posición/fuerza/impedancia, agarre bimanual, teleoperación de brazos, estimación de carga, o la localización de obstáculos mediante contacto son algunos de los experimentos presentados en esta tesis para validar los prototipos desarrollados por el auto