Predictive Context-Based Adaptive Compliance for Interaction Control of Robot Manipulators

In classical industrial robotics, robots are concealed within structured and well-known environments performing highly-repetitive tasks. In contrast, current robotic applications require more direct interaction with humans, cooperating with them to achieve a common task and entering home scenarios. Above all, robots are leaving the world of certainty to work in dynamically-changing and unstructured environments that might be partially or completely unknown to them. In such environments, controlling the interaction forces that appear when a robot contacts a certain environment (be the environment an object or a person) is of utmost importance. Common sense suggests the need to leave the stiff industrial robots and move towards compliant and adaptive robot manipulators that resemble the properties of their biological counterpart, the human arm. This thesis focuses on creating a higher level of intelligence for active compliance control methods applied to robot manipulators. This work thus proposes an architecture for compliance regulation named Predictive Context-Based Adaptive Compliance (PCAC) which is composed of three main components operating around a 'classical' impedance controller. Inspired by biological systems, the highest-level component is a Bayesian-based context predictor that allows the robot to pre-regulate the arm compliance based on predictions about the context the robot is placed in. The robot can use the information obtained while contacting the environment to update its context predictions and, in case it is necessary, to correct in real time for wrongly predicted contexts. Thus, the predictions are used both for anticipating actions to be taken 'before' proceeding with a task as well as for applying real-time corrective measures 'during' the execution of a in order to ensure a successful performance. Additionally, this thesis investigates a second component to identify the current environment among a set of known environments. This in turn allows the robot to select the proper compliance controller. The third component of the architecture presents the use of neuroevolutionary techniques for selecting the optimal parameters of the interaction controller once a certain environment has been identified

Bringing a Humanoid Robot Closer to Human Versatility : Hard Realtime Software Architecture and Deep Learning Based Tactile Sensing

For centuries, it has been a vision of man to create humanoid robots, i.e., machines that not only resemble the shape of the human body, but have similar capabilities, especially in dextrously manipulating their environment. But only in recent years it has been possible to build actual humanoid robots with many degrees of freedom (DOF) and equipped with torque controlled joints, which are a prerequisite for sensitively acting in the world. In this thesis, we extend DLR's advanced mobile torque controlled humanoid robot Agile Justin into two important directions to get closer to human versatility. First, we enable Agile Justin, which was originally built as a research platform for dextrous mobile manipulation, to also be able to execute complex dynamic manipulation tasks. We demonstrate this with the challenging task of catching up to two simultaneously thrown balls with its hands. Second, we equip Agile Justin with highly developed and deep learning based tactile sensing capabilities that are critical for dextrous fine manipulation. We demonstrate its tactile capabilities with the delicate task of identifying an objects material simply by gently sweeping with a fingertip over its surface. Key for the realization of complex dynamic manipulation tasks is a software framework that allows for a component based system architecture to cope with the complexity and parallel and distributed computational demands of deep sensor-perception-planning-action loops -- but under tight timing constraints. This thesis presents the communication layer of our aRDx (agile robot development -- next generation) software framework that provides hard realtime determinism and optimal transport of data packets with zero-copy for intra- and inter-process and copy-once for distributed communication. In the implementation of the challenging ball catching application on Agile Justin, we take full advantage of aRDx's performance and advanced features like channel synchronization. Besides developing the challenging visual ball tracking using only onboard sensing while everything is moving and the automatic and self-contained calibration procedure to provide the necessary precision, the major contribution is the unified generation of the reaching motion for the arms. The catch point selection, motion planning and the joint interpolation steps are subsumed in one nonlinear constrained optimization problem which is solved in realtime and allows for the realization of different catch behaviors. For the highly sensitive task of tactile material classification with a flexible pressure-sensitive skin on Agile Justin's fingertip, we present our deep convolutional network architecture TactNet-II. The input is the raw 16000 dimensional complex and noisy spatio-temporal tactile signal generated when sweeping over an object's surface. For comparison, we perform a thorough human performance experiment with 15 subjects which shows that Agile Justin reaches superhuman performance in the high-level material classification task (What material id?), as well as in the low-level material differentiation task (Are two materials the same?). To increase the sample efficiency of TactNet-II, we adapt state of the art deep end-to-end transfer learning to tactile material classification leading to an up to 15 fold reduction in the number of training samples needed. The presented methods led to six publication awards and award finalists and international media coverage but also worked robustly at many trade fairs and lab demos

Persuasiveness of social robot ‘Nao’ based on gaze and proximity

Social Robots have widely infiltrated the retail and public space. Mainly, social robots are being utilized across a wide range of scenarios to influence decision making, disseminate information, and act as a signage mechanism, under the umbrella of Persuasive Robots or Persuasive Technology. While there have been several studies in the afore-mentioned area, the effect of non-verbal behaviour on persuasive abilities is generally unexplored. Therefore, in this research, we report whether two key non-verbal attributes, namely proximity and gaze, can elicit persuasively, compliance, and specific personality appeals. For this, we conducted a 2 (eye gaze) x 2 (proximity) between-subjects experiment where participants viewed a video-based scenario of the Nao robot. Our initial results did not reveal any significant results based on the non-verbal attributes. However, perceived compliance and persuasion were significantly correlated with knowledge, responsiveness, and trustworthiness. In conclusion, we discuss how the design of a robot could make it more convincing as extensive marketing and brand promotion companies could use robots to enhance their advertisement operations

Machine Learning Meets Advanced Robotic Manipulation

Automated industries lead to high quality production, lower manufacturing cost and better utilization of human resources. Robotic manipulator arms have major role in the automation process. However, for complex manipulation tasks, hard coding efficient and safe trajectories is challenging and time consuming. Machine learning methods have the potential to learn such controllers based on expert demonstrations. Despite promising advances, better approaches must be developed to improve safety, reliability, and efficiency of ML methods in both training and deployment phases. This survey aims to review cutting edge technologies and recent trends on ML methods applied to real-world manipulation tasks. After reviewing the related background on ML, the rest of the paper is devoted to ML applications in different domains such as industry, healthcare, agriculture, space, military, and search and rescue. The paper is closed with important research directions for future works

An intelligent multi-floor mobile robot transportation system in life science laboratories

In this dissertation, a new intelligent multi-floor transportation system based on mobile robot is presented to connect the distributed laboratories in multi-floor environment. In the system, new indoor mapping and localization are presented, hybrid path planning is proposed, and an automated doors management system is presented. In addition, a hybrid strategy with innovative floor estimation to handle the elevator operations is implemented. Finally the presented system controls the working processes of the related sub-system. The experiments prove the efficiency of the presented system

Data augmentation using generative adversarial networks for electrical insulator anomaly detection

Master of ScienceDepartment of Computer ScienceWilliam H. HsuElectricity has been an essential part of our life. Insulators, which are widely used for electricity transmission, are prone to be damaged and need constant maintenance. Traditionally, the inspection job is time-consuming and dangerous as workers would have to climb up the electricity tower. Deep learning has offered a safe and quick way to inspections. About 3000 insulators images are taken from different angles using a drone. Due to great difference in number of good and damaged insulator, directly training a classifier on the imbalanced data lead to low recall value on the damaged insulators. Generative adversarial networks (GANs) were introduced as a novel way to augment data. However, traditional GANs are either incapable of generating high quality images or fail to generate minority class images when minority class examples are far less. In this study, a novel GAN model, Balancing and Progressive GANs (BPGANs), was proposed for effectively making use of all classes information and generating high quality minority images at the same time. Results show that PGANs, StyleGANs, and BPGANs were able to generate high-resolution images and improve classification performance. PGANs achieved the better results than BPGANs. This may be because BPGANs only provides 2 additional latent codes since it is a binary classification, having little effect on generating desired images. BPGANs seemed to have difficulties generating class-specific images, which might be because that the classification loss is too little compared to the source loss and optimization was more focused to optimize the source loss. This indicates that learning representations of data progressively from low resolution to high resolution is an effective approach, however, embedding class label information in the fashion of AC-GANs and BGANs might not be appropriate for augmenting binary class data sets