Contact Estimation in Robot Interaction

In the paper, safety issues are examined in a scenario in which a robot manipulator and a human perform the same task in the same workspace. During the task execution, the human should be able to physically interact with the robot, and in this case an estimation algorithm for both interaction forces and a contact point is proposed in order to guarantee safety conditions. The method, starting from residual joint torque estimation, allows both direct and adaptive computation of the contact point and force, based on a principle of equivalence of the contact forces. At the same time, all the unintended contacts must be avoided, and a suitable post-collision strategy is considered to move the robot away from the collision area or else to reduce impact effects. Proper experimental tests have demonstrated the applicability in practice of both the post-impact strategy and the estimation algorithms; furthermore, experiments demonstrate the different behaviour resulting from the adaptation of the contact point as opposed to direct calculation

On Sensorless Collision Detection and Measurement of External Forces in Presence of Modeling Inaccuracies

The field of human-robot interaction has garnered significant interest in the last decade. Every form of human-robot coexistence must guarantee the safety of the user. Safety in human-robot interaction is being vigorously studied, in areas such as collision avoidance, soft actuators, light-weight robots, computer vision techniques, soft tissue modeling, collision detection, etc. Despite the safety provisions, unwanted collisions can occur in case of system faults. In such cases, before post-collision strategies are triggered, it is imperative to effectively detect the collisions. Implementation of tactile sensors, vision systems, sonar and Lidar sensors, etc., allows for detection of collisions. However, due to the cost of such methods, more practical approaches are being investigated. A general goal remains to develop methods for fast detection of external contacts using minimal sensory information. Availability of position data and command torques in manipulators permits development of observer-based techniques to measure external forces/torques. The presence of disturbances and inaccuracies in the model of the robot presents challenges in the efficacy of observers in the context of collision detection. The purpose of this thesis is to develop methods that reduce the effects of modeling inaccuracies in external force/torque estimation and increase the efficacy of collision detection. It is comprised of the following four parts: 1. The KUKA Light-Weight Robot IV+ is commonly employed for research purposes. The regressor matrix, minimal inertial parameters and the friction model of this robot are identified and presented in detail. To develop the model, relative weight analysis is employed for identification. 2. Modeling inaccuracies and robot state approximation errors are considered simultaneously to develop model-based time-varying thresholds for collision detection. A metric is formulated to compare trajectories realizing the same task in terms of their collision detection and external force/torque estimation capabilities. A method for determining optimal trajectories with regards to accurate external force/torque estimation is also developed. 3. The effects of velocity on external force/torque estimation errors are studied with and without the use of joint force/torque sensors. Velocity-based thresholds are developed and implemented to improve collision detection. The results are compared with the collision detection module integrated in the KUKA Light-Weight Robot IV+. 4. An alternative joint-by-joint heuristic method is proposed to identify the effects of modeling inaccuracies on external force/torque estimation. Time-varying collision detection thresholds associated with the heuristic method are developed and compared with constant thresholds. In this work, the KUKA Light-Weight Robot IV+ is used for obtaining the experimental results. This robot is controlled via the Fast Research Interface and Visual C++ 2008. The experimental results confirm the efficacy of the proposed methodologies

Development of a Virtual Collision Sensor for Industrial Robots

Collision detection is a fundamental issue for the safety of a robotic cell. While several common methods require specific sensors or the knowledge of the robot dynamic model, the proposed solution is constituted by a virtual collision sensor for industrial manipulators, which requires as inputs only the motor currents measured by the standard sensors that equip a manipulator and the estimated currents provided by an internal dynamic model of the robot (i.e., the one used inside its controller), whose structure, parameters and accuracy are not known. The collision detection is achieved by comparing the absolute value of the current residue with a time-varying, positive-valued threshold function, including an estimate of the model error and a bias term, corresponding to the minimum collision torque to be detected. The value of such a term, defining the sensor sensitivity, can be simply imposed as constant, or automatically customized for a specific robotic application through a learning phase and a subsequent adaptation process, to achieve a more robust and faster collision detection, as well as the avoidance of any false collision warnings, even in case of slow variations of the robot behavior. Experimental results are provided to confirm the validity of the proposed solution, which is already adopted in some industrial scenarios

지도 및 비지도 학습을 이용한 로봇 머니퓰레이터 충돌 감지

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2022.2. 박종우.사람과 공유된 구조화되지 않은 동적 환경에서 작동하는 협업 로봇 머니퓰레이터는 날카로운 충돌(경성 충돌)에서 더 긴 지속 시간의 밀고 당기는 동작(연성 충돌)에 이르기까지의 다양한 충돌을 빠르고 정확하게 감지해야 한다. 모터 전류 측정값을 이용해 외부 조인트 토크를 추정하는 동역학 모델 기반 감지 방법을 사용할 경우, 정확한 마찰 파라미터 모델링 및 식별과 같은 모터 마찰에 대한 적절한 처리가 필요하다. 올바르게 적용하면 매우 효과적이지만, 동역학과 마찰 파라미터를 모델링 및 식별하고 여러 개의 감지 임계값을 수동으로 설정하는 데에는 상당한 노력이 필요하기 때문에 대량 생산되는 산업용 로봇에 이를 적용하기는 어렵다. 또한 적절한 식별 후에도 동역학에 백래시, 탄성 등 모델링되지 않은 효과나 불확실성이 여전히 존재할 수 있다. 본 논문에서는 순수 모델 기반 방법의 구현 어려움을 피하고 불확실한 동역학적 효과를 보상하는 수단으로 로봇 머니퓰레이터를 위한 총 네 가지의 학습 기반 충돌 감지 방법을 제안한다. 두 개의 방법은 학습을 위해 충돌 및 비충돌 동작 데이터가 모두 필요한 지도 학습 알고리즘(서포트 벡터 머신 회귀, 일차원 합성곱 신경망 기반)을 사용하며 나머지 두 개의 방법은 학습을 위해 비충돌 동작 데이터만을 필요로 하는 비지도 이상치 탐지 알고리즘(단일 클래스 서포트 벡터 머신, 오토인코더 기반)에 기반한다. 로봇 동역학 모델과 모터 전류 측정값만을 필요로 하며 추가적인 외부 센서나 마찰 모델링, 여러 개의 감지 임계값에 대한 수동 조정은 필요하지 않다. 먼저 지도 및 비지도 감지 방법을 학습시키고 검증하는 데 사용되는, 6자유도 협업 로봇 머니퓰레이터를 이용해 수집된 로봇 충돌 데이터를 설명한다. 우리가 고려하는 충돌 시나리오는 경성 충돌, 연성 충돌, 비충돌 동작으로, 경성 및 연성 충돌은 모두 동일하게 충돌로 간주한다. 감지 성능 검증을 위한 테스트 데이터는 총 787건의 충돌과 62.4분의 비충돌 동작으로 이루어져 있으며, 이는 로봇이 랜덤 점대점 6관절 동작을 수행하는 동안 수집된다. 데이터 수집 중 로봇의 끝단에는 미부착, 3.3 kg, 5.0 kg의 세 가지 유형의 페이로드를 부착한다. 다음으로, 수집된 테스트 데이터를 이용해 지도 감지 방법의 감지 성능을 실험적으로 검증한다. 실험 결과는 지도 감지 방법이 가벼운 네트워크를 이용해 광범위한 경성 및 연성 충돌을 실시간으로 정확하게 감지할 수 있음을 보여주며, 이를 통해 모델 파라미터의 불확실성과 측정 노이즈, 백래시, 변형 등 모델링되지 않은 효과까지 보상됨을 알 수 있다. 또한 서포트 벡터 머신 회귀 기반 방법은 하나의 감지 임계값에 대한 조정만 필요하며 일차원 합성곱 신경망 기반 방법은 하나의 아웃풋 필터 파라미터에 대한 조정만 필요한데, 두 방법 모두 직관적인 감도 조정이 가능하다. 나아가 일련의 시뮬레이션 실험을 통해 지도 감지 방법의 일반화 성능을 실험적으로 검증한다. 마지막으로, 동일한 테스트 데이터에 대해 비지도 감지 방법의 감지 성능과 일반화 성능 또한 검증한다. 실험 결과는 비지도 감지 방법 또한 가벼운 계산과 하나의 감지 임계값에 대한 조정만으로 다양한 경성 및 연성 충돌을 실시간으로 강인하게 감지할 수 있음을 보여주며, 이를 통해 모델링되지 않은 마찰을 포함한 불확실한 동역학적 효과를 비지도 학습으로도 보상할 수 있음을 알 수 있다. 지도 감지 방법이 더 나은 감지 성능을 보이지만, 비지도 감지 방법은 학습을 위해 비충돌 동작 데이터만을 필요로 하며 발생할 수 있는 모든 유형의 충돌에 대한 정보를 필요로 하지 않기 때문에 대량 생산되는 산업용 로봇에 더 적합하다.Collaborative robot manipulators operating in dynamic and unstructured environments shared with humans require fast and accurate detection of collisions, which can range from sharp impacts (hard collisions) to pulling and pushing motions of longer duration (soft collisions). When using dynamics model-based detection methods that estimate the external joint torque with motor current measurements, proper treatment for friction in the motors is required, such as accurate modeling and identification of friction parameters. Although highly effective when done correctly, modeling and identifying the dynamics and friction parameters, and manually setting multiple detection thresholds require considerable effort, making them difficult to be replicated for mass-produced industrial robots. There may also still exist unmodeled effects or uncertainties in the dynamics even after proper identification, e.g., backlash, elasticity. This dissertation presents a total of four learning-based collision detection methods for robot manipulators as a means of sidestepping some of the implementation difficulties of pure model-based methods and compensating for uncertain dynamic effects. Two methods use supervised learning algorithms – support vector machine regression and a one-dimensional convolutional neural network-based – that require both the collision and collision-free motion data for training. The other two methods are based on unsupervised anomaly detection algorithms – a one-class support vector machine and an autoencoder-based – that require only the collision-free motion data for training. Only the motor current measurements together with a robot dynamics model are required while no additional external sensors, friction modeling, or manual tuning of multiple detection thresholds are needed. We first describe the robot collision dataset collected with a six-dof collaborative robot manipulator, which is used for training and validating our supervised and unsupervised detection methods. The collision scenarios we consider are hard collisions, soft collisions, and collision-free, where both hard and soft collisions are treated in the same manner as just collisions. The test dataset for detection performance verification includes a total of 787 collisions and 62.4 minutes of collision-free motions, all collected while the robot is executing random point-to-point six-joint motions. During data collection, three types of payloads are attached to the end-effector: no payload, 3.3 kg payload, and 5.0 kg payload. Then the detection performance of our supervised detection methods is experimentally verified with the collected test dataset. Results demonstrate that our supervised detection methods can accurately detect a wide range of hard and soft collisions in real-time using a light network, compensating for uncertainties in the model parameters as well as unmodeled effects like friction, measurement noise, backlash, and deformations. Moreover, the SVMR-based method requires only one constant detection threshold to be tuned while the 1-D CNN-based method requires only one output filter parameter to be tuned, both of which allow intuitive sensitivity tuning. Furthermore, the generalization capability of our supervised detection methods is experimentally verified with a set of simulation experiments. Finally, our unsupervised detection methods are also validated for the same test dataset; the detection performance and the generalization capability are verified. The experimental results show that our unsupervised detection methods are also able to robustly detect a variety of hard and soft collisions in real-time with very light computation and with only one constant detection threshold required to be tuned, validating that uncertain dynamic effects including the unmodeled friction can be successfully compensated also with unsupervised learning. Although our supervised detection methods show better detection performance, our unsupervised detection methods are more practical for mass-produced industrial robots since they require only the data for collision-free motions for training, and the knowledge of every possible type of collision that can occur is not required.1 Introduction 1 1.1 Model-Free Methods 2 1.2 Model-Based Methods 2 1.3 Learning-Based Methods 4 1.3.1 Using Supervised Learning Algorithms 5 1.3.2 Using Unsupervised Learning Algorithms 6 1.4 Contributions of This Dissertation 7 1.4.1 Supervised Learning-Based Model-Compensating Detection 7 1.4.2 Unsupervised Learning-Based Model-Compensating Detection 8 1.4.3 Comparison with Existing Detection Methods 9 1.5 Organization of This Dissertation 14 2 Preliminaries 17 2.1 Introduction 17 2.2 Robot Dynamics 17 2.3 Momentum Observer-Based Collision Detection 19 2.4 Supervised Learning Algorithms 21 2.4.1 Support Vector Machine Regression 21 2.4.2 One-Dimensional Convolutional Neural Network 23 2.5 Unsupervised Anomaly Detection 25 2.6 One-Class Support Vector Machine 26 2.7 Autoencoder-Based Anomaly Detection 28 2.7.1 Autoencoder Network Architecture and Training 28 2.7.2 Anomaly Detection Using Autoencoders 29 3 Robot Collision Data 31 3.1 Introduction 31 3.2 True Collision Index Labeling 31 3.3 Collision Scenarios 35 3.4 Monitoring Signal 36 3.5 Signal Normalization and Sampling 37 3.6 Test Data for Detection Performance Verification 39 4 Supervised Learning-Based Model-Compensating Detection 43 4.1 Introduction 43 4.2 SVMR-Based Collision Detection 44 4.2.1 Input Feature Vector Design 44 4.2.2 SVMR Training 45 4.2.3 Collision Detection Sensitivity Adjustment 46 4.3 1-D CNN-Based Collision Detection 50 4.3.1 Network Input Design 50 4.3.2 Network Architecture and Training 50 4.3.3 An Output Filtering Method to Reduce False Alarms 53 4.4 Collision Detection Performance Criteria 54 4.4.1 Area Under the Precision-Recall Curve (PRAUC) 54 4.4.2 Detection Delay and Number of Detection Failures 54 4.5 Collision Detection Performance Analysis 56 4.5.1 Global Performance with Varying Thresholds 56 4.5.2 Detection Delay and Number of Detection Failures 57 4.5.3 Real-Time Inference 60 4.6 Generalization Capability Analysis 60 4.6.1 Generalization to Small Perturbations 60 4.6.2 Generalization to an Unseen Payload 62 5 Unsupervised Learning-Based Model-Compensating Detection 67 5.1 Introduction 67 5.2 OC-SVM-Based Collision Detection 68 5.2.1 Input Feature Vector 68 5.2.2 OC-SVM Training 70 5.2.3 Collision Detection with the Trained OC-SVM 70 5.3 Autoencoder-Based Collision Detection 70 5.3.1 Network Input and Output 71 5.3.2 Network Architecture and Training 71 5.3.3 Collision Detection with the Trained Autoencoder 72 5.4 Collision Detection Performance Analysis 74 5.4.1 Global Performance with Varying Thresholds 75 5.4.2 Detection Delay and Number of Detection Failures 75 5.4.3 Comparison with Supervised Learning-Based Methods 80 5.4.4 Real-Time Inference 83 5.5 Generalization Capability Analysis 83 5.5.1 Generalization to Small Perturbations 84 5.5.2 Generalization to an Unseen Payload 85 6 Conclusion 89 6.1 Summary and Discussion 89 6.2 Future Work 93 A Appendix 95 A.1 SVM-Based Classification of Detected Collisions 95 A.2 Direct Estimation-Based Detection Methods 97 A.3 Model-Independent Supervised Detection Methods 101 A.4 Generalization to Large Changes in the Dynamics Model 102 Bibliography 106 Abstract 112박

ROBOTIC INTERACTION AND COOPERATION. Industrial and rehabilitative applications

The main goal of the thesis is the development of human-robotic interaction control strategies, which enable close collaboration between human and robot. In this framework we studied two di erent aspects, with applications respectively in industrial and rehabilitation domains. In the rst part safety issues are examined on a scenario in which a robot manipulator and a human perform the same task and in the same workspace. During the task execution the human should be able to get into contact with the robot and in this case an estimation algorithm of both interaction forces and contact point is proposed in order to guarantee safety conditions. At the same time, all the unintended contacts have to be avoided, and a suitable post collision strategy has been studied to move away the robot from the collision area or to reduce the impact e orts. However, the second part of the thesis focus on the cooperation between an orthesis and a patient. Indeed, in order to support a rehabilitation process, gait parameters, such as hip and knee angles or the beginning of a gait phase, have been estimated. For this purpose a sensor system, consisting of accelerometers and gyroscopes, and algorithms, developed in order to avoid the error accumulation due to the gyroscopes drift and the vibrations related to the beginning of the stance phase due to the accelerometers, have been proposed.The main goal of the thesis is the development of human-robotic interaction control strategies, which enable close collaboration between human and robot. In this framework we studied two di erent aspects, with applications respectively in industrial and rehabilitation domains. In the rst part safety issues are examined on a scenario in which a robot manipulator and a human perform the same task and in the same workspace. During the task execution the human should be able to get into contact with the robot and in this case an estimation algorithm of both interaction forces and contact point is proposed in order to guarantee safety conditions. At the same time, all the unintended contacts have to be avoided, and a suitable post collision strategy has been studied to move away the robot from the collision area or to reduce the impact e orts. However, the second part of the thesis focus on the cooperation between an orthesis and a patient. Indeed, in order to support a rehabilitation process, gait parameters, such as hip and knee angles or the beginning of a gait phase, have been estimated. For this purpose a sensor system, consisting of accelerometers and gyroscopes, and algorithms, developed in order to avoid the error accumulation due to the gyroscopes drift and the vibrations related to the beginning of the stance phase due to the accelerometers, have been proposed

Model-Based Control of Flying Robots for Robust Interaction under Wind Influence

Model-Based Control of Flying Robots for Robust Interaction under Wind Influence The main goal of this thesis is to bridge the gap between trajectory tracking and interaction control for flying robots in order to allow physical interaction under wind influence by making aerial robots aware of the disturbance, interaction, and faults acting on them. This is accomplished by reasoning about the external wrench (force and torque) acting on the robot, and discriminating (distinguishing) between wind, interactions, and collisions. This poses the following research questions. First, is discrimination between the external wrench components even possible in a continuous real-time fashion for control purposes? Second, given the individual wrench components, what are effective control schemes for interaction and trajectory tracking control under wind influence? Third, how can unexpected faults, such as collisions with the environment, be detected and handled efficiently and effectively? In the interest of the first question, a fourth can be posed: is it possible to obtain a measurement of the wind speed that is independent of the external wrench? In this thesis, model-based methods are applied in the pursuit of answers to these questions. This requires a good dynamics model of the robot, as well as accurately identified parameters. Therefore, a systematic parameter identification procedure for aerial robots is developed and applied. Furthermore, external wrench estimation techniques from the field of robot manipulators are extended to be suitable for aerial robots without the need of velocity measurements, which are difficult to obtain in this context. Based on the external wrench estimate, interaction control techniques (impedance and admittance control) are extended and applied to flying robots, and a thorough stability proof is provided. Similarly, the wrench estimate is applied in a geometric trajectory tracking controller to compensate external disturbances, to provide zero steady-state error under wind influence without the need of integral control action. The controllers are finally combined into a novel compensated impedance controller, to facilitate the main goal of the thesis. Collision detection is applied to flying robots, providing a low level reflex reaction that increases safety of these autonomous robots. In order to identify aerodynamic models for wind speed estimation, flight experiments in a three-dimensional wind tunnel were performed using a custom-built hexacopter. This data is used to investigate wind speed estimation using different data-driven aerodynamic models. It is shown that good performance can be obtained using relatively simple linear regression models. In this context, the propeller aerodynamic power model is used to obtain information about wind speed from available motor power measurements. Leveraging the wind tunnel data, it is shown that power can be used to obtain the wind speed. Furthermore, a novel optimization-based method that leverages the propeller aerodynamics model is developed to estimate the wind speed. Essentially, these two methods use the propellers as wind speed sensors, thereby providing an additional measurement independent of the external force. Finally, the novel topic of simultaneously discriminating between aerodynamic, interaction, and fault wrenches is opened up. This enables the implementation of novel types of controllers that are e.g. compliant to physical interaction, while compensating wind disturbances at the same time. The previously unexplored force discrimination topic has the potential to even open a new research avenue for flying robots

Mobile Robots

The objective of this book is to cover advances of mobile robotics and related technologies applied for multi robot systems' design and development. Design of control system is a complex issue, requiring the application of information technologies to link the robots into a single network. Human robot interface becomes a demanding task, especially when we try to use sophisticated methods for brain signal processing. Generated electrophysiological signals can be used to command different devices, such as cars, wheelchair or even video games. A number of developments in navigation and path planning, including parallel programming, can be observed. Cooperative path planning, formation control of multi robotic agents, communication and distance measurement between agents are shown. Training of the mobile robot operators is very difficult task also because of several factors related to different task execution. The presented improvement is related to environment model generation based on autonomous mobile robot observations

Contact aware robust semi-autonomous teleoperation of mobile manipulators

In the context of human-robot collaboration, cooperation and teaming, the use of mobile manipulators is widespread on applications involving unpredictable or hazardous environments for humans operators, like space operations, waste management and search and rescue on disaster scenarios. Applications where the manipulator's motion is controlled remotely by specialized operators. Teleoperation of manipulators is not a straightforward task, and in many practical cases represent a common source of failures. Common issues during the remote control of manipulators are: increasing control complexity with respect the mechanical degrees of freedom; inadequate or incomplete feedback to the user (i.e. limited visualization or knowledge of the environment); predefined motion directives may be incompatible with constraints or obstacles imposed by the environment. In the latter case, part of the manipulator may get trapped or blocked by some obstacle in the environment, failure that cannot be easily detected, isolated nor counteracted remotely. While control complexity can be reduced by the introduction of motion directives or by abstraction of the robot motion, the real-time constraint of the teleoperation task requires the transfer of the least possible amount of data over the system's network, thus limiting the number of physical sensors that can be used to model the environment. Therefore, it is of fundamental to define alternative perceptive strategies to accurately characterize different interaction with the environment without relying on specific sensory technologies. In this work, we present a novel approach for safe teleoperation, that takes advantage of model based proprioceptive measurement of the robot dynamics to robustly identify unexpected collisions or contact events with the environment. Each identified collision is translated on-the-fly into a set of local motion constraints, allowing the exploitation of the system redundancies for the computation of intelligent control laws for automatic reaction, without requiring human intervention and minimizing the disturbance of the task execution (or, equivalently, the operator efforts). More precisely, the described system consist in two different building blocks. The first, for detecting unexpected interactions with the environment (perceptive block). The second, for intelligent and autonomous reaction after the stimulus (control block). The perceptive block is responsible of the contact event identification. In short, the approach is based on the claim that a sensorless collision detection method for robot manipulators can be extended to the field of mobile manipulators, by embedding it within a statistical learning framework. The control deals with the intelligent and autonomous reaction after the contact or impact with the environment occurs, and consist on an motion abstraction controller with a prioritized set of constrains, where the highest priority correspond to the robot reconfiguration after a collision is detected; when all related dynamical effects have been compensated, the controller switch again to the basic control mode