A model-based residual approach for human-robot collaboration during manual polishing operations

A fully robotized polishing of metallic surfaces may be insufficient in case of parts with complex geometric shapes, where a manual intervention is still preferable. Within the EU SYMPLEXITY project, we are considering tasks where manual polishing operations are performed in strict physical Human-Robot Collaboration (HRC) between a robot holding the part and a human operator equipped with an abrasive tool. During the polishing task, the robot should firmly keep the workpiece in a prescribed sequence of poses, by monitoring and resisting to the external forces applied by the operator. However, the user may also wish to change the orientation of the part mounted on the robot, simply by pushing or pulling the robot body and changing thus its configuration. We propose a control algorithm that is able to distinguish the external torques acting at the robot joints in two components, one due to the polishing forces being applied at the end-effector level, the other due to the intentional physical interaction engaged by the human. The latter component is used to reconfigure the manipulator arm and, accordingly, its end-effector orientation. The workpiece position is kept instead fixed, by exploiting the intrinsic redundancy of this subtask. The controller uses a F/T sensor mounted at the robot wrist, together with our recently developed model-based technique (the residual method) that is able to estimate online the joint torques due to contact forces/torques applied at any place along the robot structure. In order to obtain a reliable residual, which is necessary to implement the control algorithm, an accurate robot dynamic model (including also friction effects at the joints and drive gains) needs to be identified first. The complete dynamic identification and the proposed control method for the human-robot collaborative polishing task are illustrated on a 6R UR10 lightweight manipulator mounting an ATI 6D sensor

Contact Estimation in Robot Interaction

In the paper, safety issues are examined in a scenario in which a robot manipulator and a human perform the same task in the same workspace. During the task execution, the human should be able to physically interact with the robot, and in this case an estimation algorithm for both interaction forces and a contact point is proposed in order to guarantee safety conditions. The method, starting from residual joint torque estimation, allows both direct and adaptive computation of the contact point and force, based on a principle of equivalence of the contact forces. At the same time, all the unintended contacts must be avoided, and a suitable post-collision strategy is considered to move the robot away from the collision area or else to reduce impact effects. Proper experimental tests have demonstrated the applicability in practice of both the post-impact strategy and the estimation algorithms; furthermore, experiments demonstrate the different behaviour resulting from the adaptation of the contact point as opposed to direct calculation

지도 및 비지도 학습을 이용한 로봇 머니퓰레이터 충돌 감지

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2022.2. 박종우.사람과 공유된 구조화되지 않은 동적 환경에서 작동하는 협업 로봇 머니퓰레이터는 날카로운 충돌(경성 충돌)에서 더 긴 지속 시간의 밀고 당기는 동작(연성 충돌)에 이르기까지의 다양한 충돌을 빠르고 정확하게 감지해야 한다. 모터 전류 측정값을 이용해 외부 조인트 토크를 추정하는 동역학 모델 기반 감지 방법을 사용할 경우, 정확한 마찰 파라미터 모델링 및 식별과 같은 모터 마찰에 대한 적절한 처리가 필요하다. 올바르게 적용하면 매우 효과적이지만, 동역학과 마찰 파라미터를 모델링 및 식별하고 여러 개의 감지 임계값을 수동으로 설정하는 데에는 상당한 노력이 필요하기 때문에 대량 생산되는 산업용 로봇에 이를 적용하기는 어렵다. 또한 적절한 식별 후에도 동역학에 백래시, 탄성 등 모델링되지 않은 효과나 불확실성이 여전히 존재할 수 있다. 본 논문에서는 순수 모델 기반 방법의 구현 어려움을 피하고 불확실한 동역학적 효과를 보상하는 수단으로 로봇 머니퓰레이터를 위한 총 네 가지의 학습 기반 충돌 감지 방법을 제안한다. 두 개의 방법은 학습을 위해 충돌 및 비충돌 동작 데이터가 모두 필요한 지도 학습 알고리즘(서포트 벡터 머신 회귀, 일차원 합성곱 신경망 기반)을 사용하며 나머지 두 개의 방법은 학습을 위해 비충돌 동작 데이터만을 필요로 하는 비지도 이상치 탐지 알고리즘(단일 클래스 서포트 벡터 머신, 오토인코더 기반)에 기반한다. 로봇 동역학 모델과 모터 전류 측정값만을 필요로 하며 추가적인 외부 센서나 마찰 모델링, 여러 개의 감지 임계값에 대한 수동 조정은 필요하지 않다. 먼저 지도 및 비지도 감지 방법을 학습시키고 검증하는 데 사용되는, 6자유도 협업 로봇 머니퓰레이터를 이용해 수집된 로봇 충돌 데이터를 설명한다. 우리가 고려하는 충돌 시나리오는 경성 충돌, 연성 충돌, 비충돌 동작으로, 경성 및 연성 충돌은 모두 동일하게 충돌로 간주한다. 감지 성능 검증을 위한 테스트 데이터는 총 787건의 충돌과 62.4분의 비충돌 동작으로 이루어져 있으며, 이는 로봇이 랜덤 점대점 6관절 동작을 수행하는 동안 수집된다. 데이터 수집 중 로봇의 끝단에는 미부착, 3.3 kg, 5.0 kg의 세 가지 유형의 페이로드를 부착한다. 다음으로, 수집된 테스트 데이터를 이용해 지도 감지 방법의 감지 성능을 실험적으로 검증한다. 실험 결과는 지도 감지 방법이 가벼운 네트워크를 이용해 광범위한 경성 및 연성 충돌을 실시간으로 정확하게 감지할 수 있음을 보여주며, 이를 통해 모델 파라미터의 불확실성과 측정 노이즈, 백래시, 변형 등 모델링되지 않은 효과까지 보상됨을 알 수 있다. 또한 서포트 벡터 머신 회귀 기반 방법은 하나의 감지 임계값에 대한 조정만 필요하며 일차원 합성곱 신경망 기반 방법은 하나의 아웃풋 필터 파라미터에 대한 조정만 필요한데, 두 방법 모두 직관적인 감도 조정이 가능하다. 나아가 일련의 시뮬레이션 실험을 통해 지도 감지 방법의 일반화 성능을 실험적으로 검증한다. 마지막으로, 동일한 테스트 데이터에 대해 비지도 감지 방법의 감지 성능과 일반화 성능 또한 검증한다. 실험 결과는 비지도 감지 방법 또한 가벼운 계산과 하나의 감지 임계값에 대한 조정만으로 다양한 경성 및 연성 충돌을 실시간으로 강인하게 감지할 수 있음을 보여주며, 이를 통해 모델링되지 않은 마찰을 포함한 불확실한 동역학적 효과를 비지도 학습으로도 보상할 수 있음을 알 수 있다. 지도 감지 방법이 더 나은 감지 성능을 보이지만, 비지도 감지 방법은 학습을 위해 비충돌 동작 데이터만을 필요로 하며 발생할 수 있는 모든 유형의 충돌에 대한 정보를 필요로 하지 않기 때문에 대량 생산되는 산업용 로봇에 더 적합하다.Collaborative robot manipulators operating in dynamic and unstructured environments shared with humans require fast and accurate detection of collisions, which can range from sharp impacts (hard collisions) to pulling and pushing motions of longer duration (soft collisions). When using dynamics model-based detection methods that estimate the external joint torque with motor current measurements, proper treatment for friction in the motors is required, such as accurate modeling and identification of friction parameters. Although highly effective when done correctly, modeling and identifying the dynamics and friction parameters, and manually setting multiple detection thresholds require considerable effort, making them difficult to be replicated for mass-produced industrial robots. There may also still exist unmodeled effects or uncertainties in the dynamics even after proper identification, e.g., backlash, elasticity. This dissertation presents a total of four learning-based collision detection methods for robot manipulators as a means of sidestepping some of the implementation difficulties of pure model-based methods and compensating for uncertain dynamic effects. Two methods use supervised learning algorithms – support vector machine regression and a one-dimensional convolutional neural network-based – that require both the collision and collision-free motion data for training. The other two methods are based on unsupervised anomaly detection algorithms – a one-class support vector machine and an autoencoder-based – that require only the collision-free motion data for training. Only the motor current measurements together with a robot dynamics model are required while no additional external sensors, friction modeling, or manual tuning of multiple detection thresholds are needed. We first describe the robot collision dataset collected with a six-dof collaborative robot manipulator, which is used for training and validating our supervised and unsupervised detection methods. The collision scenarios we consider are hard collisions, soft collisions, and collision-free, where both hard and soft collisions are treated in the same manner as just collisions. The test dataset for detection performance verification includes a total of 787 collisions and 62.4 minutes of collision-free motions, all collected while the robot is executing random point-to-point six-joint motions. During data collection, three types of payloads are attached to the end-effector: no payload, 3.3 kg payload, and 5.0 kg payload. Then the detection performance of our supervised detection methods is experimentally verified with the collected test dataset. Results demonstrate that our supervised detection methods can accurately detect a wide range of hard and soft collisions in real-time using a light network, compensating for uncertainties in the model parameters as well as unmodeled effects like friction, measurement noise, backlash, and deformations. Moreover, the SVMR-based method requires only one constant detection threshold to be tuned while the 1-D CNN-based method requires only one output filter parameter to be tuned, both of which allow intuitive sensitivity tuning. Furthermore, the generalization capability of our supervised detection methods is experimentally verified with a set of simulation experiments. Finally, our unsupervised detection methods are also validated for the same test dataset; the detection performance and the generalization capability are verified. The experimental results show that our unsupervised detection methods are also able to robustly detect a variety of hard and soft collisions in real-time with very light computation and with only one constant detection threshold required to be tuned, validating that uncertain dynamic effects including the unmodeled friction can be successfully compensated also with unsupervised learning. Although our supervised detection methods show better detection performance, our unsupervised detection methods are more practical for mass-produced industrial robots since they require only the data for collision-free motions for training, and the knowledge of every possible type of collision that can occur is not required.1 Introduction 1 1.1 Model-Free Methods 2 1.2 Model-Based Methods 2 1.3 Learning-Based Methods 4 1.3.1 Using Supervised Learning Algorithms 5 1.3.2 Using Unsupervised Learning Algorithms 6 1.4 Contributions of This Dissertation 7 1.4.1 Supervised Learning-Based Model-Compensating Detection 7 1.4.2 Unsupervised Learning-Based Model-Compensating Detection 8 1.4.3 Comparison with Existing Detection Methods 9 1.5 Organization of This Dissertation 14 2 Preliminaries 17 2.1 Introduction 17 2.2 Robot Dynamics 17 2.3 Momentum Observer-Based Collision Detection 19 2.4 Supervised Learning Algorithms 21 2.4.1 Support Vector Machine Regression 21 2.4.2 One-Dimensional Convolutional Neural Network 23 2.5 Unsupervised Anomaly Detection 25 2.6 One-Class Support Vector Machine 26 2.7 Autoencoder-Based Anomaly Detection 28 2.7.1 Autoencoder Network Architecture and Training 28 2.7.2 Anomaly Detection Using Autoencoders 29 3 Robot Collision Data 31 3.1 Introduction 31 3.2 True Collision Index Labeling 31 3.3 Collision Scenarios 35 3.4 Monitoring Signal 36 3.5 Signal Normalization and Sampling 37 3.6 Test Data for Detection Performance Verification 39 4 Supervised Learning-Based Model-Compensating Detection 43 4.1 Introduction 43 4.2 SVMR-Based Collision Detection 44 4.2.1 Input Feature Vector Design 44 4.2.2 SVMR Training 45 4.2.3 Collision Detection Sensitivity Adjustment 46 4.3 1-D CNN-Based Collision Detection 50 4.3.1 Network Input Design 50 4.3.2 Network Architecture and Training 50 4.3.3 An Output Filtering Method to Reduce False Alarms 53 4.4 Collision Detection Performance Criteria 54 4.4.1 Area Under the Precision-Recall Curve (PRAUC) 54 4.4.2 Detection Delay and Number of Detection Failures 54 4.5 Collision Detection Performance Analysis 56 4.5.1 Global Performance with Varying Thresholds 56 4.5.2 Detection Delay and Number of Detection Failures 57 4.5.3 Real-Time Inference 60 4.6 Generalization Capability Analysis 60 4.6.1 Generalization to Small Perturbations 60 4.6.2 Generalization to an Unseen Payload 62 5 Unsupervised Learning-Based Model-Compensating Detection 67 5.1 Introduction 67 5.2 OC-SVM-Based Collision Detection 68 5.2.1 Input Feature Vector 68 5.2.2 OC-SVM Training 70 5.2.3 Collision Detection with the Trained OC-SVM 70 5.3 Autoencoder-Based Collision Detection 70 5.3.1 Network Input and Output 71 5.3.2 Network Architecture and Training 71 5.3.3 Collision Detection with the Trained Autoencoder 72 5.4 Collision Detection Performance Analysis 74 5.4.1 Global Performance with Varying Thresholds 75 5.4.2 Detection Delay and Number of Detection Failures 75 5.4.3 Comparison with Supervised Learning-Based Methods 80 5.4.4 Real-Time Inference 83 5.5 Generalization Capability Analysis 83 5.5.1 Generalization to Small Perturbations 84 5.5.2 Generalization to an Unseen Payload 85 6 Conclusion 89 6.1 Summary and Discussion 89 6.2 Future Work 93 A Appendix 95 A.1 SVM-Based Classification of Detected Collisions 95 A.2 Direct Estimation-Based Detection Methods 97 A.3 Model-Independent Supervised Detection Methods 101 A.4 Generalization to Large Changes in the Dynamics Model 102 Bibliography 106 Abstract 112박

Collision Detection and Contact Point Estimation Using Virtual Joint Torque Sensing Applied to a Cobot

In physical human-robot interaction (pHRI) it is essential to reliably estimate and localize contact forces between the robot and the environment. In this paper, a complete contact detection, isolation, and reaction scheme is presented and tested on a new 6-dof industrial collaborative robot. We combine two popular methods, based on monitoring energy and generalized momentum, to detect and isolate collisions on the whole robot body in a more robust way. The experimental results show the effectiveness of our implementation on the LARA5 cobot, that only relies on motor current and joint encoder measurements. For validation purposes, contact forces are also measured using an external GTE CoboSafe sensor. After a successful collision detection, the contact point location is isolated using a combination of the residual method based on the generalized momentum with a contact particle filter (CPF) scheme. We show for the first time a successful implementation of such combination on a real robot, without relying on joint torque sensor measurements

NASA Center for Intelligent Robotic Systems for Space Exploration

NASA's program for the civilian exploration of space is a challenge to scientists and engineers to help maintain and further develop the United States' position of leadership in a focused sphere of space activity. Such an ambitious plan requires the contribution and further development of many scientific and technological fields. One research area essential for the success of these space exploration programs is Intelligent Robotic Systems. These systems represent a class of autonomous and semi-autonomous machines that can perform human-like functions with or without human interaction. They are fundamental for activities too hazardous for humans or too distant or complex for remote telemanipulation. To meet this challenge, Rensselaer Polytechnic Institute (RPI) has established an Engineering Research Center for Intelligent Robotic Systems for Space Exploration (CIRSSE). The Center was created with a five year $5.5 million grant from NASA submitted by a team of the Robotics and Automation Laboratories. The Robotics and Automation Laboratories of RPI are the result of the merger of the Robotics and Automation Laboratory of the Department of Electrical, Computer, and Systems Engineering (ECSE) and the Research Laboratory for Kinematics and Robotic Mechanisms of the Department of Mechanical Engineering, Aeronautical Engineering, and Mechanics (ME,AE,&M), in 1987. This report is an examination of the activities that are centered at CIRSSE

Motion Planning for the On-orbit Grasping of a Non-cooperative Target Satellite with Collision Avoidance

A method for grasping a tumbling noncooperative target is presented, which is based on nonlinear optimization and collision avoidance. Motion constraints on the robot joints as well as on the end-effector forces are considered. Cost functions of interest address the robustness of the planned solutions during the tracking phase as well as actuation energy. The method is applied in simulation to different operational scenarios