불확실성을 포함하는 조립작업을 위한 컴플라이언스 기반 펙인홀 전략

학위논문 (박사) -- 서울대학교 대학원 : 융합과학기술대학원 융합과학부(지능형융합시스템전공), 2020. 8. 박재흥.The peg-in-hole assembly is a representative robotic task that involves physical contact with the external environment. The strategies generally involve performing the assembly task by estimating the contact state between the peg and the hole. The contact forces and moments, measured using force sensors, are primarily used to estimate the contact state. In this paper, in contrast to past research in the area, which has involved the utilization of such expensive devices as force/torque sensors or remote compliance mechanisms, an inexpensive method is proposed for peg-in-hole assembly without force feedback or passive compliance mechanisms. The method consists of an analysis of the state of contact between the peg and the hole as well as a strategy to overcome the inevitable positional uncertainty of the hole incurred in the recognition process. A control scheme was developed to yield compliant behavior from the robot with physical contact under the condition of hybrid position/force control. Proposed peg-in-hole strategy is based on compliance characteristics and generating the force and moment. The peg is inserted into the hole as it adapts to the external environment. The effectiveness of the proposed method was experimentally verified using a humanoid upper body robot with fifty degrees of freedom and a peg-in-hole apparatus with a small clearance (0.1 mm). Three cases of experiments were conducted; Assembling the peg attached to the arm in the hole fixed in the external environment, grasping a peg with an anthropomorphic hand and assembling it into a fixed hole, and grasping both peg and hole with both hands and assembling each other. In order to assemble the peg-in-hole through the proposed strategy by the humanoid upper body robot, I present a method of gripping an object, estimating the kinematics of the gripped object, and manipulating the gripped object. In addition to the cost aspect, which is the fundamental motivation for the proposed strategy, the experimental results show that the proposed strategy has advantages such as fast assembly time and high success rate, but has the disadvantage of unpredictable elapsed time. The reason for having a high variance value for the success time is that the spiral trajectory, which is most commonly used, is used. In this study, I analyze the efficiency of spiral force trajectory and propose an improved force trajectory. The proposed force trajectory reduces the distribution of elapsed time by eliminating the uncertainty in the time required to find a hole. The efficiency of the force trajectory is analyzed numerically, verified through repeated simulations, and verified by the actual experiment with humanoid upper body robot developed by Korea institute of industrial technology.펙인홀 조립은 로봇의 접촉 작업을 대표하는 작업으로, 펙인홀 조립 전략을 연구함으로써 산업 생산 분야의 조립작업에 적용할 수 있다. 펙인홀 조립작업은 일반적으로 펙과 홀 간의 접촉상태를 추정함으로써 이루어진다. 접촉상태를 추정하기 위해 가장 널리 쓰이는 방법은 힘 센서를 사용하는 것인데, 접촉 힘과 모멘트를 측정하여 접촉상태를 추정하는 방식이다. 만약 이러한 센서를 사용하지 않을 수 있다면, 하드웨어 비용과 소프트웨어 연산량 감소 등의 장점이 있음은 자명하다. 본 논문에서는 힘 센서 혹은 수동 컴플라이언스 장치를 사용하지 않는 펙인홀 전략을 제안한다. 홀에 대한 인식 오차 혹은 로봇의 제어 오차를 극복하기 위하여 먼저 펙과 홀의 접촉 가능 상태를 분석하고 로봇의 컴플라이언스 모션을 위한 제어 프레임워크를 디자인한다. 전략은 컴플라이언스 특징에 기반하며 펙에 힘과 모멘트를 생성시킴으로써 조립작업을 수행한다. 펙은 외부환경에 순응함으로써 홀에 삽입된다. 제안한 전략은 낮은 공차를 갖는 펙인홀 실험을 통해서 그 유효성이 검증된다. 펙과 홀을 로봇팔과 외부환경에 각각 고정된 환경에서의 실험, 인간형 로봇핸드를 이용하여 펙을 잡아서 고정된 홀에 삽입하는 실험, 그리고 테이블에 놓인 펙과 홀을 각각 로봇핸드로 파지하여 조립하는 총 세 가지의 실험을 수행하였다. 핸드로 펙을 파지하고 조작하기 위하여, 파지 방법과 핸드를 이용한 물체 조작 알고리즘을 간략히 소개하였다. 제안한 전략의 성능을 실험적으로 분석한 결과, 높은 조립 성공률을 갖는 대신 조립시간이 예측할 수 없는 단점이 나타나 이를 보완하기 위해서 렌치 궤적 또한 제안하였다. 먼저 가장 일반적으로 사용되는 나선 힘 궤적을 이용했을 때 조립 성공시간의 분산이 큰 이유를 확률개념을 이용해 분석하고, 이를 보완하기 위한 부분적 나선 힘 궤적을 제안한다. 제안한 힘 궤적이 나선 힘 궤적에 비해 갖는 성능의 우수성을 증명하기 위하여 수치적 분석, 반복적 시뮬레이션, 그리고 로봇을 이용한 실험을 수행하였다.1 INTRODUCTION 1 1.1 Motivation: Peg-in-Hole Assembly 1 1.2 Contributions of Thesis 2 1.3 Overview of Thesis 3 2 COMPLIANCE BASED STRATEGY 5 2.1 Background & Related Works 5 2.2 Analysis of Peg-in-Hole Procedure 6 2.2.1 Contact Analysis 7 2.2.2 Basic Idea 9 2.3 Peg-in-Hole Strategy 12 2.3.1 Unit Motions 12 2.3.2 State of Strategy 13 2.3.3 Conditions for State Transition 15 2.4 Control Frameworks 18 2.4.1 Control for Compliant Behavior 18 2.4.2 Friction Compensate 20 2.4.3 Control Input for the Strategy 25 2.5 Experiment 29 2.5.1 Experiment Environment 29 2.5.2 Fixed Peg and Fixed Hole 31 2.5.2.1 Experiment Results 31 2.5.2.2 Analysis of Force and Control Gain 36 2.5.3 Peg-in-Hole with Multi Finger Hand 41 2.5.3.1 Object Grasping 42 2.5.3.2 Object In-Hand Manipulation 44 2.5.3.3 Experiment Results 49 2.5.4 With Upper Body Robot 50 2.5.4.1 Peg-in-Hole Procedure 52 2.5.4.2 Kinematics of Grasped Object 54 2.5.4.3 Control Frameworks 54 2.5.4.4 Experiment Results 56 2.6 Discussion 59 2.6.1 Peg-in-Hole Transition 59 2.6.2 Influential Issues 59 3 WRENCH TRAJECTORY 63 3.1 Problem Statement 64 3.1.1 Hole Search Process 64 3.1.2 Spiral Force Trajectory Analysis 66 3.2 Partial Spiral Force Trajectory 70 3.2.1 Force Trajectory with Tilted Posture 70 3.2.2 Probability to Three-point Contact 76 3.3 SIMULATION & EXPERIMENT 78 3.3.1 Simulation 78 3.3.2 Experiment 83 4 CONCLUSIONS 90 Abstract (In Korean) 102Docto

Collaborative and Cooperative Robotics Applications using Visual Perception

The objective of this Thesis is to develop novel integrated strategies for collaborative and cooperative robotic applications. Commonly, industrial robots operate in structured environments and in work-cell separated from human operators. Nowadays, collaborative robots have the capacity of sharing the workspace and collaborate with humans or other robots to perform complex tasks. These robots often operate in an unstructured environment, whereby they need sensors and algorithms to get information about environment changes. Advanced vision and control techniques have been analyzed to evaluate their performance and their applicability to industrial tasks. Then, some selected techniques have been applied for the first time to an industrial context. A Peg-in-Hole task has been chosen as first case study, since it has been extensively studied but still remains challenging: it requires accuracy both in the determination of the hole poses and in the robot positioning. Two solutions have been developed and tested. Experimental results have been discussed to highlight the advantages and disadvantages of each technique. Grasping partially known objects in unstructured environments is one of the most challenging issues in robotics. It is a complex task and requires to address multiple subproblems, in order to be accomplished, including object localization and grasp pose detection. Also for this class of issues some vision techniques have been analyzed. One of these has been adapted to be used in industrial scenarios. Moreover, as a second case study, a robot-to-robot object handover task in a partially structured environment and in the absence of explicit communication between the robots has been developed and validated. Finally, the two case studies have been integrated in two real industrial setups to demonstrate the applicability of the strategies to solving industrial problems

Vision-enhanced Peg-in-Hole for automotive body parts using semantic image segmentation and object detection

Artificial Intelligence (AI) is an enabling technology in the context of Industry 4.0. In particular, the automotive sector is among those who can benefit most of the use of AI in conjunction with advanced vision techniques. The scope of this work is to integrate deep learning algorithms in an industrial scenario involving a robotic Peg-in-Hole task. More in detail, we focus on a scenario where a human operator manually positions a carbon fiber automotive part in the workspace of a 7 Degrees of Freedom (DOF) manipulator. To cope with the uncertainty on the relative position between the robot and the workpiece, we adopt a three stage strategy. The first stage concerns the Three-Dimensional (3D) reconstruction of the workpiece using a registration algorithm based on the Iterative Closest Point (ICP) paradigm. Such a procedure is integrated with a semantic image segmentation neural network, which is in charge of removing the background of the scene to improve the registration. The adoption of such network allows to reduce the registration time of about 28.8%. In the second stage, the reconstructed surface is compared with a Computer Aided Design (CAD) model of the workpiece to locate the holes and their axes. In this stage, the adoption of a Convolutional Neural Network (CNN) allows to improve the holes’ position estimation of about 57.3%. The third stage concerns the insertion of the peg by implementing a search phase to handle the remaining estimation errors. Also in this case, the use of the CNN reduces the search phase duration of about 71.3%. Quantitative experiments, including a comparison with a previous approach without both the segmentation network and the CNN, have been conducted in a realistic scenario. The results show the effectiveness of the proposed approach and how the integration of AI techniques improves the success rate from 84.5% to 99.0%

CFVS: Coarse-to-Fine Visual Servoing for 6-DoF Object-Agnostic Peg-In-Hole Assembly

Robotic peg-in-hole assembly remains a challenging task due to its high accuracy demand. Previous work tends to simplify the problem by restricting the degree of freedom of the end-effector, or limiting the distance between the target and the initial pose position, which prevents them from being deployed in real-world manufacturing. Thus, we present a Coarse-to-Fine Visual Servoing (CFVS) peg-in-hole method, achieving 6-DoF end-effector motion control based on 3D visual feedback. CFVS can handle arbitrary tilt angles and large initial alignment errors through a fast pose estimation before refinement. Furthermore, by introducing a confidence map to ignore the irrelevant contour of objects, CFVS is robust against noise and can deal with various targets beyond training data. Extensive experiments show CFVS outperforms state-of-the-art methods and obtains 100%, 91%, and 82% average success rates in 3-DoF, 4-DoF, and 6-DoF peg-in-hole, respectively

Automated freeform assembly of threaded fasteners

Over the past two decades, a major part of the manufacturing and assembly market has been driven by its customer requirements. Increasing customer demand for personalised products create the demand for smaller batch sizes, shorter production times, lower costs, and the flexibility to produce families of products - or different parts - with the same sets of equipment. Consequently, manufacturing companies have deployed various automation systems and production strategies to improve their resource efficiency and move towards right-first-time production. However, many of these automated systems, which are involved with robot-based, repeatable assembly automation, require component- specific fixtures for accurate positioning and extensive robot programming, to achieve flexibility in their production. Threaded fastening operations are widely used in assembly. In high-volume production, the fastening processes are commonly automated using jigs, fixtures, and semi-automated tools. This form of automation delivers reliable assembly results at the expense of flexibility and requires component variability to be adequately controlled. On the other hand, in low- volume, high- value manufacturing, fastening processes are typically carried out manually by skilled workers. This research is aimed at addressing the aforementioned issues by developing a freeform automated threaded fastener assembly system that uses 3D visual guidance. The proof-of-concept system developed focuses on picking up fasteners from clutter, identifying a hole feature in an imprecisely positioned target component and carry out torque-controlled fastening. This approach has achieved flexibility and adaptability without the use of dedicated fixtures and robot programming. This research also investigates and evaluates different 3D imaging technology to identify the suitable technology required for fastener assembly in a non-structured industrial environment. The proposed solution utilises the commercially available technologies to enhance the precision and speed of identification of components for assembly processes, thereby improving and validating the possibility of reliably implementing this solution for industrial applications. As a part of this research, a number of novel algorithms are developed to robustly identify assembly components located in a random environment by enhancing the existing methods and technologies within the domain of the fastening processes. A bolt identification algorithm was developed to identify bolts located in a random clutter by enhancing the existing surface-based matching algorithm. A novel hole feature identification algorithm was developed to detect threaded holes and identify its size and location in 3D. The developed bolt and feature identification algorithms are robust and has sub-millimetre accuracy required to perform successful fastener assembly in industrial conditions. In addition, the processing time required for these identification algorithms - to identify and localise bolts and hole features - is less than a second, thereby increasing the speed of fastener assembly