Model Based Teleoperation to Eliminate Feedback Delay NSF Grant BCS89-01352 First Report

We are conducting research in the area of teleoperation with feedback delay. Delay occurs with earth-based teleoperation in space and with surface-based teleoperation with untethered submersibles when acoustic communication links are involved. the delay in obtaining position and force feedback from remote slave arms makes teleoperation extremely difficult. We are proposing a novel combination of graphics and manipulator programming to solve the problem by interfacing a teleoperator master arm to a graphics based simulator of the remote environment coupled with a robot manipulator at the remote, delayed site. the operator\u27s actions will be monitored to provide both kinesthetic and visual feedback and to generate symbolic motion commands to the remote slave. the slave robot will then execute these symbolic commands delayed in time. While much of a task will proceed error free, when an error does occur the slave system will transmit data back to the master and the master environment will be reset to the error state

An integrated system for autonomous robotics manipulation

We describe the software components of a robotics system designed to autonomously grasp objects and perform dexterous manipulation tasks with only high-level supervision. The system is centered on the tight integration of several core functionalities, including perception, planning and control, with the logical structuring of tasks driven by a Behavior Tree architecture. The advantage of the implementation is to reduce the execution time while integrating advanced algorithms for autonomous manipulation. We describe our approach to 3-D perception, real-time planning, force compliant motions, and audio processing. Performance results for object grasping and complex manipulation tasks of in-house tests and of an independent evaluation team are presented

Al-Robotics team: A cooperative multi-unmanned aerial vehicle approach for the Mohamed Bin Zayed International Robotic Challenge

The Al-Robotics team was selected as one of the 25 finalist teams out of 143 applications received to participate in the first edition of the Mohamed Bin Zayed International Robotic Challenge (MBZIRC), held in 2017. In particular, one of the competition Challenges offered us the opportunity to develop a cooperative approach with multiple unmanned aerial vehicles (UAVs) searching, picking up, and dropping static and moving objects. This paper presents the approach that our team Al-Robotics followed to address that Challenge 3 of the MBZIRC. First, we overview the overall architecture of the system, with the different modules involved. Second, we describe the procedure that we followed to design the aerial platforms, as well as all their onboard components. Then, we explain the techniques that we used to develop the software functionalities of the system. Finally, we discuss our experimental results and the lessons that we learned before and during the competition. The cooperative approach was validated with fully autonomous missions in experiments previous to the actual competition. We also analyze the results that we obtained during the competition trials.Unión Europea H2020 73166

Model Based Teleoperation to Eliminate Feedback Delay NSF Grant BCS89-01352 Second Report

We are conducting research in the area of teleoperation with feedback delay. Delay occurs with earth-based teleoperation in space and with surface-based teleoperation with untethered submersibles when acoustic communication links are involved. The delay in obtaining position and force feedback from remote slave arms makes teleoperation extremely difficult leading to very low productivity. We have combined computer graphics with manipulator programming to provide a solution to the problem. A teleoperator master arm is interfaced to a graphics based simulator of the remote environment. The system is then coupled with a robot manipulator at the remote, delayed site. The operator\u27s actions are monitored to provide both kinesthetic and visual feedback and to generate symbolic motion commands to the remote slave. The slave robot then executes these symbolic commands delayed in time. While much of a task proceeds error free, when an error does occur, the slave system transmits data back to the master environment which is then reset to the error state from which the operator continues the task

불확실성을 포함하는 조립작업을 위한 컴플라이언스 기반 펙인홀 전략

학위논문 (박사) -- 서울대학교 대학원 : 융합과학기술대학원 융합과학부(지능형융합시스템전공), 2020. 8. 박재흥.The peg-in-hole assembly is a representative robotic task that involves physical contact with the external environment. The strategies generally involve performing the assembly task by estimating the contact state between the peg and the hole. The contact forces and moments, measured using force sensors, are primarily used to estimate the contact state. In this paper, in contrast to past research in the area, which has involved the utilization of such expensive devices as force/torque sensors or remote compliance mechanisms, an inexpensive method is proposed for peg-in-hole assembly without force feedback or passive compliance mechanisms. The method consists of an analysis of the state of contact between the peg and the hole as well as a strategy to overcome the inevitable positional uncertainty of the hole incurred in the recognition process. A control scheme was developed to yield compliant behavior from the robot with physical contact under the condition of hybrid position/force control. Proposed peg-in-hole strategy is based on compliance characteristics and generating the force and moment. The peg is inserted into the hole as it adapts to the external environment. The effectiveness of the proposed method was experimentally verified using a humanoid upper body robot with fifty degrees of freedom and a peg-in-hole apparatus with a small clearance (0.1 mm). Three cases of experiments were conducted; Assembling the peg attached to the arm in the hole fixed in the external environment, grasping a peg with an anthropomorphic hand and assembling it into a fixed hole, and grasping both peg and hole with both hands and assembling each other. In order to assemble the peg-in-hole through the proposed strategy by the humanoid upper body robot, I present a method of gripping an object, estimating the kinematics of the gripped object, and manipulating the gripped object. In addition to the cost aspect, which is the fundamental motivation for the proposed strategy, the experimental results show that the proposed strategy has advantages such as fast assembly time and high success rate, but has the disadvantage of unpredictable elapsed time. The reason for having a high variance value for the success time is that the spiral trajectory, which is most commonly used, is used. In this study, I analyze the efficiency of spiral force trajectory and propose an improved force trajectory. The proposed force trajectory reduces the distribution of elapsed time by eliminating the uncertainty in the time required to find a hole. The efficiency of the force trajectory is analyzed numerically, verified through repeated simulations, and verified by the actual experiment with humanoid upper body robot developed by Korea institute of industrial technology.펙인홀 조립은 로봇의 접촉 작업을 대표하는 작업으로, 펙인홀 조립 전략을 연구함으로써 산업 생산 분야의 조립작업에 적용할 수 있다. 펙인홀 조립작업은 일반적으로 펙과 홀 간의 접촉상태를 추정함으로써 이루어진다. 접촉상태를 추정하기 위해 가장 널리 쓰이는 방법은 힘 센서를 사용하는 것인데, 접촉 힘과 모멘트를 측정하여 접촉상태를 추정하는 방식이다. 만약 이러한 센서를 사용하지 않을 수 있다면, 하드웨어 비용과 소프트웨어 연산량 감소 등의 장점이 있음은 자명하다. 본 논문에서는 힘 센서 혹은 수동 컴플라이언스 장치를 사용하지 않는 펙인홀 전략을 제안한다. 홀에 대한 인식 오차 혹은 로봇의 제어 오차를 극복하기 위하여 먼저 펙과 홀의 접촉 가능 상태를 분석하고 로봇의 컴플라이언스 모션을 위한 제어 프레임워크를 디자인한다. 전략은 컴플라이언스 특징에 기반하며 펙에 힘과 모멘트를 생성시킴으로써 조립작업을 수행한다. 펙은 외부환경에 순응함으로써 홀에 삽입된다. 제안한 전략은 낮은 공차를 갖는 펙인홀 실험을 통해서 그 유효성이 검증된다. 펙과 홀을 로봇팔과 외부환경에 각각 고정된 환경에서의 실험, 인간형 로봇핸드를 이용하여 펙을 잡아서 고정된 홀에 삽입하는 실험, 그리고 테이블에 놓인 펙과 홀을 각각 로봇핸드로 파지하여 조립하는 총 세 가지의 실험을 수행하였다. 핸드로 펙을 파지하고 조작하기 위하여, 파지 방법과 핸드를 이용한 물체 조작 알고리즘을 간략히 소개하였다. 제안한 전략의 성능을 실험적으로 분석한 결과, 높은 조립 성공률을 갖는 대신 조립시간이 예측할 수 없는 단점이 나타나 이를 보완하기 위해서 렌치 궤적 또한 제안하였다. 먼저 가장 일반적으로 사용되는 나선 힘 궤적을 이용했을 때 조립 성공시간의 분산이 큰 이유를 확률개념을 이용해 분석하고, 이를 보완하기 위한 부분적 나선 힘 궤적을 제안한다. 제안한 힘 궤적이 나선 힘 궤적에 비해 갖는 성능의 우수성을 증명하기 위하여 수치적 분석, 반복적 시뮬레이션, 그리고 로봇을 이용한 실험을 수행하였다.1 INTRODUCTION 1 1.1 Motivation: Peg-in-Hole Assembly 1 1.2 Contributions of Thesis 2 1.3 Overview of Thesis 3 2 COMPLIANCE BASED STRATEGY 5 2.1 Background & Related Works 5 2.2 Analysis of Peg-in-Hole Procedure 6 2.2.1 Contact Analysis 7 2.2.2 Basic Idea 9 2.3 Peg-in-Hole Strategy 12 2.3.1 Unit Motions 12 2.3.2 State of Strategy 13 2.3.3 Conditions for State Transition 15 2.4 Control Frameworks 18 2.4.1 Control for Compliant Behavior 18 2.4.2 Friction Compensate 20 2.4.3 Control Input for the Strategy 25 2.5 Experiment 29 2.5.1 Experiment Environment 29 2.5.2 Fixed Peg and Fixed Hole 31 2.5.2.1 Experiment Results 31 2.5.2.2 Analysis of Force and Control Gain 36 2.5.3 Peg-in-Hole with Multi Finger Hand 41 2.5.3.1 Object Grasping 42 2.5.3.2 Object In-Hand Manipulation 44 2.5.3.3 Experiment Results 49 2.5.4 With Upper Body Robot 50 2.5.4.1 Peg-in-Hole Procedure 52 2.5.4.2 Kinematics of Grasped Object 54 2.5.4.3 Control Frameworks 54 2.5.4.4 Experiment Results 56 2.6 Discussion 59 2.6.1 Peg-in-Hole Transition 59 2.6.2 Influential Issues 59 3 WRENCH TRAJECTORY 63 3.1 Problem Statement 64 3.1.1 Hole Search Process 64 3.1.2 Spiral Force Trajectory Analysis 66 3.2 Partial Spiral Force Trajectory 70 3.2.1 Force Trajectory with Tilted Posture 70 3.2.2 Probability to Three-point Contact 76 3.3 SIMULATION & EXPERIMENT 78 3.3.1 Simulation 78 3.3.2 Experiment 83 4 CONCLUSIONS 90 Abstract (In Korean) 102Docto

Online Robot Introspection via Wrench-based Action Grammars

Robotic failure is all too common in unstructured robot tasks. Despite well-designed controllers, robots often fail due to unexpected events. How do robots measure unexpected events? Many do not. Most robots are driven by the sense-plan act paradigm, however more recently robots are undergoing a sense-plan-act-verify paradigm. In this work, we present a principled methodology to bootstrap online robot introspection for contact tasks. In effect, we are trying to enable the robot to answer the question: what did I do? Is my behavior as expected or not? To this end, we analyze noisy wrench data and postulate that the latter inherently contains patterns that can be effectively represented by a vocabulary. The vocabulary is generated by segmenting and encoding the data. When the wrench information represents a sequence of sub-tasks, we can think of the vocabulary forming a sentence (set of words with grammar rules) for a given sub-task; allowing the latter to be uniquely represented. The grammar, which can also include unexpected events, was classified in offline and online scenarios as well as for simulated and real robot experiments. Multiclass Support Vector Machines (SVMs) were used offline, while online probabilistic SVMs were are used to give temporal confidence to the introspection result. The contribution of our work is the presentation of a generalizable online semantic scheme that enables a robot to understand its high-level state whether nominal or abnormal. It is shown to work in offline and online scenarios for a particularly challenging contact task: snap assemblies. We perform the snap assembly in one-arm simulated and real one-arm experiments and a simulated two-arm experiment. This verification mechanism can be used by high-level planners or reasoning systems to enable intelligent failure recovery or determine the next most optima manipulation skill to be used.Comment: arXiv admin note: substantial text overlap with arXiv:1609.0494

Motion Primitives and Planning for Robots with Closed Chain Systems and Changing Topologies

When operating in human environments, a robot should use predictable motions that allow humans to trust and anticipate its behavior. Heuristic search-based planning offers predictable motions and guarantees on completeness and sub-optimality of solutions. While search-based planning on motion primitive-based (lattice-based) graphs has been used extensively in navigation, application to high-dimensional state-spaces has, until recently, been thought impractical. This dissertation presents methods we have developed for applying these graphs to mobile manipulation, specifically for systems which contain closed chains. The formation of closed chains in tasks that involve contacts with the environment may reduce the number of available degrees-of-freedom but adds complexity in terms of constraints in the high-dimensional state-space. We exploit the dimensionality reduction inherent in closed kinematic chains to get efficient search-based planning. Our planner handles changing topologies (switching between open and closed-chains) in a single plan, including what transitions to include and when to include them. Thus, we can leverage existing results for search-based planning for open chains, combining open and closed chain manipulation planning into one framework. Proofs regarding the framework are introduced for the application to graph-search and its theoretical guarantees of optimality. The dimensionality-reduction is done in a manner that enables finding optimal solutions to low-dimensional problems which map to correspondingly optimal full-dimensional solutions. We apply this framework to planning for opening and navigating through non-spring and spring-loaded doors using a Willow Garage PR2. The framework motivates our approaches to the Atlas humanoid robot from Boston Dynamics for both stationary manipulation and quasi-static walking, as a closed chain is formed when both feet are on the ground

System Architectures for Cooperative Teams of Unmanned Aerial Vehicles Interacting Physically with the Environment

Unmanned Aerial Vehicles (UAVs) have become quite a useful tool for a wide range of applications, from inspection & maintenance to search & rescue, among others. The capabilities of a single UAV can be extended or complemented by the deployment of more UAVs, so multi-UAV cooperative teams are becoming a trend. In that case, as di erent autopilots, heterogeneous platforms, and application-dependent software components have to be integrated, multi-UAV system architectures that are fexible and can adapt to the team's needs are required. In this thesis, we develop system architectures for cooperative teams of UAVs, paying special attention to applications that require physical interaction with the environment, which is typically unstructured. First, we implement some layers to abstract the high-level components from the hardware speci cs. Then we propose increasingly advanced architectures, from a single-UAV hierarchical navigation architecture to an architecture for a cooperative team of heterogeneous UAVs. All this work has been thoroughly tested in both simulation and eld experiments in di erent challenging scenarios through research projects and robotics competitions. Most of the applications required physical interaction with the environment, mainly in unstructured outdoors scenarios. All the know-how and lessons learned throughout the process are shared in this thesis, and all relevant code is publicly available.Los vehículos aéreos no tripulados (UAVs, del inglés Unmanned Aerial Vehicles) se han convertido en herramientas muy valiosas para un amplio espectro de aplicaciones, como inspección y mantenimiento, u operaciones de rescate, entre otras. Las capacidades de un único UAV pueden verse extendidas o complementadas al utilizar varios de estos vehículos simultáneamente, por lo que la tendencia actual es el uso de equipos cooperativos con múltiples UAVs. Para ello, es fundamental la integración de diferentes autopilotos, plataformas heterogéneas, y componentes software -que dependen de la aplicación-, por lo que se requieren arquitecturas multi-UAV que sean flexibles y adaptables a las necesidades del equipo. En esta tesis, se desarrollan arquitecturas para equipos cooperativos de UAVs, prestando una especial atención a aplicaciones que requieran de interacción física con el entorno, cuya naturaleza es típicamente no estructurada. Primero se proponen capas para abstraer a los componentes de alto nivel de las particularidades del hardware. Luego se desarrollan arquitecturas cada vez más avanzadas, desde una arquitectura de navegación para un único UAV, hasta una para un equipo cooperativo de UAVs heterogéneos. Todo el trabajo ha sido minuciosamente probado, tanto en simulación como en experimentos reales, en diferentes y complejos escenarios motivados por proyectos de investigación y competiciones de robótica. En la mayoría de las aplicaciones se requería de interacción física con el entorno, que es normalmente un escenario en exteriores no estructurado. A lo largo de la tesis, se comparten todo el conocimiento adquirido y las lecciones aprendidas en el proceso, y el código relevante está publicado como open-source

Study to design and develop remote manipulator system

Modeling of human performance in remote manipulation tasks is reported by automated procedures using computers to analyze and count motions during a manipulation task. Performance is monitored by an on-line computer capable of measuring the joint angles of both master and slave and in some cases the trajectory and velocity of the hand itself. In this way the operator's strategies with different transmission delays, displays, tasks, and manipulators can be analyzed in detail for comparison. Some progress is described in obtaining a set of standard tasks and difficulty measures for evaluating manipulator performance

Interação humano-robô para a transferência de objetos

Mestrado em Engenharia MecânicaRobots come into physical contact with humans under a variety of circumstances to perform useful work. This thesis has the ambitious aim of contriving a solution that leads to a simple case of physical human-robot interaction, an object transfer task. Firstly, this work presents a review of the current research within the field of Human-Robot Interaction, where two approaches are distinguished, but simultaneously required: a pre-contact approximation and an interaction by contact. Further, to achieve the proposed objectives, this dissertation addresses a possible answer to three major problems: (1) The robot control to perform the inherent movements of the transfer assignment, (2) the human-robot pre interaction and (3) the interaction by contact. The capabilities of a 3D sensor and force/tactile sensors are explored in order to prepare the robot to handover an object and to control the robot gripper actions, correspondingly. The complete software development is supported by the Robot Operating System (ROS) framework. Finally, some experimental tests are conducted to validate the proposed solutions and to evaluate the system's performance. A possible transfer task is achieved, even if some refinements, improvements and extensions are required to improve the solution performance and range.Os robôs entram em contacto físico com os humanos sob uma variedade de circunstâncias para realizar trabalho útil. Esta dissertação tem como objetivo o desenvolvimento de uma solução que permita um caso simples de interação física humano-robô, uma tarefa de transferência de objetos. Inicialmente, este trabalho apresenta uma revisão da pesquisa corrente na área da interação humano-robô, onde duas abordagens são distinguíveis, mas simultaneamente necessárias: uma aproximação pré-contacto e uma interação pós-contacto. Seguindo esta linha de pensamento, para atingir os objetivos propostos, esta dissertação procura dar resposta a três grandes problemas: (1) O controlo do robô para que este desempenhe os movimentos inerentes á tarefa de transferência, (2) a pré-interação humano-robô e (3) a interação por contacto. As capacidades de um sensor 3D e de sensores de força são exploradas com o objetivo de preparar o robô para a transferência e de controlar as ações da garra robótica, correspondentemente. O desenvolvimento de arquitetura software é suportado pela estrutura Robot Operating System (ROS). Finalmente, alguns testes experimentais são realizados para validar as soluções propostas e para avaliar o desempenho do sistema. Uma possível transferência de objetos é alcançada, mesmo que sejam necessários alguns refinamentos, melhorias e extensões para melhorar o desempenho e abrangência da solução