분산 제약하에서 원격 제어되는 다수의 논홀로노믹 이동형 로봇 대형 재구성 제어

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 기계항공공학부, 2019. 2. 이동준.본 논문에서는 변화하는 주행 환경에서 분산 제약 하에 다수의 원격으로 제어되는 논홀로노믹 이동형 로봇 대형 재구성 제어에 대한 새로운 프레임워크를 제시하였다. 센싱과 컴퓨팅 능력이 갖추어진 온보드 시스템 로봇들을 활용하여 최근 개발된 예측 디스플레이 기법을 적용, 효율적인 군집 로봇의 원격 제어가 가능하도록 하였다. 잘 알려진 논홀로노믹 패시브 디컴포지션 기법을 기반으로 대형 변경이 가능하도록 새로운 파라미터를 추가, 대형 변경간 발생할 수 있는 문제들에 대해 파악하고 이를 해결하기 위해 포텐셜 필드를 활용하였다. n대의 로봇으로 다양한 대형 변경이 가능토록 시뮬레이션 환경을 조성, 39대의 탱크를 이용하여여 5가지의 각기 다른 대형으로의 변환을 새로이 제시한 알고리즘을 적용하여 구현하였다. 또한 실제 로봇 3대를 활용하여 알고리즘의 효용성에 대한 실험을 필두로 좁은 길목, 개활지 등 연속적으로 변화하는 환경 속에서의 구동을 통해 최종적으로 제시한 프레임워크의 타당성에 대해 검증하였다.We propose a novel framework for formation reconguration of multiple nonholonomic wheeled mobile robots (WMRs) in the changing driving environment. We utilize an onboard system of WMRs with the capability of sensing and computing. Each WMR has the same computing power for visualizing the driving environment, handling the sensing information and calculating the control action. One of the WMRs is the leader with the FPV camera and SLAM, while others with monocular cameras with limited FoV, as the followers, keep a certain desired formation during driving in a distributed manner. We set two control objectives, one is group driving and the other is holding the shape of the formation. We have to capture the control objectives separately and simultaneously, we make the best use of nonholonomic passive decomposition to split the WMRs' kinematics into those of the formation maintaining and group driving. The repulsive potential function to prevent the collision among WMRs and attractive potential function to restrict the boundary of follower WMRs' moving space due to limited FoV range of the monocular cameras while switching their formation are also used. Simulation with 39 tanks and experiments with three WMRs are also performed to verify the proposed framework.Acknowledgements iii List of Figures vii Abbreviations ix 1 Introduction 1 2 Formation Reconguration Control Design 5 2.1 Nonholonomic Passive Decomposition . . . . . . . . . . . . . . . 5 2.2 Attractive and Repulsive Potential Function . . . . . . . . . . . . 10 2.3 Control Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.4 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3 Estimation and Predictive Display 20 3.1 Distributed Pose Estimation . . . . . . . . . . . . . . . . . . . . . 20 3.1.1 EKF Pose Estimation of Leader WMR . . . . . . . . . . . 20 3.1.2 EKF Pose Estimation of Follower WMRs . . . . . . . . . 22 3.2 Predictive Display for Distributed WMRs Teleoperation . . . . . 23 4 Experiment 27 4.1 Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.2 Demonstrate the Proposed Algorithm . . . . . . . . . . . . . . . 30 4.3 Teleoperation Experiment with the Algorithm . . . . . . . . . . . 33 5 Conclusion 40Maste

분산형 통신 및 구동부족 로봇시스템 을 위한 분할기법 기반의 반자율 원격제어 프레임워크 개발

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 기계항공공학부, 2018. 2. 이동준.The framework of stable bilateral teleoperation has been well established during decades. However, the standard bilateral teleoperation framework could be a baseline for a successful telerobotics but not sufficient for real-application because they usually concentrate on only the bilateral stability. The least considered in the previous research is how to apply a complex robot systems such as multiple mobile robots or a large degree of freedom mobile manipulators for real applications. The main challenges of teleoperation of complex robotic systems in real-world are to achieve two different control objectives (i.e., follow the human command and the coordination/ stabilization of the internal movement) of the slave robots simultaneously, while providing intuitive information about the complicated features of the system. In this thesis, we develop decomposition-based semi-autonomous teleoperation framework for robotic systems which have distributed communication and underactuation property, consisting of three steps: 1) decomposition step, where the human command is defined, and the robotic system is split into the command tracking space and its orthogonal complement (i.e., internal motion)2) control design of the slave robot, in which we design the slave controller for human command tracking and stabilization/coordination of internal motion spaceand 3) feedback interface design, through which we propose a multi-modal feedback interface (for example, visual and haptic) designed with the consideration of the task and the characteristics of the system. Among numerous types of robots, in this thesis, we focus on two types of robotic systems: 1) multiple nonholonomic wheeled mobile robots (WMRs) with distributed communication requirement and 2) manipulator-stage over vertical flexible beam which is under-actuated system. The proposed framework is applied to both case step by step and perform experiments and human subject study to verify/demonstrate the proposed framework for both cases. For distributed WMRs, we consider the scenario that a single user remotely operates a platoon of nonholonomic WMRs that distributively communicate each other in unknown environment. For this, in decomposition step, we utilize nonholonomic passive decomposition to split the platoon kinematics into that of the formation-keeping aspect and the collective tele-driving aspect. Next, in control design step, we design the controls for these two aspects individually and distribute them into each WMR while fully incorporating their nonholonomic constraint and distribution requirement. Finally, in the step of feedback interface design, we also propose a novel predictive display, which, by providing the user with the estimated current and predicted future pose informations of the platoon and future possibility of collision while fully incorporating the uncertainty inherent to the distribution, can significantly enhance the tele-driving performance and easiness of the platoon. The second part is the manipulator-stage over vertical flexible beam which is under-actuated system. Here, the human command defines the desired motion of the end-effector (or the manipulator), and the vibration of the beam should be subdued at the same time. Thus, at the first step, we utilize the passive decomposition to split the dynamics into manipulator motion space and its orthogonal complement, in which we design the control for the suppression of the vibration. For human command tracking, we design the passivity-based control, and, for the suppression of the vibration, we propose two controls: LQR-based control and nonlinear control based on Lyapunov function analysis. Finally, visuo-haptic feedback interface is preliminarily designed for successful peg-in-hole tasks.1 Introduction 1 1.1 Background and Contribution 1 1.2 Related Works 4 1.2.1 Related Works on Distributed Systems 5 1.2.2 Related Works on Manipulator-Stage System 6 1.3 Outline 6 2 Preliminary 7 2.1 Passive Decomposition 7 2.1.1 Basic Notations and Properties of Standard Passive Decomposition 7 2.1.2 Nonholonomic Passive Decomposition 9 3 Semi-Autonomous Teleoperation of Nonholonomic Wheeled Mobile Robots with Distributed Communication 11 3.1 Distributed Control Design 11 3.1.1 Nonholonomic Passive Decomposition 11 3.1.2 Control Design and Distribution 19 3.2 Distributed Pose Estimation 25 3.2.1 EKF Pose Estimation of Leader WMR 25 3.2.2 EKF Pose Estimation of Follower WMRs 28 3.3 Predictive Display for Distributed Robots Teleoperation 29 3.3.1 Estimation Propagation 31 3.3.2 Prediction Propagation 34 3.4 Experiments 38 3.4.1 Test Setup 38 3.4.2 Performance Experiment 39 3.4.3 Teleoperation Experiment with Predictive Display 40 3.4.4 Human Subject Study 44 4 Semi-Autonomous Teleoperatoin of Stage-Manipulator System on Flexible Vertical Beam 49 4.1 System Modeling 49 4.1.1 System Description 49 4.1.2 Assumed Mode Shapes 51 4.1.3 Exact Solution under Given Boundary Conditions 51 4.1.4 Euler-Lagrangian Equation 61 4.2 LQR-based Control Design 62 4.2.1 Passive Decomposition 63 4.2.2 Vibration Suppression Control Design 64 4.2.3 Joint Tracking Control Design 66 4.3 Lyapunov-based Control Design 68 4.3.1 Twice Passive Decomposition for Input Coupling 69 4.3.2 Interconnected System Description 70 4.3.3 Passivity-based Manipulator Motion Control 74 4.3.4 Dissipative Control for Vibration Suppression 74 4.4 Experiments 78 4.4.1 Test Setup 78 4.4.2 Joint Tracking and Vibration Suppression Experiment 81 4.4.3 Comparison Experiment between the LQR and the Nonlinear Control 82 5 Conclusion 83 5.1 Summary 83 5.2 Future Works 83 A Appendix 85 A.1 Internal Wrench Representation 85Docto

High latency unmanned ground vehicle teleoperation enhancement by presentation of estimated future through video transformation

Long-distance, high latency teleoperation tasks are difficult, highly stressful for teleoperators, and prone to over-corrections, which can lead to loss of control. At higher latencies, or when teleoperating at higher vehicle speed, the situation becomes progressively worse. To explore potential solutions, this research work investigates two 2D visual feedback-based assistive interfaces (sliding-only and sliding-and-zooming windows) that apply simple but effective video transformations to enhance teleoperation. A teleoperation simulator that can replicate teleoperation scenarios affected by high and adjustable latency has been developed to explore the effectiveness of the proposed assistive interfaces. Three image comparison metrics have been used to fine-tune and optimise the proposed interfaces. An operator survey was conducted to evaluate and compare performance with and without the assistance. The survey has shown that a 900ms latency increases task completion time by up to 205% for an on-road and 147 % for an off-road driving track. Further, the overcorrection-induced oscillations increase by up to 718 % with this level of latency. The survey has shown the sliding-only video transformation reduces the task completion time by up to 25.53 %, and the sliding-and-zooming transformation reduces the task completion time by up to 21.82 %. The sliding-only interface reduces the oscillation count by up to 66.28 %, and the sliding-and-zooming interface reduces it by up to 75.58 %. The qualitative feedback from the participants also shows that both types of assistive interfaces offer better visual situational awareness, comfort, and controllability, and significantly reduce the impact of latency and intermittency on the teleoperation task

타이어 모델을 사용한 자율 드리프트 주행 제어 설계 및 분석

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 기계항공공학부, 2019. 2. 이동준.본 논문에서는 Wheeled Mobile Robot(WMR)의자율드리프트 드라이빙 컨트롤러를 디자인 하고 분석하며, 이를 상용 프로그램인 CarSim을 사용한 시뮬레이션을 통하여 알고리즘을 검증 한다. 첫째로, WMR의 다이나믹스와 타이어 모델을 정의 하고, 이러한 모델로 인한 제약 사항에 대하여 논의한다. 다음으로, 사람의 관점에서 드리프트 드라이빙을 분석하고, 드리프트 드라이빙 제어기의 제어 목적을 정의한다. (차량의 방향과 요 각속도를 제어한다.) 드리프트 드라이빙 제어기는 고-레벨 제어, 목표 값을 찾기 위한 최적화 그리고 고-게인 제어로 구성된다. 다음으로, 제어하지 않는 속도에 대한 분석을 진행하였다. 마지막으로 제안한 알고리즘을 CarSim 시뮬 레이터를 이용하여 검증하였다. 정상 상태의 드리프트 시뮬레이션 결과와, 헤어핀 경로에 대한 드리프트 시뮬레이션 결과를 제시 한다.Control design and analysis of Wheeled Mobile Robot(WMR) autonomous drift-driving and the simulation experiment using the CarSim simulator are presented and the analysis of the controller proceeds. We first introduce WMR dynamics, tire model and problem formulation of the WMR. We then design drift-driving control using human strategy (control side slip angle and yaw rate). The drift-driving control consists of high-level control, optimization to find desired control input and high-gain control. We analyze the uncontrolled velocity dynamics and stability of the controller. The CarSim simulation results of drift-driving on steady-state equilibriums and the hairpin path with the desired yaw rate are provided.List of Figures - v List of Tables - vi Abbreviations - vii 1 Introduction - 1 1.1 Motivation and related works . . . . . . . . . . . . . . . . . . . . 1 1.2 Contribution of this work . . . . . . . . . . . . . . . . . . . . . . 3 2 System Modeling - 5 2.1 Model dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Tire model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Problemformulation . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 Drift-Driving Control Design - 10 3.1 High-level control . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.3 High-gain control . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4 Analysis of Control - 17 4.1 Internal dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.2 Stability analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 5 Simulation Results - 25 5.1 Simulation setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 5.2 Steady-state drift-driving . . . . . . . . . . . . . . . . . . . . . . 27 5.3 Hairpin turn drift-driving . . . . . . . . . . . . . . . . . . . . . . 33 6 Conclusion and Future Work - 40 6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 6.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Maste

Long future frame prediction using optical flow informed deep neural networks for enhancement of robotic teleoperation in high latency environments

High latency in teleoperation has a significant negative impact on operator performance. While deep learning has revolutionized many domains recently, it has not previously been applied to teleoperation enhancement. We propose a novel approach to predict video frames deep into the future using neural networks informed by synthetically generated optical flow information. This can be employed in teleoperated robotic systems that rely on video feeds for operator situational awareness. We have used the image-to-image translation technique as a basis for the prediction of future frames. The Pix2Pix conditional generative adversarial network (cGAN) has been selected as a base network. Optical flow components reflecting real-time control inputs are added to the standard RGB channels of the input image. We have experimented with three data sets of 20,000 input images each that were generated using our custom-designed teleoperation simulator with a 500-ms delay added between the input and target frames. Structural Similarity Index Measures (SSIMs) of 0.60 and Multi-SSIMs of 0.68 were achieved when training the cGAN with three-channel RGB image data. With the five-channel input data (incorporating optical flow) these values improved to 0.67 and 0.74, respectively. Applying Fleiss\u27 κ gave a score of 0.40 for three-channel RGB data, and 0.55 for five-channel optical flow-added data. We are confident the predicted synthetic frames are of sufficient quality and reliability to be presented to teleoperators as a video feed that will enhance teleoperation. To the best of our knowledge, we are the first to attempt to reduce the impacts of latency through future frame prediction using deep neural networks

Conference on Intelligent Robotics in Field, Factory, Service, and Space (CIRFFSS 1994), volume 1

The AIAA/NASA Conference on Intelligent Robotics in Field, Factory, Service, and Space (CIRFFSS '94) was originally proposed because of the strong belief that America's problems of global economic competitiveness and job creation and preservation can partly be solved by the use of intelligent robotics, which are also required for human space exploration missions. Individual sessions addressed nuclear industry, agile manufacturing, security/building monitoring, on-orbit applications, vision and sensing technologies, situated control and low-level control, robotic systems architecture, environmental restoration and waste management, robotic remanufacturing, and healthcare applications

PENGEMBANGAN MODUL MATA KULIAH ELEKTRONIKA DASAR II MATERI ROBOTIKA UNTUK MENINGKATKAN KEMANDIRIAN DAN PENGETAHUAN

This research aims to design basic electronics module II on robotics material and to know student perception. This research was research and development that uses the ADDIE model. The subject of this study was material experts, media experts, and 21 students. The instruments used were interview sheets, validation sheets, and questioner. Qualitative data analysis techniques are done descriptively and quantitative data is analyzed using descriptive statistics. The result was a printed module containing material about the basic robotics, controlling device, microcontroller, and an actuator. Modules were equipped with self-learning activities designing simple projects. Material expert validation results obtained a score of 84 with good categories and the validation result of media experts acquired score 109 with excellent categories.  The student perception result of the module obtained a score of 1280 with good categories.Penelitian ini bertujuan untuk mendesain Modul Elektronika Dasar II pada materi robotika dan mengetahui persepsi mahasiswa. Penelitian ini merupakan penelitian dan pengembangan yang menggunakan model ADDIE. Subjek penelitian ini yaitu ahli materi, ahli media serta 21 mahasiswa. Instrument yang digunakan adalah lembar wawancara, lembar validasi serta lembar angket persepsi. Teknik analisis data kualitatif dilakukan secara deskriptif dan data kuantitatif dianalisis menggunakan statistik deskriftif. Hasil penelitian ini berupa modul cetak yang berisi materi tentang dasar robotika, piranti pengendali, mikrokontroler dan aktuator. Modul dilengkapi dengan kegiatan belajar mandiri merancang projek sederhana. Hasil validasi ahli materi diperoleh skor 84 dengan kategori baik dan hasil validasi ahli media diperoleh skor 109 dengan kategori sangat baik.  Hasil persepsi mahasiswa terhadap modul diperoleh skor 1280 dengan kategori baik

Advances in Human-Robot Interaction

Rapid advances in the field of robotics have made it possible to use robots not just in industrial automation but also in entertainment, rehabilitation, and home service. Since robots will likely affect many aspects of human existence, fundamental questions of human-robot interaction must be formulated and, if at all possible, resolved. Some of these questions are addressed in this collection of papers by leading HRI researchers