Evaluation of automated decisionmaking methodologies and development of an integrated robotic system simulation

A generic computer simulation for manipulator systems (ROBSIM) was implemented and the specific technologies necessary to increase the role of automation in various missions were developed. The specific items developed are: (1) capability for definition of a manipulator system consisting of multiple arms, load objects, and an environment; (2) capability for kinematic analysis, requirements analysis, and response simulation of manipulator motion; (3) postprocessing options such as graphic replay of simulated motion and manipulator parameter plotting; (4) investigation and simulation of various control methods including manual force/torque and active compliances control; (5) evaluation and implementation of three obstacle avoidance methods; (6) video simulation and edge detection; and (7) software simulation validation

NASA Center for Intelligent Robotic Systems for Space Exploration

NASA's program for the civilian exploration of space is a challenge to scientists and engineers to help maintain and further develop the United States' position of leadership in a focused sphere of space activity. Such an ambitious plan requires the contribution and further development of many scientific and technological fields. One research area essential for the success of these space exploration programs is Intelligent Robotic Systems. These systems represent a class of autonomous and semi-autonomous machines that can perform human-like functions with or without human interaction. They are fundamental for activities too hazardous for humans or too distant or complex for remote telemanipulation. To meet this challenge, Rensselaer Polytechnic Institute (RPI) has established an Engineering Research Center for Intelligent Robotic Systems for Space Exploration (CIRSSE). The Center was created with a five year $5.5 million grant from NASA submitted by a team of the Robotics and Automation Laboratories. The Robotics and Automation Laboratories of RPI are the result of the merger of the Robotics and Automation Laboratory of the Department of Electrical, Computer, and Systems Engineering (ECSE) and the Research Laboratory for Kinematics and Robotic Mechanisms of the Department of Mechanical Engineering, Aeronautical Engineering, and Mechanics (ME,AE,&M), in 1987. This report is an examination of the activities that are centered at CIRSSE

Nonlinear Model Predictive Control for Motion Generation of Humanoids

Das Ziel dieser Arbeit ist die Untersuchung und Entwicklung numerischer Methoden zur Bewegungserzeugung von humanoiden Robotern basierend auf nichtlinearer modell-prädiktiver Regelung. Ausgehend von der Modellierung der Humanoiden als komplexe Mehrkörpermodelle, die sowohl durch unilaterale Kontaktbedingungen beschränkt als auch durch die Formulierung unteraktuiert sind, wird die Bewegungserzeugung als Optimalsteuerungsproblem formuliert. In dieser Arbeit werden numerische Erweiterungen basierend auf den Prinzipien der Automatischen Differentiation für rekursive Algorithmen, die eine effiziente Auswertung der dynamischen Größen der oben genannten Mehrkörperformulierung erlauben, hergeleitet, sodass sowohl die nominellen Größen als auch deren ersten Ableitungen effizient ausgewertet werden können. Basierend auf diesen Ideen werden Erweiterungen für die Auswertung der Kontaktdynamik und der Berechnung des Kontaktimpulses vorgeschlagen. Die Echtzeitfähigkeit der Berechnung von Regelantworten hängt stark von der Komplexität der für die Bewegungerzeugung gewählten Mehrkörperformulierung und der zur Verfügung stehenden Rechenleistung ab. Um einen optimalen Trade-Off zu ermöglichen, untersucht diese Arbeit einerseits die mögliche Reduktion der Mehrkörperdynamik und andererseits werden maßgeschneiderte numerische Methoden entwickelt, um die Echtzeitfähigkeit der Regelung zu realisieren. Im Rahmen dieser Arbeit werden hierfür zwei reduzierte Modelle hergeleitet: eine nichtlineare Erweiterung des linearen inversen Pendelmodells sowie eine reduzierte Modellvariante basierend auf der centroidalen Mehrkörperdynamik. Ferner wird ein Regelaufbau zur GanzkörperBewegungserzeugung vorgestellt, deren Hauptbestandteil jeweils aus einem speziell diskretisierten Problem der nichtlinearen modell-prädiktiven Regelung sowie einer maßgeschneiderter Optimierungsmethode besteht. Die Echtzeitfähigkeit des Ansatzes wird durch Experimente mit den Robotern HRP-2 und HeiCub verifiziert. Diese Arbeit schlägt eine Methode der nichtlinear modell-prädiktiven Regelung vor, die trotz der Komplexität der vollen Mehrkörperformulierung eine Berechnung der Regelungsantwort in Echtzeit ermöglicht. Dies wird durch die geschickte Kombination von linearer und nichtlinearer modell-prädiktiver Regelung auf der aktuellen beziehungsweise der letzten Linearisierung des Problems in einer parallelen Regelstrategie realisiert. Experimente mit dem humanoiden Roboter Leo zeigen, dass, im Vergleich zur nominellen Strategie, erst durch den Einsatz dieser Methode eine Bewegungserzeugung auf dem Roboter möglich ist. Neben Methoden der modell-basierten Optimalsteuerung werden auch modell-freie Methoden des verstärkenden Lernens (Reinforcement Learning) für die Bewegungserzeugung untersucht, mit dem Fokus auf den schwierig zu modellierenden Modellunsicherheiten der Roboter. Im Rahmen dieser Arbeit werden eine allgemeine vergleichende Studie sowie Leistungskennzahlen entwickelt, die es erlauben, modell-basierte und -freie Methoden quantitativ bezüglich ihres Lösungsverhaltens zu vergleichen. Die Anwendung der Studie auf ein akademisches Beispiel zeigt Unterschiede und Kompromisse sowie Break-Even-Punkte zwischen den Problemformulierungen. Diese Arbeit schlägt basierend auf dieser Grundlage zwei mögliche Kombinationen vor, deren Eigenschaften bewiesen und in Simulation untersucht werden. Außerdem wird die besser abschneidende Variante auf dem humanoiden Roboter Leo implementiert und mit einem nominellen modell-basierten Regler verglichen

Nonlinear Model Predictive Control-based Collision Avoidance for Mobile Robot

This work proposes an efficient and safe single-layer Nonlinear Model Predictive Control (NMPC) system based on LiDAR to solve the problem of autonomous navigation in cluttered environments with previously unidentified static and dynamic obstacles of any shape. Initially, LiDAR sensor data is collected. Then, the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm, is used to cluster the (Lidar) points that belong to each obstacle together. Moreover, a Minimum Euclidean Distance (MED) between the robot and each obstacle with the aid of a safety margin is utilized to implement safety-critical obstacle avoidance rather than existing methods in the literature that depend on enclosing the obstacles with a circle or minimum bounding ellipse. After that, to impose avoidance constraints with feasibility guarantees and without compromising stability, an NMPC for set-point stabilization is taken into consideration with a design strategy based on terminal inequality and equality constraints. Consequently, numerous obstacles can be avoided at the same time efficiently and rapidly through unstructured environments with narrow corridors.  Finally, a case study with an omnidirectional wheeled mobile robot (OWMR) is presented to assess the proposed NMPC formulation for set-point stabilization. Furthermore, the efficacy of the proposed system is tested by experiments in simulated scenarios using a robot simulator named CoppeliaSim in combination with MATLAB which utilizes the CasADi Toolbox, and Statistics and Machine Learning Toolbox. Two simulation scenarios are considered to show the performance of the proposed framework. The first scenario considers only static obstacles while the second scenario is more challenging and contains static and dynamic obstacles. In both scenarios, the OWMR successfully reached the target pose (1.5m, 1.5m, 0°) with a small deviation. Four performance indices are utilized to evaluate the set-point stabilization performance of the proposed control framework including the steady-state error in the posture vector which is less than 0.02 meters for position and 0.012 for orientation, and the integral of norm squared actual control inputs which is 19.96 and 21.74 for the first and second scenarios respectively. The proposed control framework shows a positive performance in a narrow-cluttered environment with unknown obstacles

Terrain Classification from Body-mounted Cameras during Human Locomotion

Abstract—This paper presents a novel algorithm for terrain type classification based on monocular video captured from the viewpoint of human locomotion. A texture-based algorithm is developed to classify the path ahead into multiple groups that can be used to support terrain classification. Gait is taken into account in two ways. Firstly, for key frame selection, when regions with homogeneous texture characteristics are updated, the fre-quency variations of the textured surface are analysed and used to adaptively define filter coefficients. Secondly, it is incorporated in the parameter estimation process where probabilities of path consistency are employed to improve terrain-type estimation. When tested with multiple classes that directly affect mobility a hard surface, a soft surface and an unwalkable area- our proposed method outperforms existing methods by up to 16%, and also provides improved robustness. Index Terms—texture, classification, recursive filter, terrain classification I

자율 주행 차량의 심층강화학습 기반 긴급 차선 변경 경로 최적화

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2021.8. 최예림.The emergency lane change is a risk itself because it is made instantaneously in emergency such as a sudden stop of the vehicle in front in the driving lane. Therefore, the optimization of the lane change trajectory is an essential research area of autonomous vehicle. This research proposes a path optimization for emergency lane change of autonomous vehicles based on deep reinforcement learning. This algorithm is developed with a focus on fast and safe avoidance behavior and lane change in an emergency. As the first step of algorithm development, a simulation environment was established. IPG CARMAKER was selected for reliable vehicle dynamics simulation and construction of driving scenarios for reinforcement learning. This program is a highly reliable and can analyze the behavior of a vehicle similar to that of a real vehicle. In this research, a simulation was performed using the Hyundai I30-PDe full car model. And as a simulator for DRL and vehicle control, Matlab Simulink which can encompass all of control, measurement, and artificial intelligence was selected. By connecting two simulators, the emergency lane change trajectory is optimized based on DRL. The vehicle lane change trajectory is modeled as a 3rd order polynomial. The start and end point of the lane change is set and analyzed as a function of the lane change distance for the coefficient of the polynomial. In order to optimize the coefficients. A DRL architecture is constructed. 12 types of driving environment data are used for the observation space. And lane change distance which is a variable of polynomial is selected as the output of action space. Reward space is designed to maximize the learning ability. Dynamic & static reward and penalty are given at each time step of simulation, so that optimization proceeds in a direction in which the accumulated rewards could be maximized. Deep Deterministic Policy Gradient agent is used as an algorithm for optimization. An algorithm is developed for driving a vehicle in a dynamic simulation program. First, an algorithm is developed that can determine when, at what velocity, and in which direction to change the lane of a vehicle in an emergency situation. By estimating the maximum tire-road friction coefficient in real-time, the minimum distance for the driving vehicle to stop is calculated to determine the risk of longitudinal collision with the vehicle in front. Also, using Gipps’ safety distance formula, an algorithm is developed that detects the possibility of a collision with a vehicle coming from the lane to be changed, and determines whether to overtake the vehicle to pass forward or to go backward after as being overtaken. Based on this, the decision-making algorithm for the final lane change is developed by determine the collision risk and safety of the left and right lanes. With the developed algorithm that outputs the emergency lane change trajectory through the configured reinforcement learning structure and the general driving trajectory such as the lane keeping algorithm and the adaptive cruise control algorithm according to the situation, an integrated algorithm that drives the ego vehicle through the adaptive model predictive controller is developed. As the last step of the research, DRL was performed to optimize the developed emergency lane change path optimization algorithm. 60,000 trial-and-error learning is performed to develop the algorithm for each driving situation, and performance is evaluated through test driving.긴급 차선 변경은 주행 차선에서 선행차량 급정거와 같은 응급상황 발생시에 순간적으로 이루어지는 것이므로 그 자체에 위험성을 안고 있다. 지나치게 느리게 조향을 하는 경우, 주행 차량은 앞에 있는 장애물과의 충돌을 피할 수 없다. 이와 반대로 지나치게 빠르게 조향을 하는 경우, 차량과 지면 사이의 작용력은 타이어 마찰 한계를 넘게 된다. 이는 차량의 조종 안정성을 떨어트려 스핀이나 전복 등 다른 양상의 사고를 야기한다. 따라서 차선 변경 경로의 최적화는 자율 주행 차량의 응급 상황 대처에 필수적인 요소이다. 본 논문에서는 심층강화학습을 기반으로 자율 주행 차량의 긴급 차선 변경 경로를 최적화한다. 이 알고리즘은 선행차량의 급정거나 장애물 출현과 같은 응급상황 발생 시, 빠르고 안전한 회피 거동 및 차선 변경에 초점을 맞추어 개발되었다. 알고리즘 개발의 첫 번째 단계로서 시뮬레이션 환경이 구축되었다. 신뢰성 있는 차량 동역학 시뮬레이션과 강화학습을 위한 주행 시나리오 구축을 위하여 IPG CARMAKER가 선정되었다. 이 프로그램은 실제 산업 현장에서 사용되는 높은 신뢰성을 가진 프로그램으로 실제 차량과 유사한 차량의 거동을 분석할 수 있다. 본 연구에서는 현대자동차의 I30-PDe 모델을 사용하여 시뮬레이션을 수행하였다. 또한 강화학습과 차량제어를 위한 프로그램으로 제어, 계측, 인공지능을 모두 아우를 수 있는 Matlab Simulink를 선정하였다. 본 연구에서는 IPG CARMAKER와 Matlab Simulink를 연동하여 심층 강화 학습을 바탕으로 긴급 차선 변경 궤적을 최적화하였다. 차량의 차선 변경 궤적은 3차 다항식의 형상으로 모델링 되었다. 차선 변경 시작 지점과 종료 지점을 설정하여 다항식의 계수를 차선 변경 거리에 대한 함수로 해석하였다. 심층 강화 학습을 기반으로 계수들을 최적화하기 위하여, 강화 학습 아키텍처를 구성하였다. 관측 공간은 12가지의 주행 환경 데이터를 이용하였고, 강화 학습의 출력으로는 3차 함수의 변수인 차선 변경 거리를 선정하였다. 그리고 강화 학습의 학습 능력을 극대화할 수 있는 보상 공간을 설계하였다. 동적 보상, 정적 보상, 동적 벌칙, 정적 벌칙을 시뮬레이션의 매 단계마다 부여함으로써 보상 총 합이 최대화될 수 있는 방향으로 학습이 진행되었다. 최적화를 위한 알고리즘으로는 Deep Deterministic Policy Gradient agent가 사용되었다. 강화학습 아키텍처와 함께 동역학 시뮬레이션 프로그램에서의 차량 구동을 위한 알고리즘을 개발하였다. 먼저 응급상황시에 차량의 차선을 언제, 어떤 속도로, 어떤 방향으로 변경할 지 결정하는 의사결정 알고리즘을 개발하였다. 타이어와 도로 사이의 최대 마찰계수를 실시간으로 추정하여 주행 차량이 정지하기 위한 최소 거리를 산출함으로써 선행 차량과의 충돌 위험을 판단하였다. 또한 Gipps의 안전거리 공식을 사용하여 변경하고자 하는 차선에서 오는 차량과의 충돌 가능성을 감지하여 그 차량을 추월해서 앞으로 지나갈지, 추월을 당해서 뒤로 갈 것인지를 결정하는 알고리즘을 개발하였다. 이를 바탕으로 좌측 차선과 우측 차선의 충돌 위험성 및 안정성을 판단하여 최종적인 차선 변경을 위한 의사결정 알고리즘을 개발하였다. 구성된 강화 학습 구조를 통한 긴급 차선 변경 궤적과 차선 유지 장치, 적응형 순항 제어와 같은 일반 주행시의 궤적을 상황에 맞추어 출력하는 알고리즘을 개발하고 적응형 모델 예측 제어기를 통해 주행 차량을 구동하는 통합 알고리즘을 개발하였다. 본 연구의 마지막 단계로서, 개발된 긴급 차선 변경 경로 생성 알고리즘의 최적화를 위하여 심층 강화 학습이 수행되었다. 총 60,000회의 시행 착오 방식의 학습을 통해 각 주행 상황 별 최적의 차선 변경 제어 알고리즘을 개발하였고, 각 주행상황 별 최적의 차선 변경 궤적을 제시하였다.Chapter 1. Introduction 1 1.1. Research Background 1 1.2. Previous Research 5 1.3. Research Objective 9 1.4. Dissertation Overview 13 Chapter 2. Simulation Environment 19 2.1. Simulator 19 2.2. Scenario 26 Chapter 3. Methodology 28 3.1. Reinforcement learning 28 3.2. Deep reinforcement learning 30 3.3. Neural network 33 Chapter 4. DRL-enhanced Lane Change 36 4.1. Necessity of Evasive Steering Trajectory Optimization 36 4.2. Trajectory Planning 39 4.3. DRL Structure 42 4.3.1. Observation 43 4.3.2. Action 47 4.3.3. Reward 49 4.3.4. Neural Network Architecture 58 4.3.5. Deep Deterministic Policy Gradient (DDPG) Agent 60 Chapter 5. Autonomous Driving Algorithm Integration 64 5.1. Lane Change Decision Making 65 5.1.1. Longitudinal Collision Detection 66 5.1.2. Lateral Collision Detection 71 5.1.3. Lane Change Direction Decision 74 5.2. Path Planning 75 5.3. Vehicle Controller 76 5.4. Algorithm Integration 77 Chapter 6. Training & Results 79 Chapter 7. Conclusion 91 References 97 국문초록 104박

The design and safety analysis of an accessible low-cost differential drive robot under LIDAR-Based obstacle avoidance

A low-cost mobile robot, OpenDani, is re-designed and built to serve as a research platform capable of autonomous tasks. A self-contained, open-source design is supported by the single-board computer mounted atop the robot. Low-cost sensors are integrated into the robot communication network to achieve online obstacle avoidance (OA). The mobile system is compared to its previous, non-autonomous design to highlight an elevated level of autonomy. An experimental analysis is performed to define a relationship between components of the OpenDani design and the safe operation of the vehicle under an obstacle avoidance task.Mechanical Engineerin

Online optimisation-based backstepping control design with application to quadrotor

In backstepping implementation, the derivatives of virtual control signals are required at each step. This study provides a novel way to solve this problem by combining online optimisation with backstepping design in an outer and inner loop manner. The properties of differential flatness and the B-spline polynomial function are exploited to transform the optimal control problem into a computationally efficient form. The optimisation process generates not only the optimised states but also their finite order derivatives which can be used to analytically calculate the derivatives of virtual control signal required in backstepping design. In addition, the online optimisation repeatedly performed in a receding horizon fashion can also realise local motion planning for obstacle avoidance. The stability of the receding horizon control scheme is analysed via Lyapunov method which is guaranteed by adding a parametrised terminal condition in the online optimisation. Numerical simulations and flight experiments of a quadrotor unmanned air vehicle are given to demonstrate the effectiveness of the proposed composite control method