Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

A path planning and path-following control framework for a general 2-trailer with a car-like tractor

Maneuvering a general 2-trailer with a car-like tractor in backward motion is a task that requires significant skill to master and is unarguably one of the most complicated tasks a truck driver has to perform. This paper presents a path planning and path-following control solution that can be used to automatically plan and execute difficult parking and obstacle avoidance maneuvers by combining backward and forward motion. A lattice-based path planning framework is developed in order to generate kinematically feasible and collision-free paths and a path-following controller is designed to stabilize the lateral and angular path-following error states during path execution. To estimate the vehicle state needed for control, a nonlinear observer is developed which only utilizes information from sensors that are mounted on the car-like tractor, making the system independent of additional trailer sensors. The proposed path planning and path-following control framework is implemented on a full-scale test vehicle and results from simulations and real-world experiments are presented.Comment: Preprin

Cluster space collision avoidance for mobile two-robot systems,”

Abstract-The cluster space state representation for multirobot systems provides a simple means of specifying and monitoring the geometry and motion characteristics of a cluster of mobile robots. In previous work, this approach has been experimentally verified and validated for controlling the motion of mobile multi-robot systems ranging from land rovers to autonomous boats. In this paper we introduce a compact collision avoidance algorithm that operates at the level of the cluster, leading to coordinated translational and rotational motions that allow obstacles to be avoided while maintaining the relative geometry of the cluster. This paper formulates the potential-field based obstacle avoidance algorithm, describes its integration within the cluster space control architecture, and presents successful experimental results of its application to two simple, diverse multi-robot testbeds

Planning under uncertainty for dynamic collision avoidance

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2011.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 157-169).We approach dynamic collision avoidance problem from the perspective of designing collision avoidance systems for unmanned aerial vehicles. Before unmanned aircraft can fly safely in civil airspace, robust airborne collision avoidance systems must be developed. Instead of hand-crafting a collision avoidance algorithm for every combination of sensor and aircraft configurations, we investigate automatic generation of collision avoidance algorithms given models of aircraft dynamics, sensor performance, and intruder behavior. We first formulate the problem within the Partially Observable Markov Decision Process (POMDP) framework, and use generic MDP/POMDP solvers offline to compute vertical-only avoidance strategies that optimize a cost function to balance flight-plan deviation with risk of collision. We then describe a second framework that performs online planning and allows for 3-D escape maneuvers by starting with possibly dangerous initial flight plans and improving them iteratively. Experimental results with four different sensor modalities and a parametric aircraft performance model demonstrate the suitability of both approaches.by Selim Temizer.Ph.D

Hardware, Software, and Low-Level Control Scheme Development for a Real-Time Autonomous Rover

The objective of this research is to develop a low-cost autonomous rover platform for experiments in autonomous navigation. This thesis describes the design, development, and testing of an autonomous rover platform, based on the commercial, off-the-shelf Tamiya TXT-1 radio controlled vehicle. This vehicle is outfitted with an onboard computer based on the Mini-ITX architecture and an array of sensors for localization and obstacle avoidance, and programmed with Matlab/SimulinkRTM Real-Time Workshop (RTW) utilizing the Linux Real-Time Application Interface (RTAI) operating system.;First, a kinematic model is developed and verified for the rover. Then a proportional-integral-derivative (PID) feedback controller is developed for translational and rotational velocity regulation. Finally, a hybrid navigation controller is developed combining a potential field controller and an obstacle avoidance controller for waypoint tracking.;Experiments are performed to verify the functionality of the kinematic model and the PID velocity controller, and to demonstrate the capabilities of the hybrid navigation controller. These experiments prove that the rover is capable of successfully navigating in an unknown indoor environment. Suggestions for future research include the integration of additional sensors for localization and creation of multiple platforms for autonomous coordination experiments

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties