Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Motion primitives and 3D path planning for fast flight through a forest

This paper presents two families of motion primitives for enabling fast, agile flight through a dense obstacle field. The first family of primitives consists of a time-delay dependent 3D circular path between two points in space and the control inputs required to fly the path. In particular, the control inputs are calculated using algebraic equations which depend on the flight parameters and the location of the waypoint. Moreover, the transition between successive maneuver states, where each state is defined by a unique combination of constant control inputs, is modeled rigorously as an instantaneous switch between the two maneuver states following a time delay which is directly related to the agility of the robotic aircraft. The second family consists of aggressive turn-around (ATA) maneuvers which the robot uses to retreat from impenetrable pockets of obstacles. The ATA maneuver consists of an orchestrated sequence of three sets of constant control inputs. The duration of the first segment is used to optimize the ATA for the spatial constraints imposed by the turning volume. The motion primitives are validated experimentally and implemented in a simulated receding horizon control (RHC)-based motion planner. The paper concludes with inverse-design pointers derived from the primitives

Trajectory Planning for Autonomous High-Speed Overtaking in Structured Environments using Robust MPC

Automated vehicles are increasingly getting mainstreamed and this has pushed development of systems for autonomous manoeuvring (e.g., lane-change, merge, overtake, etc.) to the forefront. A novel framework for situational awareness and trajectory planning to perform autonomous overtaking in high-speed structured environments (e.g., highway, motorway) is presented in this paper. A combination of a potential field like function and reachability sets of a vehicle are used to identify safe zones on a road that the vehicle can navigate towards. These safe zones are provided to a tube-based robust model predictive controller as reference to generate feasible trajectories for combined lateral and longitudinal motion of a vehicle. The strengths of the proposed framework are: (i) it is free from nonconvex collision avoidance constraints, (ii) it ensures feasibility of trajectory even if decelerating or accelerating while performing lateral motion, and (iii) it is real-time implementable. The ability of the proposed framework to plan feasible trajectories for highspeed overtaking is validated in a high-fidelity IPG CarMaker and Simulink co-simulation environment