Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Searching for improvement

Engineering design can be thought of as a search for the best solutions to engineering problems. To perform an effective search, one must distinguish between competing designs and establish a measure of design quality, or fitness. To compare different designs, their features must be adequately described in a well-defined framework, which can mean separating the creative and analytical parts of the design process. By this we mean that a distinction is drawn between coming up with novel design concepts, or architectures, and the process of detailing or refining existing design architecture. In the case of a given design architecture, one can consider the set of all possible designs that could be created by varying its features. If it were possible to measure the fitness of all designs in this set, then one could identify a fitness landscape and search for the best possible solution for this design architecture. In this Chapter, the significance of the interactions between design features in defining the metaphorical fitness landscape is described. This highlights that the efficiency of a search algorithm is inextricably linked to the problem structure (and hence the landscape). Two approaches, namely, Genetic Algorithms (GA) and Robust Engineering Design (RED) are considered in some detail with reference to a case study on improving the design of cardiovascular stents

Harnessing a multi-dimensional fibre laser using genetic wavefront shaping

The multi-dimensional laser is a fascinating platform not only for the discovery and understanding of new higher-dimensional coherent lightwaves but also for the frontier study of the complex three-dimensional (3D) nonlinear dynamics and solitary waves widely involved in physics, chemistry, biology and materials science. Systemically controlling coherent lightwave oscillation in multi-dimensional lasers, however, is challenging and has largely been unexplored; yet, it is crucial for both designing 3D coherent light fields and unveiling any underlying nonlinear complexities. Here, for the first time, we genetically harness a multi-dimensional fibre laser using intracavity wavefront shaping technology such that versatile lasing characteristics can be manipulated. We demonstrate that the output power, mode profile, optical spectrum and mode-locking operation can be genetically optimized by appropriately designing the objective function of the genetic algorithm. It is anticipated that this genetic and systematic intracavity control technology for multi-dimensional lasers will be an important step for obtaining high-performance 3D lasing and presents many possibilities for exploring multi-dimensional nonlinear dynamics and solitary waves that may enable new applications

Harnessing a multi-dimensional fibre laser using genetic wavefront shaping

The multi-dimensional laser is a fascinating platform not only for the discovery and understanding of new higher-dimensional coherent lightwaves but also for the frontier study of the complex three-dimensional (3D) nonlinear dynamics and solitary waves widely involved in physics, chemistry, biology and materials science. Systemically controlling coherent lightwave oscillation in multi-dimensional lasers, however, is challenging and has largely been unexplored; yet, it is crucial for both designing 3D coherent light fields and unveiling any underlying nonlinear complexities. Here, for the first time, we genetically harness a multi-dimensional fibre laser using intracavity wavefront shaping technology such that versatile lasing characteristics can be manipulated. We demonstrate that the output power, mode profile, optical spectrum and mode-locking operation can be genetically optimized by appropriately designing the objective function of the genetic algorithm. It is anticipated that this genetic and systematic intracavity control technology for multi-dimensional lasers will be an important step for obtaining high-performance 3D lasing and presents many possibilities for exploring multi-dimensional nonlinear dynamics and solitary waves that may enable new applications