Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

A Unifying review of linear gaussian models

Factor analysis, principal component analysis, mixtures of gaussian clusters, vector quantization, Kalman filter models, and hidden Markov models can all be unified as variations of unsupervised learning under a single basic generative model. This is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity. Through the use of other nonlinearities, we show how independent component analysis is also a variation of the same basic generative model.We show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term. We introduce a new model for static data, known as sensible principal component analysis, as well as a novel concept of spatially adaptive observation noise. We also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models

Quality-Aware Broadcasting Strategies for Position Estimation in VANETs

The dissemination of vehicle position data all over the network is a fundamental task in Vehicular Ad Hoc Network (VANET) operations, as applications often need to know the position of other vehicles over a large area. In such cases, inter-vehicular communications should be exploited to satisfy application requirements, although congestion control mechanisms are required to minimize the packet collision probability. In this work, we face the issue of achieving accurate vehicle position estimation and prediction in a VANET scenario. State of the art solutions to the problem try to broadcast the positioning information periodically, so that vehicles can ensure that the information their neighbors have about them is never older than the inter-transmission period. However, the rate of decay of the information is not deterministic in complex urban scenarios: the movements and maneuvers of vehicles can often be erratic and unpredictable, making old positioning information inaccurate or downright misleading. To address this problem, we propose to use the Quality of Information (QoI) as the decision factor for broadcasting. We implement a threshold-based strategy to distribute position information whenever the positioning error passes a reference value, thereby shifting the objective of the network to limiting the actual positioning error and guaranteeing quality across the VANET. The threshold-based strategy can reduce the network load by avoiding the transmission of redundant messages, as well as improving the overall positioning accuracy by more than 20% in realistic urban scenarios.Comment: 8 pages, 7 figures, 2 tables, accepted for presentation at European Wireless 201

Machine Learning for Fluid Mechanics

The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from field measurements, experiments and large-scale simulations at multiple spatiotemporal scales. Machine learning offers a wealth of techniques to extract information from data that could be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. It outlines fundamental machine learning methodologies and discusses their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that considers data as an inherent part of modeling, experimentation, and simulation. Machine learning provides a powerful information processing framework that can enrich, and possibly even transform, current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202