Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Sampling-based Algorithms for Optimal Motion Planning

During the last decade, sampling-based path planning algorithms, such as Probabilistic RoadMaps (PRM) and Rapidly-exploring Random Trees (RRT), have been shown to work well in practice and possess theoretical guarantees such as probabilistic completeness. However, little effort has been devoted to the formal analysis of the quality of the solution returned by such algorithms, e.g., as a function of the number of samples. The purpose of this paper is to fill this gap, by rigorously analyzing the asymptotic behavior of the cost of the solution returned by stochastic sampling-based algorithms as the number of samples increases. A number of negative results are provided, characterizing existing algorithms, e.g., showing that, under mild technical conditions, the cost of the solution returned by broadly used sampling-based algorithms converges almost surely to a non-optimal value. The main contribution of the paper is the introduction of new algorithms, namely, PRM* and RRT*, which are provably asymptotically optimal, i.e., such that the cost of the returned solution converges almost surely to the optimum. Moreover, it is shown that the computational complexity of the new algorithms is within a constant factor of that of their probabilistically complete (but not asymptotically optimal) counterparts. The analysis in this paper hinges on novel connections between stochastic sampling-based path planning algorithms and the theory of random geometric graphs.Comment: 76 pages, 26 figures, to appear in International Journal of Robotics Researc

Researching the social impact of the arts : literature, fiction and the novel

This paper offers a contribution to current debates in the field of cultural policy about the social impact of the arts. It explores the conceptual difficulties that arise in the notion of ‘the arts’ and the implications of these difficulties for attempts to generalise about their value, function and impact. It considers both ‘essentialist’ and ‘institutional’ perspectives, first on ‘the arts’ in toto and then on literature, fiction and the novel with the view of making an innovative intellectual connection between aesthetic theories and contemporary cultural policy discourse. The paper shows how literature sits uneasily in the main systems of classifying the arts and how the novel and fiction itself are seen as problematic categories. The position of the novel in the literary canon is also discussed, with particular reference to the shifting instability of the canon. The paper suggests that the dilemmas thrown up in trying to define or classify the novel are likely to be encountered in attempting to define other art forms. The implications of these findings for the interpretation and conduct of traditional ‘impact studies’ are explored

Receding Horizon Control of UAVs using Gradual Dense-Sparse Discretizations

In this paper we propose a way of increasing the efficiency of some direct Receding Horizon Control (RHC) schemes. The basic idea is to adapt the allocation of computational resources to how the iterative plans are used. By using Gradual Dense-Sparse discretizations (GDS), we make sure that the plans are detailed where they need to be, i.e., in the very near future, and less detailed further ahead. The gradual transition in discretization density reflects increased uncertainty and reduced need for detail near the end of the planning horizon. The proposed extension is natural, since the standard RHC approach already contains a computational asymmetry in terms of the coarse cost-to-go computations and the more detailed short horizon plans. Using GDS discretizations, we bring this asymmetry one step further, and let the short horizon plans themselves be detailed in the near term and more coarse in the long term. The rationale for different levels of detail is as follows. 1) Near future plans need to be implemented soon, while far future plans can be refined or revised later. 2) More accurate sensor information is available about the system and its surroundings in the near future, and detailed planning is only rational in low uncertainty situations. 3) It has been shown that reducing the node density in the later parts of fixed horizon optimal control problems gives a very small reduction in the solution quality of the first part of the trajectory. The reduced level of detail in the later parts of a plan can increase the efficiency of the RHC in two ways. If the discretization is made sparse by removing nodes, fewer computations are necessary, and if the discretization is made sparse by spreading the last nodes over a longer time-horizon, the performance will be improved. I