Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning

The majority of current studies on autonomous vehicle control via deep reinforcement learning (DRL) utilize point-mass kinematic models, neglecting vehicle dynamics which includes acceleration delay and acceleration command dynamics. The acceleration delay, which results from sensing and actuation delays, results in delayed execution of the control inputs. The acceleration command dynamics dictates that the actual vehicle acceleration does not rise up to the desired command acceleration instantaneously due to dynamics. In this work, we investigate the feasibility of applying DRL controllers trained using vehicle kinematic models to more realistic driving control with vehicle dynamics. We consider a particular longitudinal car-following control, i.e., Adaptive Cruise Control (ACC), problem solved via DRL using a point-mass kinematic model. When such a controller is applied to car following with vehicle dynamics, we observe significantly degraded car-following performance. Therefore, we redesign the DRL framework to accommodate the acceleration delay and acceleration command dynamics by adding the delayed control inputs and the actual vehicle acceleration to the reinforcement learning environment state, respectively. The training results show that the redesigned DRL controller results in near-optimal control performance of car following with vehicle dynamics considered when compared with dynamic programming solutions.Comment: Accepted to 2019 IEEE Intelligent Transportation Systems Conferenc

A review of microgrid development in the United States – A decade of progress on policies, demonstrations, controls, and software tools

Microgrids have become increasingly popular in the United States. Supported by favorable federal and local policies, microgrid projects can provide greater energy stability and resilience within a project site or community. This paper reviews major federal, state, and utility-level policies driving microgrid development in the United States. Representative U.S. demonstration projects are selected and their technical characteristics and non-technical features are introduced. The paper discusses trends in the technology development of microgrid systems as well as microgrid control methods and interactions within the electricity market. Software tools for microgrid design, planning, and performance analysis are illustrated with each tool's core capability. Finally, the paper summarizes the successes and lessons learned during the recent expansion of the U.S. microgrid industry that may serve as a reference for other countries developing their own microgrid industries

Synergistic Interactions of Dynamic Ridesharing and Battery Electric Vehicles Land Use, Transit, and Auto Pricing Policies

It is widely recognized that new vehicle and fuel technology is necessary, but not sufficient, to meet deep greenhouse gas (GHG) reductions goals for both the U.S. and the state of California. Demand management strategies (such as land use, transit, and auto pricing) are also needed to reduce passenger vehicle miles traveled (VMT) and related GHG emissions. In this study, the authors explore how demand management strategies may be combined with new vehicle technology (battery electric vehicles or BEVs) and services (dynamic ridesharing) to enhance VMT and GHG reductions. Owning a BEV or using a dynamic ridesharing service may be more feasible when distances to destinations are made shorter and alternative modes of travel are provided by demand management strategies. To examine potential markets, we use the San Francisco Bay Area activity based travel demand model to simulate business-as-usual, transit oriented development, and auto pricing policies with and without high, medium, and low dynamic ridesharing participation rates and BEV daily driving distance ranges. The results of this study suggest that dynamic ridesharing has the potential to significantly reduce VMT and related GHG emissions, which may be greater than land use and transit policies typically included in Sustainable Community Strategies (under California Senate Bill 375), if travelers are willing pay with both time and money to use the dynamic ridesharing system. However, in general, large synergistic effects between ridesharing and transit oriented development or auto pricing policies were not found in this study. The results of the BEV simulations suggest that TODs may increase the market for BEVs by less than 1% in the Bay Area and that auto pricing policies may increase the market by as much as 7%. However, it is possible that larger changes are possible over time in faster growing regions where development is currently at low density levels (for example, the Central Valley in California). The VMT Fee scenarios show larger increases in the potential market for BEV (as much as 7%). Future research should explore the factors associated with higher dynamic ridesharing and BEV use including individual attributes, characteristics of tours and trips, and time and cost benefits. In addition, the travel effects of dynamic ridesharing systems should be simulated explicitly, including auto ownership, mode choice, destination, and extra VMT to pick up a passenger

Collaborative signal and information processing for target detection with heterogeneous sensor networks

In this paper, an approach for target detection and acquisition with heterogeneous sensor networks through strategic resource allocation and coordination is presented. Based on sensor management and collaborative signal and information processing, low-capacity low-cost sensors are strategically deployed to guide and cue scarce high performance sensors in the network to improve the data quality, with which the mission is eventually completed more efficiently with lower cost. We focus on the problem of designing such a network system in which issues of resource selection and allocation, system behaviour and capacity, target behaviour and patterns, the environment, and multiple constraints such as the cost must be addressed simultaneously. Simulation results offer significant insight into sensor selection and network operation, and demonstrate the great benefits introduced by guided search in an application of hunting down and capturing hostile vehicles on the battlefield

Agents Cut Emissions On how a Multi-Agent System Contributes to a more Sustainable Energy Consumption

AbstractThe Vehicle-to-Grid technology allows electric vehicles to not only procure electric energy, but also to feed energy back into the grid network. However, by using Vehicle-to-Grid, energy literally degenerates into an article of merchandise and becomes of interest to several stakeholders. In this paper, we describe a multi-agent system, which embraces this exact view and maximises the interest of several stakeholders in using Vehicle-to-Grid capable electric vehicles. In order to emphasise the applicability of our approach we performed a field test with real electric vehicles and charging stations. We describe both, implementational details as well as the results of our field test in this paper

A short curriculum of the robotics and technology of computer lab

Our research Lab is directed by Prof. Anton Civit. It is an interdisciplinary group of 23 researchers that carry out their teaching and researching labor at the Escuela Politécnica Superior (Higher Polytechnic School) and the Escuela de Ingeniería Informática (Computer Engineering School). The main research fields are: a) Industrial and mobile Robotics, b) Neuro-inspired processing using electronic spikes, c) Embedded and real-time systems, d) Parallel and massive processing computer architecture, d) Information Technologies for rehabilitation, handicapped and elder people, e) Web accessibility and usability In this paper, the Lab history is presented and its main publications and research projects over the last few years are summarized.Nuestro grupo de investigación está liderado por el profesor Civit. Somos un grupo multidisciplinar de 23 investigadores que realizan su labor docente e investigadora en la Escuela Politécnica Superior y en Escuela de Ingeniería Informática. Las principales líneas de investigaciones son: a) Robótica industrial y móvil. b) Procesamiento neuro-inspirado basado en pulsos electrónicos. c) Sistemas empotrados y de tiempo real. d) Arquitecturas paralelas y de procesamiento masivo. e) Tecnología de la información aplicada a la discapacidad, rehabilitación y a las personas mayores. f) Usabilidad y accesibilidad Web. En este artículo se reseña la historia del grupo y se resumen las principales publicaciones y proyectos que ha conseguido en los últimos años