Data-Driven Transferred Energy Management Strategy for Hybrid Electric Vehicles via Deep Reinforcement Learning

Real-time applications of energy management strategies (EMSs) in hybrid electric vehicles (HEVs) are the harshest requirements for researchers and engineers. Inspired by the excellent problem-solving capabilities of deep reinforcement learning (DRL), this paper proposes a real-time EMS via incorporating the DRL method and transfer learning (TL). The related EMSs are derived from and evaluated on the real-world collected driving cycle dataset from Transportation Secure Data Center (TSDC). The concrete DRL algorithm is proximal policy optimization (PPO) belonging to the policy gradient (PG) techniques. For specification, many source driving cycles are utilized for training the parameters of deep network based on PPO. The learned parameters are transformed into the target driving cycles under the TL framework. The EMSs related to the target driving cycles are estimated and compared in different training conditions. Simulation results indicate that the presented transfer DRL-based EMS could effectively reduce time consumption and guarantee control performance.Comment: 25 pages, 12 figure

Optimization-Driven Powertrain-Oriented Adaptive Cruise Control to Improve Energy Saving and Passenger Comfort

Assessing the potential of advanced driver assistance systems requires developing dedicated control algorithms for controlling the longitudinal speed of automated vehicles over time. In this paper, a multiobjective off-line optimal control approach for planning the speed of the following vehicle in adaptive cruise control (ACC) driving is proposed. The implemented method relies on the principle of global optimality fostered by dynamic programming (DP) and aims to minimize propelling energy consumption and enhance passenger comfort. The powertrain model and onboard control system are integrated within the proposed car-following optimization framework. The retained ACC approach ensures that the distance between the following vehicle and the preceding vehicle is always maintained within allowed limits. The flexibility of the proposed method is demonstrated here through ease of implementation on a wide range of powertrain categories, including a conventional vehicle propelled by an internal combustion engine solely, a pure electric vehicle, a parallel P2 hybrid electric vehicle (HEV) and a power-split HEV. Moreover, different driving conditions are considered to prove the effectiveness of the proposed optimization-driven ACC approach. Obtained simulation results suggest that up to 22% energy-saving and 48% passenger comfort improvement might be achieved for the ACC-enabled vehicle compared with the preceding vehicle by implementing the proposed optimization-driven ACC approach. Engineers may adopt the proposed workflow to evaluate corresponding real-time ACC approaches and assess optimal powertrain design solutions for ACC driving

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Development of hybrid electric vehicles depends on an advanced and efficient energy management strategy (EMS). With online and real-time requirements in mind, this article presents a human-like energy management framework for hybrid electric vehicles according to deep reinforcement learning methods and collected historical driving data. The hybrid powertrain studied has a series-parallel topology, and its control-oriented modeling is founded first. Then, the distinctive deep reinforcement learning (DRL) algorithm, named deep deterministic policy gradient (DDPG), is introduced. To enhance the derived power split controls in the DRL framework, the global optimal control trajectories obtained from dynamic programming (DP) are regarded as expert knowledge to train the DDPG model. This operation guarantees the optimality of the proposed control architecture. Moreover, the collected historical driving data based on experienced drivers are employed to replace the DP-based controls, and thus construct the human-like EMSs. Finally, different categories of experiments are executed to estimate the optimality and adaptability of the proposed human-like EMS. Improvements in fuel economy and convergence rate indicate the effectiveness of the constructed control structure.Comment: 8 pages, 10 figure

Toward Holistic Energy Management Strategies for Fuel Cell Hybrid Electric Vehicles in Heavy-Duty Applications

The increasing need to slow down climate change for environmental protection demands further advancements toward regenerative energy and sustainable mobility. While individual mobility applications are assumed to be satisfied with improving battery electric vehicles (BEVs), the growing sector of freight transport and heavy-duty applications requires alternative solutions to meet the requirements of long ranges and high payloads. Fuel cell hybrid electric vehicles (FCHEVs) emerge as a capable technology for high-energy applications. This technology comprises a fuel cell system (FCS) for energy supply combined with buffering energy storages, such as batteries or ultracapacitors. In this article, recent successful developments regarding FCHEVs in various heavy-duty applications are presented. Subsequently, an overview of the FCHEV drivetrain, its main components, and different topologies with an emphasis on heavy-duty trucks is given. In order to enable system layout optimization and energy management strategy (EMS) design, functionality and modeling approaches for the FCS, battery, ultracapacitor, and further relevant subsystems are briefly described. Afterward, common methodologies for EMS are structured, presenting a new taxonomy for dynamic optimization-based EMS from a control engineering perspective. Finally, the findings lead to a guideline toward holistic EMS, encouraging the co-optimization of system design, and EMS development for FCHEVs. For the EMS, we propose a layered model predictive control (MPC) approach, which takes velocity planning, the mitigation of degradation effects, and the auxiliaries into account simultaneously

Self-Learning Neural controller for Hybrid Power Management using Neuro-Dynamic Programming

A supervisory controller strategy for a hybrid vehicle coordinates the operation of the two power sources onboard of a vehicle to maximize objectives like fuel economy. In the past, various control strategies have been developed using heuristics as well as optimal control theory. The Stochastic Dynamic Programming (SDP) has been previously applied to determine implementable optimal control policies for discrete time dynamic systems whose states evolve according to given transition probabilities. However, the approach is constrained by the curse of dimensionality, i.e. an exponential increase in computational effort with increase in system state space, faced by dynamic programming based algorithms. This paper proposes a novel approach capable of overcoming the curse of dimensionality and solving policy optimization for a system with very large design state space. We propose developing a supervisory controller for hybrid vehicles based on the principles of reinforcement learning and neuro-dynamic programming, whereby the cost-to-go function is approximated using a neural network. The controller learns and improves its performance over time. The simulation results obtained for a series hydraulic hybrid vehicle over a driving schedule demonstrate the effectiveness of the proposed technique.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/89874/1/draft_01.pd

Energy Efficiency and Emission Testing for Connected and Automated Vehicles Using Real-World Driving Data

By using the onboard sensing and external connectivity technology, connected and automated vehicles (CAV) could lead to improved energy efficiency, better routing, and lower traffic congestion. With the rapid development of the technology and adaptation of CAV, it is more critical to develop the universal evaluation method and the testing standard which could evaluate the impacts on energy consumption and environmental pollution of CAV fairly, especially under the various traffic conditions. In this paper, we proposed a new method and framework to evaluate the energy efficiency and emission of the vehicle based on the unsupervised learning methods. Both the real-world driving data of the evaluated vehicle and the large naturalistic driving dataset are used to perform the driving primitive analysis and coupling. Then the linear weighted estimation method could be used to calculate the testing result of the evaluated vehicle. The results show that this method can successfully identify the typical driving primitives. The couples of the driving primitives from the evaluated vehicle and the typical driving primitives from the large real-world driving dataset coincide with each other very well. This new method could enhance the standard development of the energy efficiency and emission testing of CAV and other off-cycle credits