Stochastic model predictive control for energy management of power-split plug-in hybrid electric vehicles based on reinforcement learning

In this paper, a stochastic model predictive control (MPC) method based on reinforcement learning is proposed for energy management of plug-in hybrid electric vehicles (PHEVs). Firstly, the power transfer of each component in a power-split PHEV is described in detail. Then an effective and convergent reinforcement learning controller is trained by the Q-learning algorithm according to the driving power distribution under multiple driving cycles. By constructing a multi-step Markov velocity prediction model, the reinforcement learning controller is embedded into the stochastic MPC controller to determine the optimal battery power in predicted time domain. Numerical simulation results verify that the proposed method achieves superior fuel economy that is close to that by stochastic dynamic programming method. In addition, the effective state of charge tracking in terms of different reference trajectories highlight that the proposed method is effective for online application requiring a fast calculation speed

Advanced Control and Estimation Concepts, and New Hardware Topologies for Future Mobility

According to the National Research Council, the use of embedded systems throughout society could well overtake previous milestones in the information revolution. Mechatronics is the synergistic combination of electronic, mechanical engineering, controls, software and systems engineering in the design of processes and products. Mechatronic systems put “intelligence” into physical systems. Embedded sensors/actuators/processors are integral parts of mechatronic systems. The implementation of mechatronic systems is consistently on the rise. However, manufacturers are working hard to reduce the implementation cost of these systems while trying avoid compromising product quality. One way of addressing these conflicting objectives is through new automatic control methods, virtual sensing/estimation, and new innovative hardware topologies

Transportation Mission-Based Optimization of Heavy Combination Road Vehicles and Distributed Propulsion, Including Predictive Energy and Motion Control

This thesis proposes methodologies to improve heavy vehicle design by reducing the total cost of ownership and by increasing energy efficiency and safety.Environmental issues, consumers expectations and the growing demand for freight transport have created a competitive environment in providing better transportation solutions. In this thesis, it is proposed that freight vehicles can be designed in a more cost- and energy-efficient manner if they are customized for narrow ranges of operational domains and transportation use-cases. For this purpose, optimization-based methods were applied to minimize the total cost of ownership and to deliver customized vehicles with tailored propulsion components that best fit the given transportation missions and operational environment. Optimization-based design of the vehicle components was found to be effective due to the simultaneous consideration of the optimization of the transportation mission infrastructure, including charging stations, loading-unloading, routing and fleet composition and size, especially in case of electrified propulsion. Implementing integrated vehicle hardware-transportation optimization could reduce the total cost of ownership by up to 35% in the case of battery electric heavy vehicles. Furthermore, in this thesis, the impacts of two future technological advancements, i.e., heavy vehicle electrification and automation, on road freight transport were discussed. It was shown that automation helps the adoption of battery electric heavy vehicles in freight transport. Moreover, the optimizations and simulations produced a large quantity of data that can help users to select the best vehicle in terms of the size, propulsion system, and driving system for a given transportation assignment. The results of the optimizations revealed that battery electric and hybrid heavy combination vehicles exhibit the lowest total cost of ownership in certain transportation scenarios. In these vehicles, propulsion can be distributed over different axles of different units, thus the front units may be pushed by the rear units. Therefore, online optimal energy management strategies were proposed in this thesis to optimally control the vehicle motion and propulsion in terms of the minimum energy usage and lateral stability. These involved detailed multitrailer vehicle modeling and the design and solution of nonlinear optimal control problems

Predictive Energy Management in Connected Vehicles: Utilizing Route Information Preview for Energy Saving

This dissertation formulates algorithms that use preview information of road terrain and traffic flow for reducing energy use and emissions of modern vehicles with conventional or hybrid powertrains. Energy crisis, long term energy deficit, and more restrictive environmental protection policies require developing more efficient and cleaner vehicle powertrain systems. An alternative to making advanced technology engines or electrifying the vehicle powertrain is utilizing ambient terrain and traffic information in the energy management of vehicles, a topic which has not been emphasized in the past. Today\u27s advances in vehicular telematics and advances in GIS (Geographic Information System), GPS (Global Positioning Systems), ITS (Intelligent Transportation Systems), V2V (Vehicle to Vehicle) communication, and VII (Vehicle Infrastructure Integration ) create more opportunities for predicting a vehicle\u27s trip information with details such as the future road grade, the distance to the destination, speed constraints imposed by the traffic flow, which all can be utilized for better vehicle energy management. Optimal or near optimal decision-making based on this available information requires optimal control methods, whose fundamental theories were well studied in the past but are not directly applicable due to the complexity of real problems and uncertainty in the available preview information. This dissertation proposes the use of optimal control theories and tools including Pontryagin minimum principle, Dynamic Programming (DP) which is a numerical realization of Bellman\u27s principle of optimality, and Model Predictive Control (MPC) in the optimization-based control of hybrid electric vehicles (HEVs), plug-in hybrid electric vehicles (PHEVs), and conventional vehicles based on preview of future route information. The dissertation includes three parts introduced as follows: First, the energy saving benefit in HEV energy management by previewing future terrain information and applying optimal control methods is explored. The potential gain in fuel economy is evaluated, if road grade information is integrated in energy management of hybrid vehicles. Real-world road geometry information is taken into account in power management decisions by using both Dynamic Programming (DP) and a standard Equivalent Consumption Minimization Strategy (ECMS), derived using Pontryagin minimum principle. Secondly, the contribution of different levels of preview to energy management of plug-in hybrid vehicles (PHEVs) is studied. The gains to fuel economy of plug-in hybrid vehicles with availability of velocity and terrain preview and knowledge of distance to the next charging station are investigated. Access to future driving information is classified into full, partial, or no future information and energy management strategies for real-time implementation with partial future preview are proposed. ECMS as well as Dynamic Programming (DP) is systematically utilized to handle the resulting optimal control problems with different levels of preview. We also study the benefit of future traffic flow information preview in improving the fuel economy of conventional vehicles by predictive control methods. According to the time-scale of the preview information and its importance to the driver, the energy optimization problem is decomposed into different levels. In the microscopic level, a model predictive controller as well as a car following model is employed for predictive adaptive cruise control by stochastically forecasting the driving behavior of the lead car. In the macroscopic level, we propose to incorporate the estimated macroscopic future traffic flow information and optimize the cost-to-go by utilizing a two-dimension Dynamic Programming (2D-DP). The algorithm yields the optimal trip velocity as the reference velocity for the driver or a low level controller to follow. Through the study, we show that energy use and emissions can be reduced considerably by using preview route information. The methodologies discussed in this dissertation provide an alternative mean for the automotive industry to develop more efficient and environmentally friendly vehicles by relying mostly on software and information and with minimal hardware investments

A novel strategy for power sources management in connected plug-in hybrid electric vehicles based on mobile edge computation framework

This paper proposes a novel control framework and the corresponding strategy for power sources management in connected plug-in hybrid electric vehicles (cPHEVs). A mobile edge computation (MEC) based control framework is developed first, evolving the conventional on-board vehicle control unit (VCU) into the hierarchically asynchronous controller that is partly located in cloud. Elaborately contrastive analysis on the performance of processing capacity, communication frequency and communication delay manifests dramatic potential of the proposed framework in sustaining development of the cooperative control strategy for cPHEVs. On the basis of MEC based control framework, a specific cooperative strategy is constructed. The novel strategy accomplishes energy flow management between different power sources with incorporation of the active energy consumption plan and adaptive energy consumption management. The method to generate the reference battery state-of-charge (SOC) trajectories in energy consumption plan stage is emphatically investigated, fast outputting reference trajectories that are tightly close to results by global optimization methods. The estimation of distribution algorithm (EDA) is employed to output reference control policies under the specific terminal conditions assigned via the machine learning based method. Finally, simulation results highlight that the novel strategy attains superior performance in real-time application that is close to the offline global optimization solutions

REAL-TIME PREDICTIVE CONTROL OF CONNECTED VEHICLE POWERTRAINS FOR IMPROVED ENERGY EFFICIENCY

The continued push for the reduction of energy consumption across the automotive vehicle fleet has led to widespread adoption of hybrid and plug-in hybrid electric vehicles (PHEV) by auto manufacturers. In addition, connected and automated vehicle (CAV) technologies have seen rapid development in recent years and bring with them the potential to significantly impact vehicle energy consumption. This dissertation studies predictive control methods for PHEV powertrains that are enabled by CAV technologies with the goal of reducing vehicle energy consumption. First, a real-time predictive powertrain controller for PHEV energy management is developed. This controller utilizes predictions of future vehicle velocity and power demand in order to optimize powersplit decisions of the vehicle. This predictive powertrain controller utilizes nonlinear model predictive control (NMPC) to perform this optimization while being cognizant of future vehicle behavior. Second, the developed NMPC powertrain controller is thoroughly evaluated both in simulation and real-time testing. The controller is assessed over a large number of standardized and real-world drive cycles in simulation in order to properly quantify the energy savings benefits of the controller. In addition, the NMPC powertrain controller is deployed onto a real-time rapid prototyping embedded controller installed in a test vehicle. Using this real-time testing setup, the developed NMPC powertrain controller is evaluated using on-road testing for both energy savings performance and real-time performance. Third, a real-time integrated predictive powertrain controller (IPPC) for a multi-mode PHEV is presented. Utilizing predictions of future vehicle behavior, an optimal mode path plan is computed in order to determine a mode command best suited to the future conditions. In addition, this optimal mode path planning controller is integrated with the NMPC powertrain controller to create a real-time integrated predictive powertrain controller that is capable of full supervisory control for a multi-mode PHEV. Fourth, the IPPC is evaluated in simulation testing across a range of standard and real-world drive cycles in order to quantify the energy savings of the controller. This analysis is comprised of the combined benefit of the NMPC powertrain controller and the optimal mode path planning controller. The IPPC is deployed onto a rapid prototyping embedded controller for real-time evaluation. Using the real-time implementation of the IPPC, on-road testing was performed to assess both energy benefits and real-time performance of the IPPC. Finally, as the controllers developed in this research were evaluated for a single vehicle platform, the applicability of these controllers to other platforms is discussed. Multiple cases are discussed on how both the NMPC powertrain controller and the optimal mode path planning controller can be applied to other vehicle platforms in order to broaden the scope of this research

A Novel Learning Based Model Predictive Control Strategy for Plug-in Hybrid Electric Vehicle

The multi-source electromechanical coupling renders energy management of plug-in hybrid electric vehicles (PHEVs) highly nonlinear and complex. Furthermore, the complicated nonlinear management process highly depends on knowledge of driving conditions, and hinders the control strategies efficiently applied instantaneously, leading to massive challenges in energy saving improvement of PHEVs. To address these issues, a novel learning based model predictive control (LMPC) strategy is developed for a serial-parallel PHEV with the reinforced optimal control effect in real time application. Rather than employing the velocity-prediction based MPC methods favored in the literature, an original reference-tracking based MPC solution is proposed with strong instant application capacity. To guarantee the optimal control effect, an online learning process is implemented in MPC via the Gaussian process (GP) model to address the uncertainties during state estimation. The tracking reference in LMPC based control problem in PHEV is achieved by a microscopic traffic flow analysis (MTFA) method. The simulation results validate that the proposed method can optimally manage energy flow within vehicle power sources in real time, highlighting its anticipated preferable performance