Optimal Energy Management for a Hybrid Vehicle Using Neuro-Dynamic Programming to Consider Transient Engine Operation

This paper proposes a self-learning approach to develop optimal power management with multiple objectives, e.g. to minimize fuel consumption and transient engine-out NOx and particulate matter emission for a series hydraulic hybrid vehicle. Addressing multiple objectives is particularly relevant in the case of a diesel powered hydraulic hybrid since it has been shown that managing engine transients can significantly reduce real-world emissions. The problem is formulated as an infinite time horizon stochastic sequential decision making/markovian problem. The problem is computationally intractable by conventional Dynamic programming due to large number of states and complex modeling issues. Therefore, the paper proposes an online self-learning neural controller based on the fundamental principles of Neuro-Dynamic Programming (NDP) and reinforcement learning. The controller learns from its interactions with the environment and improves its performance over time. The controller tries to minimize multiple objectives and continues to evolve until a global solution is achieved. The control law is a stationary full state feedback based on 5 states and can be directly implemented. The controller performance is then evaluated in the Engine-in-the-Loop (EIL) facility.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/89876/1/draft_01.pd

Optimal Vehicle Dynamics and Powertrain Control for Connected and Automated Vehicles

The implementation of connected and automated vehicle technologies enables opportunities for a novel computational framework for real-time control actions aimed at optimizing energy consumption and associated benefits. In this paper, we present a two-level control architecture for a connected and automated plug-in hybrid electric vehicle to optimize simultaneously its speed profile and powertrain efficiency. We evaluate the proposed architecture through simulation in a network of vehicles.Comment: 6 pages, 2 figures, 1 table, conferenc

Neuro_Dynamic Programming and Reinforcement Learning for Optimal Energy Management of a Series Hydraulic Hybrid Vehicle Considering Engine Transient Emissions.

Sequential decision problems under uncertainty are encountered in various fields such as optimal control and operations research. In this dissertation, Neuro-Dynamic Programming (NDP) and Reinforcement Learning (RL) are applied to address policy optimization problems with multiple objectives and large design state space. Dynamic Programming (DP) is well suited for determining an optimal solution for constrained nonlinear model based systems. However, DP suffers from curse of dimensionality i.e. computational effort grows exponentially with state space. The new algorithms address this problem and enable practical application of DP to a much broader range of problems. The other contribution is to design fast and computationally efficient transient emission models. The power management problem for a hybrid vehicle can be formulated as an infinite time horizon stochastic sequential decision-making problem. In the past, policy optimization has been applied successfully to design optimal supervisory controller for best fuel economy. Static emissions have been considered too but engine research has shown that transient operation can have significant impact on real-world emissions. Modeling transient emissions results in addition of more states. Therefore, the problem with multiple objectives i.e. minimize fuel consumption and transient particulate and NOX emissions, becomes computationally intractable by DP. This research captures the insight with models and brings it into the supervisory controller design. A self-learning supervisory controller is designed based on the principles of NDP and RL. The controller starts “naïve” i.e. with no knowledge to control the onboard power but learns to do so in an optimal manner after interacting with the system. The controller tries to minimize multiple objectives and continues to evolve until a global solution is achieved. Virtual sensors for predicting real-time transient particulate and NOX emissions are developed using neuro-fuzzy modeling technique, which utilizes a divide-and-conquer strategy. The highly nonlinear engine operating space is partitioned into smaller subspaces and a separate local model is trained to for each subspace. Finally, the supervisory controller along with virtual emission sensors is implemented and evaluated using the Engine-In-the-Loop (EIL) setup. EIL is a unique facility to systematically evaluate control methodologies through concurrent running of real engine and a virtual hybrid powertrain.Ph.D.Mechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/89829/1/rajit_1.pd

Real-time Power Management of Hybrid Power Systems in All Electric Ship Applications.

Motivated by the need for achieving flexible shipboard arrangement and meeting future on-board power demand, the concept of all-electric ships (AES) has been pursued. The integrated power systems enable this initiative by providing a common electrical platform for the propulsion and ship-service loads and are a classic example of hybrid power systems (HPS). In order to leverage the complementary dynamic characteristics of the diverse sources, effective power management (PM) is essential to coordinate the sources and energy storage to achieve efficient power generation and fast load following. Although extensive research has been done on the PM of hybrid land vehicles for commercial applications, this problem for shipboard military applications remains largely unaddressed, leading to its exclusive focus in this dissertation. While HPS brings in many opportunities for power management, there are many associated challenges for systems used in military applications since both performance as well as survivability criteria have to be satisfied. While the on-demand goal for the power management problem makes real-time control a key requirement, leveraging the look-ahead opportunities for the shipboard missions makes it difficult to attain this goal. Furthermore, the nonlinearity and the complexity of hybrid power systems, make the optimal control of HPS challenging. In this dissertation, we address real-time power management for the AES and general hybrid power systems targeting military applications. The central theme of this work is the development of power management schemes with real-time computational efficiency by exploring HPS dynamic properties, for improved performance (namely fuel economy and fast load following) during normal mode conditions as well as increased survivability during component failure. A reduced order dynamic HPS model and a scaled test bed is developed as a numerical tool for controller design and validation. The power management (PM) schemes for both normal as well as failure mode conditions are proposed and implemented on a real-time simulator which demonstrated the real-time performance of the proposed method. While the normal mode PM leverages the complementary dynamic characteristics of the HPS for real-time look-ahead control and performance, the failure mode PM uses a reference governor approach for real-time constraint enforcement.Ph.D.Mechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/77863/1/gseenuma_1.pd

Energy Efficient Control of Hydrostatic Drive Transmissions: A Nonlinear Model-Based Approach

The high standard of living in industrial countries is based on the utilization of machines. In particular, the tasks performed with hydraulic work machines (HWMs) are essential in numerous industrial fields. Agriculture, mining, and construction are just a few examples of the lines of business that would be inconceivable today without HWMs. However, rising oil prices and competing technologies are challenging the manufacturers of these machines to improve their fuel economy.Despite the fact that energy efficiency research of hydraulic systems has been active for more than a decade, there seems to be a significant gap between industry and academia. The manufacturers of HWMs have not adopted, for example, novel system layouts, prototype components, or algorithms that require powerful control units in their products.The fuel economy of HWMs can be increased by utilizing system information in control algorithms. This cost-effective improvement enables operation in challenging regions and closer to the operating boundaries of the system. Consequently, the information about the system has to be accurate. For example, reducing the rotational speed of the engine has proven effective in improving the energy efficiency, but it increases the risk of even stalling the engine, for instance in situations where the power generation cannot meet the high transient demand. If this is considered in the controller with low uncertainty, fuel economy can be improved without decreasing the functionality of the machine.This thesis studies the advantages of model-based control in the improvement of the fuel economy of HWMs. The focus is on hydrostatic drive transmissions, which is the main consumer of energy in certain applications, such as wheel loaders.We started by developing an instantaneous optimization algorithm based on a quasi-static system model. The control commands of this fuel optimal controller (FOC) were determined based on cost function, which includes terms for fuel economy, steady-state velocity error, and changes in the control commands.Although the use of quasi-static models is adequate for steady-state situations, the velocity tracking during transients and under load changes has proven to be inadequate. To address this issue, a high-performance velocity-tracking controller was devised. Full state feedback was assumed, and we resorted to a so-called D-implementation, which eliminates, for example, the need for the equilibrium values of pressure signals. The nonlinearities of the system were considered with the state-dependent parameters of the linear model.In the next step, a nonlinear model predictive controller combined fuel economy control and velocity tracking. To the best of the author’s knowledge, this is the first time that the model predictive control scheme has been utilized with such a detailed system model that also considers the hydraulic efficiencies and torque generation of the engine. This enables utilizing the controller as a benchmark of control algorithms for non-hybrid hydrostatic drive transmissions that do not require information about the future.The initial tests of all the controllers were conducted with a validated simulation model of a research platform machine, a five-ton municipal tractor. In addition, the FOC and velocity-tracking controller were implemented into the control system of the machine. The practical worth of the FOC was proven with a relatively unique field experiment set-up that included, for example, an online measurement system of fuel consumption and autonomous path following. The fuel economy improved up to 16.6% when compared with an industrial baseline controller. The devised velocity-tracking concept was also proven as a significant reduction of error was observed in comparison with classic literature solutions, namely state feedback and proportional-integral-derivative controllers

Development of a Simulation based Powertrain Design Framework for Evaluation of Transient Soot Emissions from Diesel Engine Vehicles.

This dissertation presents the development of a modeling and simulation framework for diesel engine vehicles to enable soot emissions as a constraint in powertrain design and control. To this end, numerically efficient models for predicting temporallyresolved transient soot emissions are identified in the form of a third-order dual-input single-output (DISO) Volterra series from transient soot data recorded by integrating real-time (RT) vehicle level models in Engine-in-the-loop (EIL) experiments. It is shown that the prediction accuracy of transient soot significantly improves over the steady-state maps, while the model remains computationally efficient for systemslevel work. The evaluation of powertrain design also requires a systematic procedure for dealing with the issue that drivers potentially adapt their driving styles to a given design. In order to evaluate the implications of different powertrain design changes on transient soot production it is essential to compare these design changes on a consistent basis. This problem is explored in the context of longitudinal motion of a vehicle following a standard drive-cycle repeatedly. This dissertation develops a proportional-derivative (PD) type iterative learning based algorithm to synthesize driver actuator inputs that seek to minimize soot emissions using the Volterra series based transient soot models. The solution is compared to the one obtained using linear programming. Results show that about 19% reduction in total soot can be achieved for the powertrain design considered in about 40 iterations. The two contributions of this dissertation: development of computationally efficient system level transient soot models and the synthesis of driver inputs via iterative learning for reducing soot, both contribute to improving the art of modeling and simulation for diesel powertrain design and control.Ph.D.Mechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/86386/1/ahlawatr_1.pd