A Personalized Human Drivers\u27 Risk Sensitive Characteristics Depicting Stochastic Optimal Control Algorithm for Adaptive Cruise Control

This paper presents a personalized stochastic optimal adaptive cruise control (ACC) algorithm for automated vehicles (AVs) incorporating human drivers\u27 risk-sensitivity under system and measurement uncertainties. The proposed controller is designed as a linear exponential-of-quadratic Gaussian (LEQG) problem, which utilizes the stochastic optimal control mechanism to feedback the deviation from the design car-following target. With the risk-sensitive parameter embedded in LEQG, the proposed method has the capability to characterize risk preference heterogeneity of each AV against uncertainties according to each human drivers\u27 preference. Further, the established control theory can achieve both expensive control mode and non-expensive control mode via changing the weighting matrix of the cost function in LEQG to reveal different treatments on input. Simulation tests validate the proposed approach can characterize different driving behaviors and its effectiveness in terms of reducing the deviation from equilibrium state. The ability to produce different trajectories and generate smooth control of the proposed algorithm is also verified

An Extended Mean Field Game for Storage in Smart Grids

We consider a stylized model for a power network with distributed local power generation and storage. This system is modeled as network connection a large number of nodes, where each node is characterized by a local electricity consumption, has a local electricity production (e.g. photovoltaic panels), and manages a local storage device. Depending on its instantaneous consumption and production rates as well as its storage management decision, each node may either buy or sell electricity, impacting the electricity spot price. The objective at each node is to minimize energy and storage costs by optimally controlling the storage device. In a non-cooperative game setting, we are led to the analysis of a non-zero sum stochastic game with $N$ players where the interaction takes place through the spot price mechanism. For an infinite number of agents, our model corresponds to an Extended Mean-Field Game (EMFG). In a linear quadratic setting, we obtain and explicit solution to the EMFG, we show that it provides an approximate Nash-equilibrium for $N$-player game, and we compare this solution to the optimal strategy of a central planner.Comment: 27 pages, 5 figures. arXiv admin note: text overlap with arXiv:1607.02130 by other author

Stochastic and Optimal Distributed Control for Energy Optimization and Spatially Invariant Systems

Improving energy efficiency and grid responsiveness of buildings requires sensing, computing and communication to enable stochastic decision-making and distributed operations. Optimal control synthesis plays a significant role in dealing with the complexity and uncertainty associated with the energy systems. The dissertation studies general area of complex networked systems that consist of interconnected components and usually operate in uncertain environments. Specifically, the contents of this dissertation include tools using stochastic and optimal distributed control to overcome these challenges and improve the sustainability of electric energy systems. The first tool is developed as a unifying stochastic control approach for improving energy efficiency while meeting probabilistic constraints. This algorithm is applied to demonstrate energy efficiency improvement in buildings and improving operational efficiency of virtualized web servers, respectively. Although all the optimization in this technique is in the form of convex optimization, it heavily relies on semidefinite programming (SP). A generic SP solver can handle only up to hundreds of variables. This being said, for a large scale system, the existing off-the-shelf algorithms may not be an appropriate tool for optimal control. Therefore, in the sequel I will exploit optimization in a distributed way. The second tool is itself a concrete study which is optimal distributed control for spatially invariant systems. Spatially invariance means the dynamics of the system do not vary as we translate along some spatial axis. The optimal H2 [H-2] decentralized control problem is solved by computing an orthogonal projection on a class of Youla parameters with a decentralized structure. Optimal H∞ [H-infinity] performance is posed as a distance minimization in a general L∞ [L-infinity] space from a vector function to a subspace with a mixed L∞ and H∞ space structure. In this framework, the dual and pre-dual formulations lead to finite dimensional convex optimizations which approximate the optimal solution within desired accuracy. Furthermore, a mixed L2 [L-2] /H∞ synthesis problem for spatially invariant systems as trade-offs between transient performance and robustness. Finally, we pursue to deal with a more general networked system, i.e. the Non-Markovian decentralized stochastic control problem, using stochastic maximum principle via Malliavin Calculus

A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers

Dynamical selection of Nash equilibria using Experience Weighted Attraction Learning: emergence of heterogeneous mixed equilibria

We study the distribution of strategies in a large game that models how agents choose among different double auction markets. We classify the possible mean field Nash equilibria, which include potentially segregated states where an agent population can split into subpopulations adopting different strategies. As the game is aggregative, the actual equilibrium strategy distributions remain undetermined, however. We therefore compare with the results of Experience-Weighted Attraction (EWA) learning, which at long times leads to Nash equilibria in the appropriate limits of large intensity of choice, low noise (long agent memory) and perfect imputation of missing scores (fictitious play). The learning dynamics breaks the indeterminacy of the Nash equilibria. Non-trivially, depending on how the relevant limits are taken, more than one type of equilibrium can be selected. These include the standard homogeneous mixed and heterogeneous pure states, but also \emph{heterogeneous mixed} states where different agents play different strategies that are not all pure. The analysis of the EWA learning involves Fokker-Planck modeling combined with large deviation methods. The theoretical results are confirmed by multi-agent simulations.Comment: 35 pages, 16 figure