Hierarchical Optimization Approach for Cooperative Vehicle Networks

This research presents a control algorithm for the cooperative control of unmanned mobile robots. The algorithm relies on continuously solving an open loop mixed integer linear programming optimization problem. Since the model can become quite complex when the number of robots and the complexity of the environment increase, computational problems can arise. To overcome these problems an approach involving a hierarchical decentralized formulation of the optimization problem is proposed.School of Electrical & Computer Engineerin

On-board Trajectory Computation for Mars Atmospheric Entry Based on Parametric Sensitivity Analysis of Optimal Control Problems

This thesis develops a precision guidance algorithm for the entry of a small capsule into the atmosphere of Mars. The entry problem is treated as nonlinear optimal control problem and the thesis focuses on developing a suboptimal feedback law. Therefore parametric sensitivity analysis of optimal control problems is combined with dynamic programming. This approach enables a real-time capable, locally suboptimal feedback scheme. The optimal control problem is initially considered in open loop fashion. To synthesize the feedback law, the optimal control problem is embedded into a family of neighboring problems, which are described by a parameter vector. The optimal solution for a nominal set of parameters is determined using direct optimization methods. In addition the directional derivatives (sensitivities) of the optimal solution with respect to the parameters are computed. Knowledge of the nominal solution and the sensitivities allows, under certain conditions, to apply Taylor series expansion to approximate the optimal solution for disturbed parameters almost instantly. Additional correction steps can be applied to improve the optimality of the solution and to eliminate errors in the constraints. To transfer this strategy to the closed loop system, the computation of the sensitivities is performed with respect to different initial conditions. Determining the perturbation direction and interpolating between sensitivities of neighboring initial conditions allows the approximation of the extremal field in a neighborhood of the nominal trajectory. This constitutes a locally suboptimal feedback law. The proposed strategy is applied to the atmospheric entry problem. The developed algorithm is part of the main control loop, i.e. optimal controls and trajectories are computed at a fixed rate, taking into account the current state and parameters. This approach is combined with a trajectory tracking controller based on the aerodynamic drag. The performance and the strengthsa and weaknesses of this two degree of freedom guidance system are analyzed using Monte Carlo simulation. Finally the real-time capability of the proposed algorithm is demonstrated in a flight representative processor-in-the-loop environment

Reinforcement learning for enhancing the stability and management of power systems with new resources

Modern power systems face numerous challenges due to uncertainties arising from factors such as renewable energy source intermittency, stochastic load demand, and evolving grid dynamics. These uncertainties can lead to imbalances in power supply and demand, resulting in frequency and voltage deviations and, in extreme cases, blackouts. To address these challenges, advanced control and optimization techniques, particularly reinforcement learning (RL), have gained significant interest in ensuring efficient and reliable power system operations. RL offers a promising approach for decision-making under uncertainty, enabling agents to learn optimal policies without explicit uncertainty modeling. This thesis explores the application of RL to two classes of operational problems within power systems. The first class focuses on power system resource management, including optimal battery control (OBC) and electric vehicle charging station (EVCS) operation. Challenges arise when formulating these problems as Markov Decision Process (MDP) to adopt RL. For example, incorporating cycle-based degradation costs into the MDP for OBC is not straightforward due to its dependence on past state of charge (SoC) trajectories. Similarly, the state and action spaces in EVCS problem scale with the number of EVs, leading to high-dimensional MDP formulations. This thesis proposes RL-based solutions for these resource management problems, while addressing the challenges by incorporating precise battery degradation model and efficient aggregation schemes to MDP. The second class of problems deals with wide-area dynamics control for power system stability enhancement. Here, it is crucial for RL approaches to account for risk measures in offline-trained RL policies, considering uncertainties and perturbations in practice. The thesis focuses on load frequency control (LFC), which is vulnerable to variability due to high load perturbations, especially in small-scale systems like networked microgrids. Additionally, wide-area damping control (WADC) relies on communication networks, and communication delays can negatively impact its performance, given its fast time-scale. Moreover, the increasing integration of grid-forming inverters (GFMs) poses challenges in accurately modeling the overall system dynamics, which results in high variability in the system. To address these uncertainties and perturbations, this thesis integrates a mean-variance risk constraint into classic linear quadratic regulator (LQR) problems with linearized dynamics, limiting deviations of state costs from their expected values and reducing system variability in worst-case scenarios. In addition, structured feedback controllers need to be considered to match specific information-exchange graphs, which complicates the geometry of feasible region. To design risk-aware controllers for constrained LQR problems, a stochastic gradient-descent with max-oracle (SGDmax) algorithm is developed. This algorithm ensures convergence to a stationary point with a high probability, making it computationally efficient as it solves the inner loop problem of a dual problem easily and utilizes zero-order policy gradients (ZOPG) to estimate unbiased gradients, eliminating the need to compute first-order values. The policy gradient nature of SGDmax also allows the incorporation of structure by considering only non-zero entries in the ZOPG. In summary, this thesis presents RL applications for effectively managing emerging energy resources and enhancing the stability of interconnected power systems. The analytical and numerical results offer efficient and reliable solutions to address uncertainty, supporting the transition towards a sustainable and resilient electricity infrastructure.Electrical and Computer Engineerin

Moving horizon optimization methods, applications and tools for learning and controlling dynamical systems

Mathematical models based on dynamical systems are crucial for understanding complex phenomena across a wide range of scientific and engineering disciplines. Optimizing these models can significantly improve the performance (e.g., in the sense of socioeconomic, environmental, and safety concerns) of various processes and systems that support our modern society, such as e.g. supply chain networks and chemical manufacturing processes. However, controlling these systems in the presence of uncertainty and for high-dimensional models is challenging. Developing robust and efficient optimization models and solution algorithms for this purpose is therefore crucial. Similarly, optimization techniques can be used to infer the governing equations for such dynamical systems from available measurement data. Learning such models is important not only for performing the aforementioned control tasks, but also for advancing our understanding of the physical laws that govern the phenomena we have so long observed but cannot quantitatively explain. Motivated by the above, this dissertation contributes novel moving horizon optimization methods, applications and tools for learning and controlling a variety of dynamical systems. The first part of this dissertation introduces the background and theory of moving horizon estimation and control methods. As a motivating example, I present a novel application of these existing methods to the optimal data-driven management of the COVID-19 pandemic in the US. The proposed approach identifies optimal social distancing and testing policies that minimize socioeconomic impact, while keeping the the number of infected individuals under a specified threshold. Subsequently, I focus on dynamical system models corresponding networks of integrators for optimal supply chain management under uncertainty. The first methodological contribution corresponds to a tube-based robust economic model predictive control framework for sparse storage systems, which I shown to have improved feasibility for supply chain management under demand disturbances. The proposed approach significantly improves computational performance relative to the available methods. Subsequently, I develop an extensive and systematic case study evaluating the performance of deterministic (feedback-based, closed-loop, or online) moving horizon optimization in comparison to stochastic and robust methods for supply chain management under increasing levels of uncertainty, forecasting errors, and recourse availability. Having demonstrated the overall robust and computationally efficient performance of deterministic moving horizon optimization techniques, the second part of the dissertation is focused on a class of multi-scale dynamical systems corresponding to supply chains of highly perishable inventory. This type of supply chains require integration of the inventory management problem with quality control by manipulating environmental conditions (e.g., temperature) during shipment and storage, which directly impact the product deterioration rate. To this end, I introduce a novel modeling approach for incorporating complex, multivariate physico-chemical product quality dynamics within the supply chain inventory balances, and provide a computationally efficient reformulation thereof. Based on this modeling approach and the results introduced in Part I of the dissertation, I develop a stabilizing closed-loop optimal supply chain production and distribution planning framework to handle uncertainties, such as random customer demand and/or random product quality spoilage. I then propose a scalable solution heuristic approach to cope with larger supply chain networks, and I present several case studies to demonstrate robustness to demand uncertainty. Lastly, I develop a simultaneous state estimation and closed-loop control approach to account for the fact that product quality may not be completely measurable in practical settings. In the third and final part of the dissertation, the focus shifts from controlling dynamical systems to learning their governing equations from data via moving horizon optimization. Here, I develop methods based on dynamic nonlinear optimization which, compared to existing efforts, demonstrate greater flexibility for handling highly nonlinear systems, for incorporating prior domain knowledge, and coping with high amounts of measurement noise in the training data. I then demonstrate the extension of this learning framework to the case of reactive dynamical system and present numerical experiments for non-isothermal continuous and batch chemical reactors. Lastly, I develop a sequential dynamic nonlinear optimization approach for discovering and performing dimensionality reduction of microkinetic reaction networks.Chemical Engineerin