Robust Engineering of Dynamic Structures in Complex Networks

Populations of nearly identical dynamical systems are ubiquitous in natural and engineered systems, in which each unit plays a crucial role in determining the functioning of the ensemble. Robust and optimal control of such large collections of dynamical units remains a grand challenge, especially, when these units interact and form a complex network. Motivated by compelling practical problems in power systems, neural engineering and quantum control, where individual units often have to work in tandem to achieve a desired dynamic behavior, e.g., maintaining synchronization of generators in a power grid or conveying information in a neuronal network; in this dissertation, we focus on developing novel analytical tools and optimal control policies for large-scale ensembles and networks. To this end, we first formulate and solve an optimal tracking control problem for bilinear systems. We developed an iterative algorithm that synthesizes the optimal control input by solving a sequence of state-dependent differential equations that characterize the optimal solution. This iterative scheme is then extended to treat isolated population or networked systems. We demonstrate the robustness and versatility of the iterative control algorithm through diverse applications from different fields, involving nuclear magnetic resonance (NMR) spectroscopy and imaging (MRI), electrochemistry, neuroscience, and neural engineering. For example, we design synchronization controls for optimal manipulation of spatiotemporal spike patterns in neuron ensembles. Such a task plays an important role in neural systems. Furthermore, we show that the formation of such spatiotemporal patterns is restricted when the network of neurons is only partially controllable. In neural circuitry, for instance, loss of controllability could imply loss of neural functions. In addition, we employ the phase reduction theory to leverage the development of novel control paradigms for cyclic deferrable loads, e.g., air conditioners, that are used to support grid stability through demand response (DR) programs. More importantly, we introduce novel theoretical tools for evaluating DR capacity and bandwidth. We also study pinning control of complex networks, where we establish a control-theoretic approach to identifying the most influential nodes in both undirected and directed complex networks. Such pinning strategies have extensive practical implications, e.g., identifying the most influential spreaders in epidemic and social networks, and lead to the discovery of degenerate networks, where the most influential node relocates depending on the coupling strength. This phenomenon had not been discovered until our recent study

Annealing Optimization for Progressive Learning with Stochastic Approximation

In this work, we introduce a learning model designed to meet the needs of applications in which computational resources are limited, and robustness and interpretability are prioritized. Learning problems can be formulated as constrained stochastic optimization problems, with the constraints originating mainly from model assumptions that define a trade-off between complexity and performance. This trade-off is closely related to over-fitting, generalization capacity, and robustness to noise and adversarial attacks, and depends on both the structure and complexity of the model, as well as the properties of the optimization methods used. We develop an online prototype-based learning algorithm based on annealing optimization that is formulated as an online gradient-free stochastic approximation algorithm. The learning model can be viewed as an interpretable and progressively growing competitive-learning neural network model to be used for supervised, unsupervised, and reinforcement learning. The annealing nature of the algorithm contributes to minimal hyper-parameter tuning requirements, poor local minima prevention, and robustness with respect to the initial conditions. At the same time, it provides online control over the performance-complexity trade-off by progressively increasing the complexity of the learning model as needed, through an intuitive bifurcation phenomenon. Finally, the use of stochastic approximation enables the study of the convergence of the learning algorithm through mathematical tools from dynamical systems and control, and allows for its integration with reinforcement learning algorithms, constructing an adaptive state-action aggregation scheme.Comment: arXiv admin note: text overlap with arXiv:2102.0583

Adaptive dynamic programming with eligibility traces and complexity reduction of high-dimensional systems

This dissertation investigates the application of a variety of computational intelligence techniques, particularly clustering and adaptive dynamic programming (ADP) designs especially heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Moreover, a one-step temporal-difference (TD(0)) and n-step TD (TD(λ)) with their gradients are utilized as learning algorithms to train and online-adapt the families of ADP. The dissertation is organized into seven papers. The first paper demonstrates the robustness of model order reduction (MOR) for simulating complex dynamical systems. Agglomerative hierarchical clustering based on performance evaluation is introduced for MOR. This method computes the reduced order denominator of the transfer function by clustering system poles in a hierarchical dendrogram. Several numerical examples of reducing techniques are taken from the literature to compare with our work. In the second paper, a HDP is combined with the Dyna algorithm for path planning. The third paper uses DHP with an eligibility trace parameter (λ) to track a reference trajectory under uncertainties for a nonholonomic mobile robot by using a first-order Sugeno fuzzy neural network structure for the critic and actor networks. In the fourth and fifth papers, a stability analysis for a model-free action-dependent HDP(λ) is demonstrated with batch- and online-implementation learning, respectively. The sixth work combines two different gradient prediction levels of critic networks. In this work, we provide a convergence proofs. The seventh paper develops a two-hybrid recurrent fuzzy neural network structures for both critic and actor networks. They use a novel n-step gradient temporal-difference (gradient of TD(λ)) of an advanced ADP algorithm called value-gradient learning (VGL(λ)), and convergence proofs are given. Furthermore, the seventh paper is the first to combine the single network adaptive critic with VGL(λ). --Abstract, page iv

Advanced Modeling, Design, and Control of ac-dc Microgrids

An interconnected dc grid that comprises resistive and constant-power loads (CPLs) that is fed by Photovoltaic (PV) units is studied first. All the sources and CPLs are connected to the grid via dc-dc buck converters. Nonlinear behavior of PV units in addition to the effect of the negative-resistance CPLs can destabilize the dc grid. A decentralized nonlinear model and control are proposed where an adaptive output-feedback controller is employed to stabilize the dc grid with assured stability through Lyapunov stability method while each converter employs only local measurements. Adaptive Neural Networks (NNs) are utilized to overcome the unknown dynamics of the dc-dc converters at Distributed Energy Resources (DERs) and CPLs and those of the interconnected network imposed on the converters. Additionally, the use of the output feedback control makes possible the utilization of other measured signals, in case of loss of main signal, at the converter location and creates measurement redundancy that improves reliability of the dc network. The switching between measurement signals of different types are performed through using the NNs without the need to further tuning. Then, in a small-scale ac grid, PV-based Distributed Generation (DG) units, including dc/dc converters and inverters, are controlled such that mimic a synchronous generator behavior. While other control schemes such as Synchronverters are used to control the inverter frequency and power at a fixed dc link voltage, the proposed approach considers both the dc-link voltage and the inverter ac voltage and frequency regulation. The dc-link capacitor stores kinetic energy similar to the rotor of a synchronous generator, providing inertia and contributes to the system stability. Additionally, a reduced Unified Power Flow Controller (UPFC) structure is proposed to enhance transient stability of small-scale micro grids. The reduced UPFC model exploits dc link of the DG unit to generate appropriate series voltage and inject it to the power line to enhance transient stability. It employs optimal control to ensure that the stability of the system is realized through minimum cost for the system. A neural network is used to approximate the cost function based on the weighted residual method

Machine Learning

Machine Learning can be defined in various ways related to a scientific domain concerned with the design and development of theoretical and implementation tools that allow building systems with some Human Like intelligent behavior. Machine learning addresses more specifically the ability to improve automatically through experience

Health Management and Adaptive Control of Distributed Spacecraft Systems

As the development of challenging missions like on-orbit construction and collaborative inspection that involve multi-spacecraft systems increases, the requirements needed to improve post-failure safety to maintain the mission performance also increases, especially when operating under uncertain conditions. In particular, space missions that involve Distributed Spacecraft Systems (e.g, inspection, repairing, assembling, or deployment of space assets) are susceptible to failures and threats that are detrimental to the overall mission performance. This research applies a distributed Health Management System that uses a bio-inspired mechanism based on the Artificial Immune System coupled with a Support Vector Machine to obtain an optimized health monitoring system capable of detecting nominal and off-nominal system conditions. A simulation environment is developed for a fleet of spacecraft performing a low-Earth orbit inspection within close proximity of a target space asset, where the spacecraft observers follow stable relative orbits with respect to the target asset, allowing dynamics to be expressed using the Clohessy-Wiltshire-Hill equations. Additionally, based on desired points of inspection, the observers have specific attitude requirements that are achieved using Reaction Wheels as the control moment device. An adaptive control based on Deep Reinforcement Learning using an Actor-Critic-Adverse architecture is implemented to achieve high levels of mission protection, especially under disturbances that might lead to performance degradation. Numerical simulations to evaluate the capabilities of the health management architecture when the spacecraft network is subjected to failures are performed. A comparison of different attitude controllers such as Nonlinear Dynamic Inversion and Pole Placement against Deep Reinforcement Learning based controller is presented. The Dynamic Inversion controller showed better tracking performance but large control effort, while the Deep Reinforcement controller showed satisfactory tracking performance with minimal control effort. Numerical simulations successfully demonstrated the potential of both the bioinspired Health Monitoring System architecture and the controller, to detect and identify failures and overcome bounded disturbances, respectively

Models for Human Navigation and Optimal Path Planning Using Level Set Methods and Hamilton-Jacobi Equations

We present several models for different physical scenarios which are centered around human movement or optimal path planning, and use partial differential equations and concepts from control theory. The first model is a game-theoretic model for environmental crime which tracks criminals' movement using the level set method, and improves upon previous continuous models by removing overly restrictive assumptions of symmetry. Next, we design a method for determining optimal hiking paths in mountainous regions using an anisotropic level set equation. After this, we present a model for optimal human navigation with uncertainty which is rooted in dynamic programming and stochastic optimal control theory. Lastly, we consider optimal path planning for simple, self-driving cars in the Hamilton-Jacobi formulation. We improve upon previous models which simplify the car to a point mass, and present a reasonably general upwind, sweeping scheme to solve the relevant Hamilton-Jacobi equation

Overcoming the timescale barrier in molecular dynamics: Transfer operators, variational principles and machine learning

One of the main challenges in molecular dynamics is overcoming the ‘timescale barrier’: in many realistic molecular systems, biologically important rare transitions occur on timescales that are not accessible to direct numerical simulation, even on the largest or specifically dedicated supercomputers. This article discusses how to circumvent the timescale barrier by a collection of transfer operator-based techniques that have emerged from dynamical systems theory, numerical mathematics and machine learning over the last two decades. We will focus on how transfer operators can be used to approximate the dynamical behaviour on long timescales, review the introduction of this approach into molecular dynamics, and outline the respective theory, as well as the algorithmic development, from the early numerics-based methods, via variational reformulations, to modern data-based techniques utilizing and improving concepts from machine learning. Furthermore, its relation to rare event simulation techniques will be explained, revealing a broad equivalence of variational principles for long-time quantities in molecular dynamics. The article will mainly take a mathematical perspective and will leave the application to real-world molecular systems to the more than 1000 research articles already written on this subject