Stochastic and Optimal Distributed Control for Energy Optimization and Spatially Invariant Systems

Improving energy efficiency and grid responsiveness of buildings requires sensing, computing and communication to enable stochastic decision-making and distributed operations. Optimal control synthesis plays a significant role in dealing with the complexity and uncertainty associated with the energy systems. The dissertation studies general area of complex networked systems that consist of interconnected components and usually operate in uncertain environments. Specifically, the contents of this dissertation include tools using stochastic and optimal distributed control to overcome these challenges and improve the sustainability of electric energy systems. The first tool is developed as a unifying stochastic control approach for improving energy efficiency while meeting probabilistic constraints. This algorithm is applied to demonstrate energy efficiency improvement in buildings and improving operational efficiency of virtualized web servers, respectively. Although all the optimization in this technique is in the form of convex optimization, it heavily relies on semidefinite programming (SP). A generic SP solver can handle only up to hundreds of variables. This being said, for a large scale system, the existing off-the-shelf algorithms may not be an appropriate tool for optimal control. Therefore, in the sequel I will exploit optimization in a distributed way. The second tool is itself a concrete study which is optimal distributed control for spatially invariant systems. Spatially invariance means the dynamics of the system do not vary as we translate along some spatial axis. The optimal H2 [H-2] decentralized control problem is solved by computing an orthogonal projection on a class of Youla parameters with a decentralized structure. Optimal H∞ [H-infinity] performance is posed as a distance minimization in a general L∞ [L-infinity] space from a vector function to a subspace with a mixed L∞ and H∞ space structure. In this framework, the dual and pre-dual formulations lead to finite dimensional convex optimizations which approximate the optimal solution within desired accuracy. Furthermore, a mixed L2 [L-2] /H∞ synthesis problem for spatially invariant systems as trade-offs between transient performance and robustness. Finally, we pursue to deal with a more general networked system, i.e. the Non-Markovian decentralized stochastic control problem, using stochastic maximum principle via Malliavin Calculus

Computational Intelligence Approaches for Energy Optimization in Microgrids

The future electrical system termed as smart grid represents a significant paradigm shift for power industry. Nowadays, microgrids are becoming smarter with the integration of renewable energy resources (RESs) , diesel generators , energy storage systems (ESS), and plug-in electric vehicles (PEV or EV) . However, these integration bring with new challenges for intelligent management systems. The classical power generation approaches can no longer be applied to a microgrid with unpredictable renewable energy resources. To relive these problem, a proper power system optimization and a suitable coordination strategy are needed to balance the supply and demand. This thesis presents three projects to study the optimization and control for smart community and to investigate the strategic impact and the energy trading techniques for interconnected microgrids. The first goal of this thesis is to propose a new game-theoretic framework to study the optimization and decision making of multi-players in the distributed power system. The proposed game theoretic special concept-rational reaction set (RRS) is capable to model the game of the distributed energy providers and the large residential consumers. Meanwhile, the residential consumers are able to participate in the retail electricity market to control the market price. Case studies are conducted to validate the system framework using the proposed game theoretic method. The simulation results show the effectiveness and the accuracy of the proposed strategic framework for obtaining the optimum profits for players participating in this market. The second goal of the thesis is to study a distributed convex optimization framework for energy trading of interconnected microgrids to improve the reliability of system operation. In this work, a distributed energy trading approach for interconnected operation of islanded microgrids is studied. Specifically, the system includes several islanded microgrids that can trade energy in a given topology. A distributed iterative deep cut ellipsoid (DCE) algorithm is implemented with limited information exchange. This approach will address the scalability issue and also secure local information on cost functions. During the iterative process, the information exchange among interconnected microgrids is restricted to electricity prices and expected trading energy. Numerical results are presented in terms of the convergent rate of the algorithm for different topologies, and the performance of the DCE algorithm is compared with sub-gradient algorithm. The third goal of this thesis is to use proper optimization approaches to motivate the household consumers to either shift their loads from peaking periods or reduce their consumption. Genetic algorithm (GA) and dynamic programming (DP) based smart appliance scheduling schemes and time-of-use pricing are investigated for comparative studies with demand response

Managing power system congestion and residential demand response considering uncertainty

Electric power grids are becoming increasingly stressed due to political and environmental difficulties in upgrading transmission capacity. This challenge receives even more interest with the paradigm change of increasing renewable energy sources and demand response (DR) programs. Among DR technologies, existing DR programs are primarily designed for industrial and commercial customers. However, household energy consumption accounts for 38% of total electricity consumption in the U.S., suggesting a significant missed opportunity. This dissertation presents an in-depth study to investigate managing power system congestion and residential DR program under uncertainty.First, an interval optimization model is presented for available transfer capability (ATC) evaluation under uncertainties. The conventional approaches of ATC assessment include deterministic and probabilistic methods. However, the proposed interval optimization model can effectively reduce the accuracy requirements on the renewable forecasting, and lead to acceptable interval results by mitigating the impacts of wind forecasting and modeling errors. Second, a distributed and scalable residential DR program is proposed for reducing the peak load at the utility level. The proposed control approach has the following features: 1) it has a distributed control scheme with limited data exchange among agents to ensure scalability and data privacy, and 2) it reduces the utility peak load and customers’ electricity bills while considering household temperature dynamics and network flow.Third, the impacts of weather and customers’ behavior uncertainties on residential DR are also studied in this dissertation. A new stochastic programming-alternating direction method of multipliers (SP-ADMM) algorithm is proposed to solve problems related to weather and uncertain customer behavior. The case study suggests that the performance of residential DR programs can be further improved by considering these stochastic parameters.Finally, a deep deterministic policy gradient-based (DDPG-based) HVAC control strategy is presented for residential DR programs. Simulation results demonstrate that the DDPG-based approach can considerably reduce system peak load, and it requires much less input information than the model-based methods. Also, it only takes each agent less than 3 seconds to make HVAC control actions. Therefore, the proposed approach is applicable to online controls or the cases where accurate building models or weather forecast information are not available

Distributed Market-Grid Coupling Using Model Predictive Control

In this dissertation, a feedback control concept is proposed for modeling a market-grid coupling. The contributions are fourfold: 1) Identification and characterization of an interoperable control between the power market and the power grid; 2) Design of a closed-loop MPC for the market-grid coupling; 3) Extension of the single control loop with a collaborative distributed MPC strategy for coupling distributed markets and grids; 4) Development of an adaptive load forecasting framework

Optimal and Efficient Decision-Making for Power System Expansion Planning

A typical power system consists of three major sectors: generation, transmission, and distribution. Due to ever increasing electricity consumption and aging of the existing components, generation, transmission, and distribution systems and equipment must be analyzed frequently and if needed be replaced and/or expanded timely. By definition, the process of power system expansion planning aims to decide on new as well as upgrading existing system components in order to adequately satisfy the load for a foreseen future. In this dissertation, multiple economically optimal and computationally efficient methods are proposed for expanding power generation, transmission, and distribution systems. First, a computationally efficient model is proposed for transmission expansion planning (TEP). While the existing TEP models use bus voltage angles, the proposed TEP takes advantages of linear sensitivity factors to omit voltage angles from the formulation and replace all nodal power balance constraints by one equivalent constraint. Simulation results show that the proposed model provides the same results as the classical angle-based model while being much faster. Second, a distributed collaborative TEP algorithm for interconnected multi-regional power systems is proposed. The information privacy is respected as each local planner shares limited information related to cross-border tie-lines with its neighboring planners. To coordinate the local planners, a two-level distributed optimization algorithm is proposed based on the concept of analytical target cascading for multidisciplinary design optimization. Third, a security-constrained generation and transmission expansion planning (G&TEP) model with respect to the risk of possible N-1 contingencies is proposed. Using the concept of risk, non-identical probability and severity of individual contingencies are modeled in the proposed G&TEP model. Finally, a mixed-integer linear programming model is proposed for resilient feeder routing in power distribution systems. Geographical information system (GIS) data is used in the proposed model. As it is proven, having GIS facilities will lead to a more cost-efficient and resilient feeder routing scheme than the scheme obtained using electrical points. The proposed model and solution algorithm are comprehensive from several practical aspects such as economic objectives (installation cost, power losses, resiliency), technical constraints (voltage drops, radially constraint, reliability), and geographical constraints (obstacles, right-of-ways)

Structure of complex networks: Quantifying edge-to-edge relations by failure-induced flow redistribution

The analysis of complex networks has so far revolved mainly around the role of nodes and communities of nodes. However, the dynamics of interconnected systems is commonly focalised on edge processes, and a dual edge-centric perspective can often prove more natural. Here we present graph-theoretical measures to quantify edge-to-edge relations inspired by the notion of flow redistribution induced by edge failures. Our measures, which are related to the pseudo-inverse of the Laplacian of the network, are global and reveal the dynamical interplay between the edges of a network, including potentially non-local interactions. Our framework also allows us to define the embeddedness of an edge, a measure of how strongly an edge features in the weighted cuts of the network. We showcase the general applicability of our edge-centric framework through analyses of the Iberian Power grid, traffic flow in road networks, and the C. elegans neuronal network.Comment: 24 pages, 6 figure

Stochastic Optimization For Multi-Agent Statistical Learning And Control

The goal of this thesis is to develop a mathematical framework for optimal, accurate, and affordable complexity statistical learning among networks of autonomous agents. We begin by noting the connection between statistical inference and stochastic programming, and consider extensions of this setup to settings in which a network of agents each observes a local data stream and would like to make decisions that are good with respect to information aggregated across the entire network. There is an open-ended degree of freedom in this problem formulation, however: the selection of the estimator function class which defines the feasible set of the stochastic program. Our central contribution is the design of stochastic optimization tools in reproducing kernel Hilbert spaces that yield optimal, accurate, and affordable complexity statistical learning for a multi-agent network. To obtain this result, we first explore the relative merits and drawbacks of different function class selections. In Part I, we consider multi-agent expected risk minimization this problem setting for the case that each agent seems to learn a common globally optimal generalized linear models (GLMs) by developing a stochastic variant of Arrow-Hurwicz primal-dual method. We establish convergence to the primal-dual optimal pair when either consensus or ``proximity constraints encode the fact that we want all agents\u27 to agree, or nearby agents to make decisions that are close to one another. Empirically, we observe that these convergence results are substantiated but that convergence may not translate into statistical accuracy. More broadly, optimality within a given estimator function class is not the same as one that makes minimal inference errors. The optimality-accuracy tradeoff of GLMs motivates subsequent efforts to learn more sophisticated estimators based upon learned feature encodings of the data that is fed into the statistical model. The specific tool we turn to in Part II is dictionary learning, where we optimize both over regression weights and an encoding of the data, which yields a non-convex problem. We investigate the use of stochastic methods for online task-driven dictionary learning, and obtain promising performance for the task of a ground robot learning to anticipate control uncertainty based on its past experience. Heartened by this implementation, we then consider extensions of this framework for a multi-agent network to each learn globally optimal task-driven dictionaries based on stochastic primal-dual methods. However, it is here the non-convexity of the optimization problem causes problems: stringent conditions on stochastic errors and the duality gap limit the applicability of the convergence guarantees, and impractically small learning rates are required for convergence in practice. Thus, we seek to learn nonlinear statistical models while preserving convexity, which is possible through kernel methods ( Part III). However, the increased descriptive power of nonparametric estimation comes at the cost of infinite complexity. Thus, we develop a stochastic approximation algorithm in reproducing kernel Hilbert spaces (RKHS) that ameliorates this complexity issue while preserving optimality: we combine the functional generalization of stochastic gradient method (FSGD) with greedily constructed low-dimensional subspace projections based on matching pursuit. We establish that the proposed method yields a controllable trade-off between optimality and memory, and yields highly accurate parsimonious statistical models in practice. % Then, we develop a multi-agent extension of this method by proposing a new node-separable penalty function and applying FSGD together with low-dimensional subspace projections. This extension allows a network of autonomous agents to learn a memory-efficient approximation to the globally optimal regression function based only on their local data stream and message passing with neighbors. In practice, we observe agents are able to stably learn highly accurate and memory-efficient nonlinear statistical models from streaming data. From here, we shift focus to a more challenging class of problems, motivated by the fact that true learning is not just revising predictions based upon data but augmenting behavior over time based on temporal incentives. This goal may be described by Markov Decision Processes (MDPs): at each point, an agent is in some state of the world, takes an action and then receives a reward while randomly transitioning to a new state. The goal of the agent is to select the action sequence to maximize its long-term sum of rewards, but determining how to select this action sequence when both the state and action spaces are infinite has eluded researchers for decades. As a precursor to this feat, we consider the problem of policy evaluation in infinite MDPs, in which we seek to determine the long-term sum of rewards when starting in a given state when actions are chosen according to a fixed distribution called a policy. We reformulate this problem as a RKHS-valued compositional stochastic program and we develop a functional extension of stochastic quasi-gradient algorithm operating in tandem with the greedy subspace projections mentioned above. We prove convergence with probability 1 to the Bellman fixed point restricted to this function class, and we observe a state of the art trade off in memory versus Bellman error for the proposed method on the Mountain Car driving task, which bodes well for incorporating policy evaluation into more sophisticated, provably stable reinforcement learning techniques, and in time, developing optimal collaborative multi-agent learning-based control systems

On the Performance and Linear Convergence of Decentralized Primal-Dual Methods

This dissertation studies the performance and linear convergence properties of primal-dual methods for the solution of decentralized multi-agent optimization problems. Decentralized multi-agent optimization is a powerful paradigm that finds applications in diverse fields in learning and engineering design. In these setups, a network of agents is connected through some topology and agents are allowed to share information only locally. Their overall goal is to seek the minimizer of a global optimization problem through localized interactions. In decentralized consensus problems, the agents are coupled through a common consensus variable that they need to agree upon. While in decentralized resource allocation problems, the agents are coupled through global affine constraints. Various decentralized consensus optimization algorithms already exist in the literature. Some methods are derived from a primal-dual perspective, while other methods are derived as gradient tracking mechanisms meant to track the average of local gradients. Among the gradient tracking methods are the adapt-then-combine implementations motivated by diffusion strategies, which have been observed to perform better than other implementations. In this dissertation, we develop a novel adapt-then-combine primal-dual algorithmic framework that captures most state-of-the-art gradient based methods as special cases including all the variations of the gradient-tracking methods. We also develop a concise and novel analysis technique that establishes the linear convergence of this general framework under strongly-convex objectives. Due to our unified framework, the analysis reveals important characteristics for these methods such as their convergence rates and step-size stability ranges. Moreover, the analysis reveals how the augmented Lagrangian penalty term, which is utilized in most of these methods, affects the performance of decentralized algorithms. Another important question that we answer is whether decentralized proximal gradient methods can achieve global linear convergence for non-smooth composite optimization. For centralized algorithms, linear convergence has been established in the presence of a non-smooth composite term. In this dissertation, we close the gap between centralized and decentralized proximal gradient algorithms and show that decentralized proximal algorithms can also achieve linear convergence in the presence of a non-smooth term. Furthermore, we show that when each agent possesses a different local non-smooth term then global linear convergence cannot be established in the worst case. Most works that study decentralized optimization problems assume that all agents are involved in computing all variables. However, in many applications the coupling across agents is sparse in the sense that only a few agents are involved in computing certain variables. We show how to design decentralized algorithms in sparsely coupled consensus and resource allocation problems. More importantly, we establish analytically the importance of exploiting the sparsity structure in coupled large-scale networks