The Power of Predictions in Online Control

We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics. In both settings, we characterize the optimal policy and derive tight bounds on the minimum cost and dynamic regret. Perhaps surprisingly, our analysis shows that the conventional greedy MPC approach is a near-optimal policy in both stochastic and adversarial settings. Specifically, for length-$T$ problems, MPC requires only $O(\log T)$ predictions to reach $O(1)$ dynamic regret, which matches (up to lower-order terms) our lower bound on the required prediction horizon for constant regret

Learning Schemes for Power System Protection

In this paper, learning algorithms are leveraged to advance power system protection. Advancements in power system protection have come in different forms such as the development of new control strategies and the introduction of a new system architecture such as a microgrid. In this paper, we propose two learning schemes to make accurate predictions and optimal decisions related to power system protection and microgrid control. First, we present a neural network approach to learn a classifier that can predict stable reconnection timings for an islanded sub-network. Second, we present a learning-based control scheme for power system protection based on the policy rollout. In the proposed scheme, we incorporate online simulation using the commercial PSS/e simulator. Optimal decisions are obtained in real time to prevent cascading failures as well as maximize the load served. We validate our methods with the dynamics simulator and test cases RTS-96 and Poland

An Online Learning Method for Microgrid Energy Management Control*

We propose a novel Model Predictive Control (MPC) scheme based on online-learning (OL) for microgrid energy management, where the control optimisation is embedded as the last layer of the neural network. The proposed MPC scheme deals with uncertainty on the load and renewable generation power profiles and on electricity prices, by employing the predictions provided by an online trained neural network in the optimisation problem. In order to adapt to possible changes in the environment, the neural network is online trained based on continuously received data. The network hyperparameters are selected by performing a hyperparameter optimisation before the deployment of the controller, using a pretraining dataset. We show the effectiveness of the proposed method for microgrid energy management through extensive experiments on real microgrid datasets. Moreover, we show that the proposed algorithm has good transfer learning (TL) capabilities among different microgrids

Asynchronous event driven distributed energy management using profile steering

Distributed Energy Management methodologies with a scheduling approach based on predictions require means to avoid problems related to prediction errors. Various approaches deal with such prediction errors by applying a different online control mechanism, such as a double-sided auction. However, this results in two separate control mechanisms for the planning phase and the real-time control phase. In this paper, we present a two-phase approach with profile steering based control in both phases. The first phase is synchronous and uses predictions to create a planning. The second phase uses profile steering to schedule individual devices in an event driven and asynchronous manner. Simulation results show that this methodology results in an improved power quality and follows the planning better with a RMSE reduction of up to 34%. In addition, it provides more robustness to failure of connection and improves transparency of its actions to prosumers

Power Flow Control of the Grid-Integrated Hybrid DG System using an ARFMF Optimization

A power flow control scheme for a grid-integrated Hybrid DG System (HDGS) is presented in this work, utilizing an advanced random forest algorithm combined with the moth-flame optimization (ARFMF) approach. The proposed control scheme combines the random forest algorithm (RFA) and moth-flame optimization algorithm (MFO) for consolidated execution. The random forest algorithm (RFA), an AI technique, is well-suited for nonlinear systems due to its accurate interpolation and extrapolation capabilities. It is an ensemble learning method that combines multiple decision trees to make predictions. The algorithm constructs a forest of decision trees and aggregates their predictions to produce the final output. The moth-flame optimization (MFO) process is a meta-heuristic optimization procedure inspired by the transverse orientation of moths in nature. It improves initial random solutions and converges to superior positions in the search area. Similarly, the MFO is effective in nonlinear systems as it accurately interpolates and extrapolates arbitrary information. In the proposed technique, the RFA performs the calculation process to determine precise control gains for the HDGS through online implementation based on power variation between the source side and the load side. The recommended dataset is used to implement the AI approach for online execution, reducing optimization process time. The learning process of the RFA is guided by the MFO optimization algorithm. The MFO technique defines the objective function using system information based on equal and unequal constraints, including the accessibility of renewable energy sources, power demand, and state of charge (SOC) of storage systems. Storage devices such as batteries stabilize the energy generated by renewable energy systems to maintain a constant, stable output power. The proposed model is implemented on the MATLAB/Simulink platform, and its execution is compared to previous approaches

Online Algorithms: From Prediction to Decision

Making use of predictions is a crucial, but under-explored, area of sequential decision problems with limited information. While in practice most online algorithms rely on predictions to make real time decisions, in theory their performance is only analyzed in simplified models of prediction noise, either adversarial or i.i.d. The goal of this thesis is to bridge this divide between theory and practice: to study online algorithm under more practical predictions models, gain better understanding about the value of prediction, and design online algorithms that make the best use of predictions. This thesis makes three main contributions. First, we propose a stochastic prediction error model that generalizes prior models in the learning and stochastic control communities, incorporates correlation among prediction errors, and captures the fact that predictions improve as time passes. Using this general prediction model, we prove that Averaging Fixed Horizon Control (AFHC) can simultaneously achieve sublinear regret and constant competitive ratio in expectation using only a constant- sized prediction window, overcoming the hardnesss results in adversarial prediction models. Second, to understand the optimal use of noisy prediction, we introduce a new class of policies, Committed Horizon Control (CHC), that generalizes both popular policies Receding Horizon Control (RHC) and Averaging Fixed Horizon Control (AFHC). Our results provide explicit results characterizing the optimal use of prediction in CHC policy as a function of properties of the prediction noise, e.g., variance and correlation structure. Third, we apply the general prediction model and algorithm design framework to the deferrable load control problem in power systems. Our proposed model predictive algorithm provides significant reduction in variance of total load in the power system. Throughout this thesis, we provide both average-case analysis and concentration results for our proposed online algorithms, highlighting that the typical performance is tightly concentrated around the average-case performance.</p

Online Trajectory Planning Through Combined Trajectory Optimization and Function Approximation: Application to the Exoskeleton Atalante

Autonomous robots require online trajectory planning capability to operate in the real world. Efficient offline trajectory planning methods already exist, but are computationally demanding, preventing their use online. In this paper, we present a novel algorithm called Guided Trajectory Learning that learns a function approximation of solutions computed through trajectory optimization while ensuring accurate and reliable predictions. This function approximation is then used online to generate trajectories. This algorithm is designed to be easy to implement, and practical since it does not require massive computing power. It is readily applicable to any robotics systems and effortless to set up on real hardware since robust control strategies are usually already available. We demonstrate the computational performance of our algorithm on flat-foot walking with the self-balanced exoskeleton Atalante

Energy Efficient Task Mapping and Resource Management on Multi-core Architectures

Reducing energy consumption of parallel applications executing on chip multi- processors (CMPs) is important for green computing. Hardware vendors have been developing a variety of system features to support energy efficient computing, for example, integrating asymmetric core types on a single chip referred to as static asymmetry and supporting dynamic voltage and frequency scaling (DVFS) referred to as dynamic asymmetry.A common parallelization scheme to exploit CMPs is task parallelism, which can express a wide range of computations in the form of task directed acyclic graphs (DAGs). Existing studies that target energy efficient task scheduling have demonstrated the benefits of leveraging DVFS, particularly per-core DVFS. Their scheduling decisions are mainly based on heuristics, such as task criticality, task dependencies and workload sizes. To enable energy efficient task scheduling, we identify multiple crucial factors that influence energy consumption - varying task characteristics, exploitation of intra-task parallelism (task moldability), and task granularity - which we collectively refer to as task heterogeneity. Task heterogeneity and architecture asymmetry features together complicate the task scheduling problem, since the most energy efficient configuration of resource allocation and frequency setting varies with each task. Our analysis shows that leveraging task heterogeneity in conjunction with static and dynamic asymmetry provides significant opportunities for energy reduction.This thesis contributes two scheduling techniques - ERASE and STEER - that target different scenarios. ERASE focuses on fine-grained tasking and in environments where DVFS is not under user control. It leverages the insights of task characteristics, task moldability, and instantaneous task parallelism detection for guiding scheduling decisions. ERASE comprises four modules: online performance modeling, power profiling, core activity tracing and a task scheduler. Online performance modeling and power profiling provide runtime with execution time and power predictions. Core activity tracing offers the instantaneous task parallelism and the task scheduler combines these information to enable the energy predictions and dynamically determine the best resource allocation for each task during runtime. STEER focuses on environments where DVFS is under user control and where the platform comprises multiple asymmetric cores grouped into clusters. STEER explores how much energy could be potentially saved by leveraging static asymmetry, dynamic asymmetry and task heterogeneity in conjunction. STEER comprises two predictive models for performance and power predictions, and a task scheduler that utilizes models for energy predictions and then identifies the best resource allocation and frequency settings for tasks. Moreover, it applies adaptive scheduling techniques based on task granularity to manage DVFS overheads, and coordinates the cluster frequency settings to reduce interference from concurrent running tasks on cluster-based architectures.The evaluation on an NVIDIA Jetson TX2 shows that ERASE achieves 10% energy savings on average compared to the state-of-the-art DVFS-based schedulers and can adapt to external DVFS changes, and STEER consumes 38% less energy on average than both the state-of-the-art and ERASE

OPTIMAL AND ADAPTIVE CONTROL FRAMEWORKS USING REINFORCEMENT LEARNING FOR TIME-VARYING DYNAMICAL SYSTEMS

Performance of complex propulsion and power systems are affected by a vast number of varying factors such as gradual system degradation, engine build differences and changing operating conditions. Owing to these variations, prior characterisation of the system performance metrics such as fuel efficiency function and constraints is infeasible. Existing model-based control approaches are therefore inherently conservative at the expense of the system performance as they are unable to fully characterise the system variations. The system performance characteristics affected by these variations are typically used for health monitoring and maintenance management, but the opportunities to complement the control design have received little attention. It is therefore increasingly important to use the information about the system performance characteristics in the control system design whilst considering the reliability of its implementation. This thesis therefore considers the design of direct adaptive frameworks that exploit emerging diagnostic technologies and enable the direct use of complex performance metrics to deliver self-optimising control systems in the face of disturbances and system variations. These frameworks are termed condition-based control techniques and this thesis extends reinforcement learning (RL) theory which has achieved significant successes in the area of computing and artificial intelligence to the new frameworks and applications. Consequently, an online RL framework was developed for the class of complex propulsion and power systems that make use of the performance metrics to directly learn and adapt the system control. The RL adaptations were further integrated into existing baseline controller structures whilst maintaining the safety and reliability of the underlying system. Furthermore, two online optimal RL tracking control frameworks were developed for time-varying dynamical systems that use a new augmented formulation with integral control. The proposed online RL frameworks advance the state-of-the-art for use in tracking control applications by not making restrictive assumptions on reference model dynamics or use of discounted tracking costs, and guaranteeing zero steady-state tracking error. Finally, an online power management optimisation scheme for hybrid systems that uses a condition-based RL adaptation was developed. The proposed power management optimisation scheme is able to learn and compensate for the gradual system variations and learn online the optimal power management strategy between the hybrid power source given future load predictions. This way, improved system performance is delivered and providing a through-life adaptation strategy