Chance-Constrained Control with Lexicographic Deep Reinforcement Learning

This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes, in which the controller seeks to ensure that the probability of satisfying the constraint is above a given threshold. Standard DeepRL approaches require i) the constraints to be included as additional weighted terms in the cost function, in a multi-objective fashion, and ii) the tuning of the introduced weights during the training phase of the Deep Neural Network (DNN) according to the probability thresholds. The proposed approach, instead, requires to separately train one constraint-free DNN and one DNN associated to each constraint and then, at each time-step, to select which DNN to use depending on the system observed state. The presented solution does not require any hyper-parameter tuning besides the standard DNN ones, even if the probability thresholds changes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations

Optimal departure aircraft trajectories minimising population annoyance

Best Paper in Track Award de l’International Congress on Research in Air TransportThis paper presents a strategy for designing noise abatement procedures aimed at reducing the global annoyance perceived for the population living around the airports. By using fuzzy logic techniques it is shown how annoyance can be modelled in function of the maximum perceived noise level at a specific noise sensitive location and the time of the day when the departure takes place. Thus, the annoyance is computed for different kinds of sensibility areas, such as residential zones, industrial zones, schools or hospitals and an annoyance figure is obtained for each possible trajectory. Then, a non-linear multiobjective optimal control problem is presented in order to obtain the minimum annoyance trajectory for all sensitive locations. Lexicographic optimisation is used to cope with the difficulties that arise when several criteria appear in the optimisation process. Finally, a practical example is given for an hypothetical scenario where different optimal trajectories are obtained at different day periods.Peer ReviewedAward-winnin

Advanced wind farm control strategies for enhancing grid support

Nowadays, there is rising concern among Transmission System Operators about the declining of system inertia due to the increasing penetration of wind energy, and other renewable energy systems, and the retirements of conventional power plants. On the other hand, by properly operating wind farms, wind generation may be capable of enhancing grid stability and ensuring continued security of power supply. In this doctoral thesis, new control approaches for designing wind farm optimization-based control strategies are proposed to improve the participation of wind farms in grid support, specially in primary frequency support.Hoy en día, existe una significativa preocupación entre los Operadores de Sistemas de Transmisión sobre la cresciente penetración de le energía eólica y la tendiente eliminación de las centrales eléctricas convencionales que implica la disminución de la inercia del sistema eléctrico. Operando adecuadamente los parques eólicos, la generación eólica puede mejorar la estabilidad de la red eléctrica y garantizar la seguridad y un continuo suministro de energía. Esta tesis doctoral propone nuevas estrategias para diseñar controladores basados en optimización dinámica para parques eólicos y mejorar la participación de los parques eólicos en el soporte de la red eléctrica. En primer lugar, esta tesis doctoral presenta los enfoques clásicos para el control de parques y turbinas eólicas y cómo los conceptos de control existentes pueden ser explotados para hacer frente a los nuevos desafíos que se esperan de los parques eólicos. Esta tesis doctoral asigna un interés especial a cómo formular la función objetivo de que la reserva de potencia sea maximizada, para ayudar por el suporte de frequencia, y al mismo tiempo seguir la potencia demandada por la red. Sin embargo, el impacto de la estela de viento generada por una turbina sobre otras turbinas necesita ser minimizado para mejorar la reserva de potencia. Por lo tanto, a lo largo de esta tesis se proponen estrategias de control centralizado para parques eólicos enfocadas en distribuir óptimamente la energía entre las turbinas para que el impacto negativo de la estela en la reserva de potencia total se reduzca. Se discuten dos técnicas de control para proporcionar los objetivos de control mencionados anteriormente. Un algoritmo de control óptimo para encontrar la mejor distribución de potencia entre las turbinas en el parque mientras se resuelve un problema iterativo de programación lineal. En segundo lugar, se utiliza la técnica de control predictivo basada en modelo para resolver un problema de control multi-objetivo que también podría incluir, junto con la maximización de reserva de potencia, otros objetivos de control, tales como la minimización de las perdidas eléctricas en los cables de la red de interconexión entre turbinas y un controlador/supervisor. Además, la investigación realizada resalta la capacidad de las estrategias de control propuestas en esta tesis para proporcionar mayor reserva de potencia respecto a los conceptos comúnmente usados para distribuir la potencia total del parque eólico. La idea principal detrás del diseño de una estrategia de control de parques eólico es de encontrar una solución óptima dentro de un cálculo computacional relativamente bajo. Aunque los controladores centralizados propuestos en esta tesis reaccionan rápidamente a los cambios en la potencia de referencia enviada desde el controlador, algunos problemas pueden ocurrir cuando se consideran parques eólicos de gran escala debido a los retrasos existentes por el viento entre turbinas. Bajo estas circunstancias, la producción de energía de cada turbina está altamente acoplada con la propagación de la estela y, por ende, con las condiciones de funcionamiento de las otras turbinas. Esta tesis doctoral propone un esquema de control de parques eólicos no centralizados basado en una estrategia de partición para dividir el parque eólico en sub-conjuntos independientes de turbinas. Con la estrategia de control propuesta, el tiempo de cálculo se reduce adecuadamente en comparación con la estrategia de control centralizado mientras que el rendimiento en la distribución óptima de potencia es ligeramente afectado. El rendimiento de todas las estrategias de control propuestas en esta tesis se prueba con un simulador de parque eólico que modela el comportamiento dinámico del efecto de estela mediante el uso de un conocido y consolidado modelo dinámico de estela y, para un análisis más realista, algunas simulaciones se realizan con software avanzado basado en la técnica de Large Eddy Simulation

Advanced wind farm control strategies for enhancing grid support

Aplicat embargament des de la data de defensa fins al maig 2020Nowadays, there is rising concern among Transmission System Operators about the declining of system inertia due to the increasing penetration of wind energy, and other renewable energy systems, and the retirements of conventional power plants. On the other hand, by properly operating wind farms, wind generation may be capable of enhancing grid stability and ensuring continued security of power supply. In this doctoral thesis, new control approaches for designing wind farm optimization-based control strategies are proposed to improve the participation of wind farms in grid support, specially in primary frequency support.Hoy en día, existe una significativa preocupación entre los Operadores de Sistemas de Transmisión sobre la cresciente penetración de le energía eólica y la tendiente eliminación de las centrales eléctricas convencionales que implica la disminución de la inercia del sistema eléctrico. Operando adecuadamente los parques eólicos, la generación eólica puede mejorar la estabilidad de la red eléctrica y garantizar la seguridad y un continuo suministro de energía. Esta tesis doctoral propone nuevas estrategias para diseñar controladores basados en optimización dinámica para parques eólicos y mejorar la participación de los parques eólicos en el soporte de la red eléctrica. En primer lugar, esta tesis doctoral presenta los enfoques clásicos para el control de parques y turbinas eólicas y cómo los conceptos de control existentes pueden ser explotados para hacer frente a los nuevos desafíos que se esperan de los parques eólicos. Esta tesis doctoral asigna un interés especial a cómo formular la función objetivo de que la reserva de potencia sea maximizada, para ayudar por el suporte de frequencia, y al mismo tiempo seguir la potencia demandada por la red. Sin embargo, el impacto de la estela de viento generada por una turbina sobre otras turbinas necesita ser minimizado para mejorar la reserva de potencia. Por lo tanto, a lo largo de esta tesis se proponen estrategias de control centralizado para parques eólicos enfocadas en distribuir óptimamente la energía entre las turbinas para que el impacto negativo de la estela en la reserva de potencia total se reduzca. Se discuten dos técnicas de control para proporcionar los objetivos de control mencionados anteriormente. Un algoritmo de control óptimo para encontrar la mejor distribución de potencia entre las turbinas en el parque mientras se resuelve un problema iterativo de programación lineal. En segundo lugar, se utiliza la técnica de control predictivo basada en modelo para resolver un problema de control multi-objetivo que también podría incluir, junto con la maximización de reserva de potencia, otros objetivos de control, tales como la minimización de las perdidas eléctricas en los cables de la red de interconexión entre turbinas y un controlador/supervisor. Además, la investigación realizada resalta la capacidad de las estrategias de control propuestas en esta tesis para proporcionar mayor reserva de potencia respecto a los conceptos comúnmente usados para distribuir la potencia total del parque eólico. La idea principal detrás del diseño de una estrategia de control de parques eólico es de encontrar una solución óptima dentro de un cálculo computacional relativamente bajo. Aunque los controladores centralizados propuestos en esta tesis reaccionan rápidamente a los cambios en la potencia de referencia enviada desde el controlador, algunos problemas pueden ocurrir cuando se consideran parques eólicos de gran escala debido a los retrasos existentes por el viento entre turbinas. Bajo estas circunstancias, la producción de energía de cada turbina está altamente acoplada con la propagación de la estela y, por ende, con las condiciones de funcionamiento de las otras turbinas. Esta tesis doctoral propone un esquema de control de parques eólicos no centralizados basado en una estrategia de partición para dividir el parque eólico en sub-conjuntos independientes de turbinas. Con la estrategia de control propuesta, el tiempo de cálculo se reduce adecuadamente en comparación con la estrategia de control centralizado mientras que el rendimiento en la distribución óptima de potencia es ligeramente afectado. El rendimiento de todas las estrategias de control propuestas en esta tesis se prueba con un simulador de parque eólico que modela el comportamiento dinámico del efecto de estela mediante el uso de un conocido y consolidado modelo dinámico de estela y, para un análisis más realista, algunas simulaciones se realizan con software avanzado basado en la técnica de Large Eddy Simulation.Postprint (published version

Modeling and real-time control of urban drainage systems: A review

Urban drainage systems (UDS) may be considered large-scale systems given their large number of associated states and decision actions, making challenging their real-time control (RTC) design. Moreover, the complexity of the dynamics of the UDS makes necessary the development of strategies for the control design. This paper reviews and discusses several techniques and strategies commonly used for the control of UDS. Moreover, the models to describe, simulate, and control the transport of wastewater in UDS are also reviewed.This work has been partially supported by Mexichem, Colombia through the project “Drenaje Urbano y Cambio Climático: Hacia los Sistemas de Alcantarillado del Futuro.” Fase II, with reference No. 548-2012, the scholarships of Colciencias No. 567-2012 and 647-2013, and the project ECOCIS (Ref. DPI2013-48243-C2-1-R).Peer Reviewe

Non-linear economic model predictive control of water distribution networks

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/This paper addresses a non-linear economic model predictive control (EMPC) strategy for water distribution networks (WDNs). A WDN could be considered as a non-linear system described by differential-algebraic equations (DAEs) when flow and hydraulic head equations are considered. As in other process industries, the main operational goal of WDNs is the minimisation of the economic costs associated to pumping and water treatment, while guaranteeing water supply with required flows and pressures at all the control/demand nodes in the network. Other operational goals related to safety and reliability are usually sought. From a control point of view, EMPC is a suitable control strategy for WDNs since the optimal operation of the network cannot be established a priori by fixing reference volumes in the tanks. Alternatively, the EMPC strategy should determine the optimal filling/emptying sequence of the tanks taking into account that electricity price varies between day and night and that the demand also follows a 24-hour repetitive pattern. On the other hand, as a result of the ON/OFF operation of parallel pumps in pumping stations, a two-layer control scheme has been used: a non-linear EMPC strategy with hourly control interval is chosen in the upper layer and a pump scheduling approach with one-minute sampling time in the lower layer. Finally, closed-loop simulation results of applying the proposed control strategy to the D-Town water network are shown.Peer ReviewedPostprint (author's final draft

Supply chain management for the process industry

This thesis investigates some important problems in the supply chain management (SCM) for the process industry to fill the gap in the literature work, covering production planning and scheduling, production, distribution planning under uncertainty, multiobjective supply chain optimisation and water resources management in the water supply chain planning. To solve these problems, models and solution approaches are developed using mathematical programming, especially mixed-integer linear programming (MILP), techniques. First, the medium-term planning of continuous multiproduct plants with sequence-dependent changeovers is addressed. An MILP model is developed using Travelling Salesman Problem (TSP) classic formulation. A rolling horizon approach is also proposed for large instances. Compared with several literature models, the proposed models and approaches show significant computational advantage. Then, the short-term scheduling of batch multiproduct plants is considered. TSP-based formulation is adapted to model the sequence-dependent changeovers between product groups. An edible-oil deodoriser case study is investigated. Later, the proposed TSP-based formulation is incorporated into the supply chain planning with sequence-dependent changeovers and demand elasticity of price. Model predictive control (MPC) is applied to the production, distribution and inventory planning of supply chains under demand uncertainty. A multiobjective optimisation problem for the production, distribution and capacity planning of a global supply chain of agrochemicals is also addressed, considering cost, responsiveness and customer service level as objectives simultaneously. Both ε- constraint method and lexicographic minimax method are used to find the Pareto-optimal solutions Finally, the integrated water resources management in the water supply chain management is addressed, considering desalinated water, wastewater and reclaimed water, simultaneously. The optimal production, distribution and storage systems are determined by the proposed MILP model. Real cases of two Greek islands are studied

Modelling and predictive control techniques for building heating systems

Model predictive control (MPC) has often been referred to in literature as a potential method for more efficient control of building heating systems. Though a significant performance improvement can be achieved with an MPC strategy, the complexity introduced to the commissioning of the system is often prohibitive. Models are required which can capture the thermodynamic properties of the building with sufficient accuracy for meaningful predictions to be made. Furthermore, a large number of tuning weights may need to be determined to achieve a desired performance. For MPC to become a practicable alternative, these issues must be addressed. Acknowledging the impact of the external environment as well as the interaction of occupants on the thermal behaviour of the building, in this work, techniques have been developed for deriving building models from data in which large, unmeasured disturbances are present. A spatio-temporal filtering process was introduced to determine estimates of the disturbances from measured data, which were then incorporated with metaheuristic search techniques to derive high-order simulation models, capable of replicating the thermal dynamics of a building. While a high-order simulation model allowed for control strategies to be analysed and compared, low-order models were required for use within the MPC strategy itself. The disturbance estimation techniques were adapted for use with system-identification methods to derive such models. MPC formulations were then derived to enable a more straightforward commissioning process and implemented in a validated simulation platform. A prioritised-objective strategy was developed which allowed for the tuning parameters typically associated with an MPC cost function to be omitted from the formulation by separation of the conflicting requirements of comfort satisfaction and energy reduction within a lexicographic framework. The improved ability of the formulation to be set-up and reconfigured in faulted conditions was shown

Adaptive Multi-objective Optimizing Flight Controller

The problem of synthesizing online optimal flight controllers in the presence of multiple objectives is considered. A hybrid adaptive-optimal control architecture is presented, which is suitable for implementation on systems with fast, nonlinear and uncertain dynamics subject to constraints. The problem is cast as an adaptive Multi-Objective Optimization (MO-Op) flight control problem wherein control policy is sought that attempts to optimize over multiple, sometimes conflicting objectives. A solution strategy utilizing Gaussian Process (GP)-based adaptive-optimal control is presented, in which the system uncertainties are learned with an online updated budgeted GP. The mean of the GP is used to feedback-linearize the system and reference model shaping Model Predictive Control (MPC) is utilized for optimization. To make the MO-Op problem online-realizable, a relaxation strategy that poses some objectives as adaptively updated soft constraints is proposed. The strategy is validated on a nonlinear roll dynamics model with simulated state-dependent flexible-rigid mode interaction. In order to demonstrate low probability of failure in the presence of stochastic uncertainty and state constraints, we can take advantage of chance-constrained programming in Model Predictive Control. The results for the single objective case of chance-constrained MPC is also shown to reflect the low probability of constraint violation in safety critical systems such as aircrafts. Optimizing the system over multiple objectives is only one application of the adapive-optimal controller. Another application we considered using the adaptive-optimal controller setup is to design an architecture capable of adapting to the dynamics of different aerospace platforms. This architecture brings together three key elements, MPC-based reference command shaping, Gaussian Process (GP)-based Bayesian nonparametric Model Reference Adaptive Control (MRAC) which both were used in the previous application as well, and online GP clustering over nonstationary (time-varying) GPs. The key salient feature of our architecture is that not only can it detect changes, but it uses online GP clustering to enable the controller to utilize past learning of similar models to significantly reduce learning transients. Stability of the architecture is argued theoretically and performance is validated empirically.Mechanical & Aerospace Engineerin