Hybrid Solution of Stochastic Optimal Control Problems using Gauss Pseudospectral Method and Generalized Polynomial Chaos Algorithms

Two numerical methods, Gauss Pseudospectral Method and Generalized Polynomial Chaos Algorithm, were combined to form a hybrid algorithm for solving nonlinear optimal control and optimal path planning problems with uncertain parameters. The algorithm was applied to two concept demonstration problems: a nonlinear optimal control problem with multiplicative uncertain elements and a mission planning problem sponsored by USSTRATCOM. The mission planning scenario was constructed to find the path that minimizes the probability of being killed by lethal threats whose locations are uncertain to statistically quantify the effects those uncertainties have on the flight path solution, and to use the statistical properties to estimate the probability that the vehicle will be killed during mission execution. The results demonstrated that the method is able to effectively characterize how the optimal solution changes with uncertainty and that the results can be presented in a form that can be used by mission planners and aircrews to assess risks associated with a mission profile

Robust aircraft trajectory optimization under meteorological uncertainty

Mención Internacional en el título de doctorThe Air Traffic Management (ATM) system in the busiest airspaces in the world is currently being overhauled to deal with multiple capacity, socioeconomic, and environmental challenges. One major pillar of this process is the shift towards a concept of operations centered on aircraft trajectories (called Trajectory-Based Operations or TBO in Europe) instead of rigid airspace structures. However, its successful implementation (and, thus, the realization of the associated improvements in ATM performance) rests on appropriate understanding and management of uncertainty. Due to its complex socio-technical structure, the design and operations of the ATM system are heavily impacted by uncertainty, proceeding from multiple sources and propagating through the interconnections between its subsystems. One major source of ATM uncertainty is weather. Due to its nonlinear and chaotic nature, a number of meteorological phenomena of interest cannot be forecasted with complete accuracy at arbitrary lead times, which leads to uncertainty or disruption in individual air and ground operations that propagates to all ATM processes. Therefore, in order to achieve the goals of SESAR and similar programs, it is necessary to deal with meteorological uncertainty at multiple scales, from the trajectory prediction and planning processes to flow and traffic management operations. This thesis addresses the problem of single-aircraft flight planning considering two important sources of meteorological uncertainty: wind prediction error and convective activity. As the actual wind field deviates from its forecast, the actual trajectory will diverge in time from the planned trajectory, generating uncertainty in arrival times, sector entry and exit times, and fuel burn. Convective activity also impacts trajectory predictability, as it leads pilots to deviate from their planned route, creating challenging situations for controllers. In this work, we aim to develop algorithms and methods for aircraft trajectory optimization that are able to integrate information about the uncertainty in these meteorological phenomena into the flight planning process at both pre-tactical (before departure) and tactical horizons (while the aircraft is airborne), in order to generate more efficient and predictable trajectories. To that end, we frame flight planning as an optimal control problem, modeling the motion of the aircraft with a point-mass model and the BADA performance model. Optimal control methods represent a flexible and general approach that has a long history of success in the aerospace field. As a numerical scheme, we use direct methods, which can deal with nonlinear systems of moderate and high-dimensional state spaces in a computationally manageable way. Nevertheless, while this framework is well-developed in the context of deterministic problems, the techniques for the solution of practical optimal control problems under uncertainty are not as mature, and the methods proposed in the literature are not applicable to the flight planning problem as it is now understood. The first contribution of this thesis addresses this challenge by introducing a framework for the solution of general nonlinear optimal control problems under parametric uncertainty. It is based on an ensemble trajectory scheme, where the trajectories of the system under multiple scenarios are considered simultaneously within the same dynamical system and the uncertain optimal control problem is turned into a large conventional optimal control problem that can be then solved by standard, well-studied direct methods in optimal control. We then employ this approach to solve the robust flight plan optimization problem at the planning horizon. In order to model uncertainty in the wind and estimating the probability of convective conditions, we employ Ensemble Prediction System (EPS) forecasts, which are composed by multiple predictions instead of a single deterministic one. The resulting method can be used to optimize flight plans for maximum expected efficiency according to the cost structure of the airline; additionally, predictability and exposure to convection can be incorporated as additional objectives. The inherent tradeoffs between these objectives can be assessed with this methodology. The second part of this thesis presents a solution for the rerouting of aircraft in uncertain convective weather scenarios at the tactical horizon. The uncertain motion of convective weather cells is represented with a stochastic model that has been developed from the output of a deterministic satellite-based nowcast product, Rapidly Developing Thunderstorms (RDT). A numerical optimal control framework, based on the pointmass model with the addition of turn dynamics, is employed for optimizing efficiency and predictability of the proposed trajectories in the presence of uncertainty about the future evolution of the storm. Finally, the optimization process is initialized by a randomized heuristic procedure that generates multiple starting points. The combined framework is able to explore and as exploit the space of solution trajectories in order to provide the pilot or the air traffic controller with a set of different suggested avoidance trajectories, as well as information about their expected cost and risk. The proposed methods are tested on example scenarios based on real data, showing how different user priorities lead to different flight plans and what tradeoffs are then present. These examples demonstrate that the solutions described in this thesis are adequate for the problems that have been formulated. In this way, the flight planning process can be enhanced to increase the efficiency and predictability of individual aircraft trajectories, which would lead to higher predictability levels of the ATM system and thus improvements in multiple performance indicators.El sistema de gestión del tráfico aéreo (Air Traffic Management, ATM) en los espacios aéreos más congestionados del mundo está siendo reformado para lidiar con múltiples desafíos socioeconómicos, medioambientales y de capacidad. Un pilar de este proceso es el gradual reemplazo de las estructuras rígidas de navegación, basadas en aerovías y waypoints, hacia las operaciones basadas en trayectorias. No obstante, la implementación exitosa de este concepto y la realización de las ganancias esperadas en rendimiento ATM requiere entender y gestionar apropiadamente la incertidumbre. Debido a su compleja estructura socio-técnica, el diseño y operaciones del sistema ATM se encuentran marcadamente influidos por la incertidumbre, que procede de múltiples fuentes y se propaga por las interacciones entre subsistemas y operadores humanos. Uno de los principales focos de incertidumbre en ATM es la meteorología. Debido a su naturaleza no-linear y caótica, muchos fenómenos de interés no pueden ser pronosticados con completa precisión en cualquier horizonte temporal, lo que crea disrupción en las operaciones en aire y tierra que se propaga a otros procesos de ATM. Por lo tanto, para lograr los objetivos de SESAR e iniciativas análogas, es imprescindible tener en cuenta la incertidumbre en múltiples escalas espaciotemporales, desde la predicción de trayectorias hasta la planificación de flujos y tráfico. Esta tesis aborda el problema de la planificación de vuelo de aeronaves individuales considerando dos fuentes importantes de incertidumbre meteorológica: el error en la predicción del viento y la actividad convectiva. Conforme la realización del viento se desvía de su previsión, la trayectoria real se desviará temporalmente de la planificada, lo que implica incertidumbre en tiempos de llegada a sectores y aeropuertos y en consumo de combustible. La actividad convectiva también tiene un impacto en la predictibilidad de las trayectorias, puesto que obliga a los pilotos a desviarse de sus planes de vuelo para evitarla, cambiado así la situación de tráfico. En este trabajo, buscamos desarrollar métodos y algoritmos para la optimización de trayectorias que puedan integrar información sobre la incertidumbre en estos fenómenos meteorológicos en el proceso de diseño de planes de vuelo en horizontes de planificación (antes del despegue) y tácticos (durante el vuelo), con el objetivo de generar trayectorias más eficientes y predecibles. Con este fin, formulamos la planificación de vuelo como un problema de control óptimo, modelando la dinámica del avión con un modelo de masa puntual y el modelo de rendimiento BADA. El control óptimo es un marco flexible y general con un largo historial de éxito en el campo de la ingeniería aeroespacial. Como método numérico, empleamos métodos directos, que son capaces de manejar sistemas dinámicos de alta dimensión con costes computacionales moderados. No obstante, si bien esta metodología es madura en contextos deterministas, la solución de problemas prácticas de control óptimo bajo incertidumbre en la literatura no está tan desarrollada, y los métodos propuestos en la literatura no son aplicables al problema de interés. La primera contribución de esta tesis hace frente a este reto mediante la introducción de un marco numérico para la resolución de problemas generales de control óptimo no-lineal bajo incertidumbre paramétrica. El núcleo de este método es un esquema de conjunto de trayectorias, en el que las trayectorias del sistema dinámico bajo múltiples escenarios son consideradas de forma simultánea, y el problema de control óptimo bajo incertidumbre es así transformado en un problema convencional que puede ser tratado mediante métodos existentes en control óptimo. A continuación, empleamos este método para resolver el problema de la planificación de vuelo robusta. La incertidumbre en el viento y la probabilidad de ocurrencia de condiciones convectivas son modeladas mediante el uso de previsiones de conjunto o ensemble, compuestas por múltiples predicciones en lugar de una única previsión determinista. Este método puede ser empleado para maximizar la eficiencia esperada de los planes de vuelo de acuerdo a la estructura de costes de la aerolínea; además, la predictibilidad de la trayectoria y la exposición a la convección pueden ser incorporadas como objetivos adicionales. El trade-off entre estos objetivos puede ser evaluado mediante la metodología propuesta. La segunda parte de la tesis presenta una solución para reconducir aviones en escenarios tormentosos en un horizonte táctico. La evolución de las células convectivas es representada con un modelo estocástico basado en las proyecciones de Rapidly Developing Thunderstorms (RDT), un sistema determinista basado en imágenes de satélite. Este modelo es empleado por un método de control óptimo numérico, basado en un modelo de masa puntual en el que se modela la dinámica de viraje, con el objetivo de maximizar la eficiencia y predictibilidad de la trayectoria en presencia de incertidumbre sobre la evolución futura de las tormentas. Finalmente, el proceso de optimizatión es inicializado por un método heurístico aleatorizado que genera múltiples puntos de inicio para las iteraciones del optimizador. Esta combinación permite explorar y explotar el espacio de trayectorias solución para proporcionar al piloto o al controlador un conjunto de trayectorias propuestas, así como información útil sobre su coste y el riesgo asociado. Los métodos propuestos son probados en escenarios de ejemplo basados en datos reales, ilustrando las diferentes opciones disponibles de acuerdo a las prioridades del planificador y demostrando que las soluciones descritas en esta tesis son adecuadas para los problemas que se han formulado. De este modo, es posible enriquecer el proceso de planificación de vuelo para incrementar la eficiencia y predictibilidad de las trayectorias individuales, lo que contribuiría a mejoras en el rendimiento del sistema ATM.These works have been financially supported by Universidad Carlos III de Madrid through a PIF scholarship; by Eurocontrol, through the HALA! Research Network grant 10-220210-C2; by the Spanish Ministry of Economy and Competitiveness (MINECO)'s R&D program, through the OptMet project (TRA2014-58413-C2-2-R); and by the European Commission's SESAR Horizon 2020 program, through the TBO-Met project (grant number 699294).Programa de Doctorado en Mecánica de Fluidos por la Universidad Carlos III de Madrid; la Universidad de Jaén; la Universidad de Zaragoza; la Universidad Nacional de Educación a Distancia; la Universidad Politécnica de Madrid y la Universidad Rovira iPresidente: Damián Rivas Rivas.- Secretario: Xavier Prats Menéndez.- Vocal: Benavar Sridha

Fault tolerant flight control system design for unmanned aerial vehicles

Safety and reliability of air vehicles is of the utmost importance. This is particularly true for large civil transport aircraft where a large number of human lives depend on safety critical design. With the increase in the use of unmanned aerial vehicles (UAVs) in our airspace it is essential that UAV safety is also given attention to prevent devastating failures which could ultimately lead to loss of human lives. While civil aircraft have human operators, the pilot, to counteract any unforeseen faults, autonomous UAVs are only as good as the on board flight computer. Large civil aircraft also have the luxury of weight hence redundant actuators (control surfaces) can be installed and in the event of a faulty set of actuators the redundant actuators can be brought into action to negate the effects of any faults. Again weight is a luxury that UAVs do not have. The main objective of this research is to study the design of a fault tolerant flight controller that can exploit the mathematical redundancies in the flight dynamic equations as opposed to adding hardware redundancies that would result in significant weight increase. This thesis presents new research into fault tolerant control for flight vehicles. Upon examining the flight dynamic equations it can be seen, for example, that an aileron, which is primarily used to perform a roll manoeuvre, can be used to execute a limited pitch moment. Hence a control method is required that moves away from the traditional fixed structure model where control surface roles are clearly defined. For this reason, in this thesis, I have chosen to study the application of model predictive control (MPC) to fault tolerant control systems. MPC is a model based method where a model of the plant forms an integral part of the controller. An optimisation is performed based on model estimations of the plant and the inputs are chosen via an optimisation process. One of the main contributions of this thesis is the development of a nonlinear model predictive controller for fault tolerant flight control. An aircraft is a highly nonlinear system hence if a nonlinear model can be integrated into the control process the cross-coupling effects of the control surface contributions can be easily exploited. An active fault tolerant control system comprises not only of the fault tolerant controller but also a fault detection and isolation subsystem. A common fault detection method is based on parameter estimation using filtering techniques. The solution proposed in this thesis uses an unscented Kalman filter (UKF) for parameter estimation and controller updates. In summary the main contribution of this thesis is the development of a new active fault tolerant flight control system. This new innovative controller exploits the idea of analytical redundancy as opposed to hardware redundancy. It comprises of a nonlinear model predictive based controller using pseudospectral discretisation to solve the nonlinear optimal control problem. Furthermore a UKF is incorporated into the design of the active fault tolerant flight control system

A review of safe online learning for nonlinear control systems

Learning for autonomous dynamic control systems that can adapt to unforeseen environmental changes are of great interest but the realisation of a practical and safe online learning algorithm is incredibly challenging. This paper highlights some of the main approaches for safe online learning of stabilisable nonlinear control systems with a focus on safety certification for stability. We categorise a non-exhaustive list of salient techniques, with a focus on traditional control theory as opposed to reinforcement learning and approximate dynamic programming. This paper also aims to provide a simplified overview of techniques as an introduction to the field. It is the first paper to our knowledge that compares key attributes and advantages of each technique in one paper

Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation

A learning approach for optimal feedback gains for nonlinear continuous time control systems is proposed and analysed. The goal is to establish a rigorous framework for computing approximating optimal feedback gains using neural networks. The approach rests on two main ingredients. First, an optimal control formulation involving an ensemble of trajectories with 'control' variables given by the feedback gain functions. Second, an approximation to the feedback functions via realizations of neural networks. Based on universal approximation properties we prove the existence and convergence of optimal stabilizing neural network feedback controllers.Comment: 55 pages, 13 figure

Numerical optimal control with applications in aerospace

This thesis explores various computational aspects of solving nonlinear, continuous-time dynamic optimization problems (DOPs) numerically. Firstly, a direct transcription method for solving DOPs is proposed, named the integrated residual method (IRM). Instead of forcing the dynamic constraints to be satisfied only at a selected number of points as in direct collocation, this new approach alternates between minimizing and constraining the squared norm of the dynamic constraint residuals integrated along the whole solution trajectories. The method is capable of obtaining solutions of higher accuracy for the same mesh compared to direct collocation methods, enabling a flexible trade-off between solution accuracy and optimality, and providing reliable solutions for challenging problems, including those with singular arcs and high-index differential-algebraic equations. A number of techniques have also been proposed in this work for efficient numerical solution of large scale and challenging DOPs. A general approach for direct implementation of rate constraints on the discretization mesh is proposed. Unlike conventional approaches that may lead to singular control arcs, the solution of this on-mesh implementation has better numerical properties, while achieving computational speedups. Another development is related to the handling of inactive constraints, which do not contribute to the solution of DOPs, but increase the problem size and burden the numerical computations. A strategy to systematically remove the inactive and redundant constraints under a mesh refinement framework is proposed. The last part of this work focuses on the use of DOPs in aerospace applications, with a number of topics studied. Using example scenarios of intercontinental flights, the benefits of formulating DOPs directly according to problem specifications are demonstrated, with notable savings in fuel usage. The numerical challenges with direct collocation are also identified, with the IRM obtaining solutions of higher accuracy, and at the same time suppressing the singular arc fluctuations.Open Acces

Optimal control and model reduction for wave energy systems: A moment-based approach

Following the sharp increase in the price of traditional fossil fuels, in combination with issues of security of supply, and pressure to honor greenhouse gas emission limits, much attention has turned to renewable energy sources in recent years. Ocean wave energy is a massive and untapped resource, which can make a valuable contribution towards a sustainable, global, energy mix. Despite the fact that ocean waves constitute a vast resource, wave energy converters (WECs) have yet to make significant progress towards commercialisation. One stepping stone to achieve this objective is the availability of appropriate control technology, suchthatenergyconversionisperformedaseconomicallyaspossible,minimisingthedelivered energy cost, while also maintaining the structural integrity of the device, minimising wear on WEC components, and operating across a wide range of sea conditions. Suitable energy-maximising control technology depends upon the availability of two fundamental ‘pieces’: A control-oriented dynamical model, describing the motion of the WEC, and a model-based optimal control framework, able to efficiently compute the corresponding energy-maximising control law, subject to a set of constraints, defined according to the physical limitations of the device. FollowingtherequirementsforsuccessfulWECcontrol,andbothusingandextendingkeytools arising from the framework of model reduction by moment-matching, this thesis presents two main contributions. Firstly, this monograph proposes a comprehensive moment-based model reduction framework, tailored for WEC systems, addressing linear and nonlinear model reduction cases, providing a systematic method to compute control-oriented models from complex target structures. These approximating models inherit steady-state response characteristics of the target system, via the proposed moment-matching reduction framework. Secondly, by recognising that, besides being a powerful model reduction tool, the parameterisation of the steady-state response of a system in terms of moment-based theory can be explicitly used to transcribe the energy-maximising control problem to a finite-dimensional nonlinear program, a comprehensive moment-based optimal control framework, tailored for WEC systems, is proposed. This framework considers both linear and nonlinear optimal control cases, while also including robust solutions with respect to both system, and input uncertainty, providing an efficient method to compute the energy-maximising control law for WECs, under different modelling assumptions. Throughout this thesis both model reduction, and optimal control frameworks, are presented for a general class of WEC devices, and their performance is analysed via multiple case studies, considering different devices, under different sea state conditions

Uncertainty Quantification via Polynomial Chaos Expansion – Methods and Applications for Optimization of Power Systems

Fossil fuels paved the way to prosperity for modern societies, yet alarmingly, we can exploit our planet’s soil only so much. Renewable energy sources inherit the burden to quench our thirst for energy, and to reduce the impact on our environment simultaneously. However, renewables are inherently volatile; they introduce uncertainties. What is the effect of uncertainties on the operation and planning of power systems? What is a rigorous mathematical formulation of the problems at hand? What is a coherent methodology to approaching power system problems under uncertainty? These are among the questions that motivate the present thesis that provides a collection of methods for uncertainty quantification for (optimization of) power systems. We cover power flow (PF) and optimal power flow (OPF) under uncertainty (as well as specific derivative problems). Under uncertainty---we view "uncertainty" as continuous random variables of finite variance---the state of the power system is no longer certain, but a random variable. We formulate PF and OPF problems in terms of random variables, thusly exposing the infinite-dimensional nature in terms of L2-functions. For each problem formulation we discuss a solution methodology that renders the problem tractable: we view the problem as a mapping under uncertainty; uncertainties are propagated through a known mapping. The method we employ to propagate uncertainties is called polynomial chaos expansion (PCE), a Hilbert space technique that allows to represent random variables of finite variance in terms of real-valued coefficients. The main contribution of this thesis is to provide a rigorous formulation of several PF and OPF problems under uncertainty in terms of infinite-dimensional problems of random variables, and to provide a coherent methodology to tackle these problems via PCE. As numerical methods are moot without numerical software another contribution of this thesis is to provide PolyChaos.jl: an open source software package for orthogonal polynomials, quadrature rules, and PCE written in the Julia programming language