Approximate dynamic programming based solutions for fixed-final-time optimal control and optimal switching

Optimal solutions with neural networks (NN) based on an approximate dynamic programming (ADP) framework for new classes of engineering and non-engineering problems and associated difficulties and challenges are investigated in this dissertation. In the enclosed eight papers, the ADP framework is utilized for solving fixed-final-time problems (also called terminal control problems) and problems with switching nature. An ADP based algorithm is proposed in Paper 1 for solving fixed-final-time problems with soft terminal constraint, in which, a single neural network with a single set of weights is utilized. Paper 2 investigates fixed-final-time problems with hard terminal constraints. The optimality analysis of the ADP based algorithm for fixed-final-time problems is the subject of Paper 3, in which, it is shown that the proposed algorithm leads to the global optimal solution providing certain conditions hold. Afterwards, the developments in Papers 1 to 3 are used to tackle a more challenging class of problems, namely, optimal control of switching systems. This class of problems is divided into problems with fixed mode sequence (Papers 4 and 5) and problems with free mode sequence (Papers 6 and 7). Each of these two classes is further divided into problems with autonomous subsystems (Papers 4 and 6) and problems with controlled subsystems (Papers 5 and 7). Different ADP-based algorithms are developed and proofs of convergence of the proposed iterative algorithms are presented. Moreover, an extension to the developments is provided for online learning of the optimal switching solution for problems with modeling uncertainty in Paper 8. Each of the theoretical developments is numerically analyzed using different real-world or benchmark problems --Abstract, page v

Optimal and Robust Neural Network Controllers for Proximal Spacecraft Maneuvers

Recent successes in machine learning research, buoyed by advances in computational power, have revitalized interest in neural networks and demonstrated their potential in solving complex controls problems. In this research, the reinforcement learning framework is combined with traditional direct shooting methods to generate optimal proximal spacecraft maneuvers. Open-loop and closed-loop feedback controllers, parameterized by multi-layer feed-forward artificial neural networks, are developed with evolutionary and gradient-based optimization algorithms. Utilizing Clohessy- Wiltshire relative motion dynamics, terminally constrained fixed-time, fuel-optimal trajectories are solved for intercept, rendezvous, and natural motion circumnavigation transfer maneuvers using three different thrust models: impulsive, finite, and continuous. In addition to optimality, the neurocontroller performance robustness to parametric uncertainty and bounded initial conditions is assessed. By bridging the gap between existing optimal and nonlinear control techniques, this research demonstrates that neurocontrollers offer a flexible and robust alternative approach to the solution of complex controls problems in the space domain and present a promising path forward to more capable, autonomous spacecraft

Stochastic Model Predictive Control via Fixed Structure Policies

In this work, the model predictive control problem is extended to include not only open-loop control sequences but also state-feedback control laws by directly optimizing parameters of a control policy. Additionally, continuous cost functions are developed to allow training of the control policy in making discrete decisions, which is typically done with model-free learning algorithms. This general control policy encompasses a wide class of functions and allows the optimization to occur both online and offline while adding robustness to unmodelled dynamics and outside disturbances. General formulations regarding nonlinear discrete-time dynamics and abstract cost functions are formed for both deterministic and stochastic problems. Analytical solutions are derived for linear cases and compared to existing theory, such as the classical linear quadratic regulator. It is shown that, given some assumptions hold, there exists a finite horizon in which a constant linear state-feedback control law will stabilize a nonlinear system around the origin. Several control policy architectures are used to regulate the cart-pole system in deterministic and stochastic settings, and neural network-based policies are trained to analyze and intercept bodies following stochastic projectile motion

Sufficient Conditions for Optimal Control Problems with Terminal Constraints and Free Terminal Times with Applications to Aerospace

Motivated by the flight control problem of designing control laws for a Ground Collision Avoidance System (GCAS), this thesis formulates sufficient conditions for a strong local minimum for a terminally constrained optimal control problem with a free-terminal time. The conditions develop within the framework of a construction of a field of extremals by means of the method of characteristics, a procedure for the solution of first-order linear partial differential equations, but modified to apply to the Hamilton-Jacobi-Bellman equation of optimal control. Additionally, the thesis constructs these sufficient conditions for optimality with a mathematically rigorous development. The proof uses an approach which generalizes and differs significantly from procedures outlined in the classical literature on control engineering, where similar formulas are derived, but only in a cursory, formal and sometimes incomplete way. Additionally, the thesis gives new arrangements of the relevant expressions arising in the formulation of sufficient conditions for optimality that lead to more concise formulas for the resulting perturbation feedback control schemes. These results are applied to an emergency perturbation-feedback guidance scheme which recovers an aircraft from a dangerous flight-path angle to a safe one. Discussion of required background material contrasts nonlinear and linear optimal control theory are contrasted in the context of aerospace applications. A simplified version of the classical model for an F-16 fighter aircraft is used in numerical computation to very, by example, that the sufficient conditions for optimality developed in this thesis can be used off-line to detect possible failures in perturbation feedback control schemes, which arise if such methods are applied along extremal controlled trajectories and which only satisfy the necessary conditions for optimality without being locally optimal. The sufficient conditions for optimality developed in this thesis, on the other hand, guarantee the local validity of such perturbation feedback control schemes. This thesis presents various graphs that compare the neighboring extremals which were derived from the perturbation feedback control scheme with optimal ones that start from the same initial condition. Future directions for this work include extending the perturbation feedback control schemes to optimization problems that are further constrained, possibly through control constraints, state-space constraints or mixed state-control constraints

On-board Trajectory Computation for Mars Atmospheric Entry Based on Parametric Sensitivity Analysis of Optimal Control Problems

This thesis develops a precision guidance algorithm for the entry of a small capsule into the atmosphere of Mars. The entry problem is treated as nonlinear optimal control problem and the thesis focuses on developing a suboptimal feedback law. Therefore parametric sensitivity analysis of optimal control problems is combined with dynamic programming. This approach enables a real-time capable, locally suboptimal feedback scheme. The optimal control problem is initially considered in open loop fashion. To synthesize the feedback law, the optimal control problem is embedded into a family of neighboring problems, which are described by a parameter vector. The optimal solution for a nominal set of parameters is determined using direct optimization methods. In addition the directional derivatives (sensitivities) of the optimal solution with respect to the parameters are computed. Knowledge of the nominal solution and the sensitivities allows, under certain conditions, to apply Taylor series expansion to approximate the optimal solution for disturbed parameters almost instantly. Additional correction steps can be applied to improve the optimality of the solution and to eliminate errors in the constraints. To transfer this strategy to the closed loop system, the computation of the sensitivities is performed with respect to different initial conditions. Determining the perturbation direction and interpolating between sensitivities of neighboring initial conditions allows the approximation of the extremal field in a neighborhood of the nominal trajectory. This constitutes a locally suboptimal feedback law. The proposed strategy is applied to the atmospheric entry problem. The developed algorithm is part of the main control loop, i.e. optimal controls and trajectories are computed at a fixed rate, taking into account the current state and parameters. This approach is combined with a trajectory tracking controller based on the aerodynamic drag. The performance and the strengthsa and weaknesses of this two degree of freedom guidance system are analyzed using Monte Carlo simulation. Finally the real-time capability of the proposed algorithm is demonstrated in a flight representative processor-in-the-loop environment

Multi-agent Collision Avoidance Using Interval Analysis and Symbolic Modelling with its Application to the Novel Polycopter

Coordination is fundamental component of autonomy when a system is defined by multiple mobile agents. For unmanned aerial systems (UAS), challenges originate from their low-level systems, such as their flight dynamics, which are often complex. The thesis begins by examining these low-level dynamics in an analysis of several well known UAS using a novel symbolic component-based framework. It is shown how this approach is used effectively to define key model and performance properties necessary of UAS trajectory control. This is demonstrated initially under the context of linear quadratic regulation (LQR) and model predictive control (MPC) of a quadcopter. The symbolic framework is later extended in the proposal of a novel UAS platform, referred to as the ``Polycopter" for its morphing nature. This dual-tilt axis system has unique authority over is thrust vector, in addition to an ability to actively augment its stability and aerodynamic characteristics. This presents several opportunities in exploitative control design. With an approach to low-level UAS modelling and control proposed, the focus of the thesis shifts to investigate the challenges associated with local trajectory generation for the purpose of multi-agent collision avoidance. This begins with a novel survey of the state-of-the-art geometric approaches with respect to performance, scalability and tolerance to uncertainty. From this survey, the interval avoidance (IA) method is proposed, to incorporate trajectory uncertainty in the geometric derivation of escape trajectories. The method is shown to be more effective in ensuring safe separation in several of the presented conditions, however performance is shown to deteriorate in denser conflicts. Finally, it is shown how by re-framing the IA problem, three dimensional (3D) collision avoidance is achieved. The novel 3D IA method is shown to out perform the original method in three conflict cases by maintaining separation under the effects of uncertainty and in scenarios with multiple obstacles. The performance, scalability and uncertainty tolerance of each presented method is then examined in a set of scenarios resembling typical coordinated UAS operations in an exhaustive Monte-Carlo analysis

Efficient algorithms for risk-averse air-ground rendezvous missions

Demand for fast and inexpensive parcel deliveries in urban environments has risen considerably in recent years. A framework is envisioned to enforce efficient last-mile delivery in urban environments by leveraging a network of ride-sharing vehicles, where Unmanned Aerial Systems (UASs) drop packages on said vehicles, which then cover the majority of the distance before final aerial delivery. By combining existing networks we show that the range and efficiency of UAS-based delivery logistics are greatly increased. This approach presents many engineering challenges, including the safe rendezvous of both agents: the UAS and the human-operated ground vehicle. This dissertation presents tools that guarantee risk-optimal rendezvous between the two vehicles. We present mechanical and algorithmic tools that achieve this goal. Mechanically, we develop a novel aerial manipulator and controller that improves in-flight stability during the pickup and drop-off of packages. At a higher level and the core of this dissertation, we present planning algorithms that mitigate risks associated with human behavior at the longest time scales. First, we discuss the downfalls of traditional approaches. In aerial manipulation, we show that popular anthropomorphic designs are unsuitable for flying platforms, which we tackle with a combination of lightweight design of a delta-type parallel manipulator, and L1 adaptive control with feedforward. In planning algorithms, we present evidence of erratic driver behavior that can lead to catastrophic failures. Such a failure occurs when the UAS depletes its resource (battery, fuel) and has to crash land on an unplanned location. This is particularly dangerous in urban environments where population density is high, and the probability of harming a person or property in the event of a failure is unsafe. Studies have shown that two types of erratic behavior are common: speed variation and route choice. Speed variation refers to a common disregard for speed limits combined with different levels of comfort per driver. Route choice is conscious, unconscious, or purely random action of deviating from a prescribed route. Route choice uncertainty is high dimensional and complex both in space and time. Dealing with these types of uncertainty is important to many fields, namely traffic flow modeling. The critical difference to our interpretation is that we frame them in a motion planning framework. As such, we assume each driver has an unknown stochastic model for their behavior, a model that we aim to approximate through different methods. We aim to guarantee safety by quantifying motion planning risks associated with erratic human behavior. Only missions that plan on using all of the UAS's resources have inherent risk. We postulate that if we have a high assurance of success, any mission can be made to use more resources and be more efficient for the network by completing its objective faster. Risk management is addressed at three different scales. First, we focus on speed variation. We approach this problem with a combination of risk-averse Model Predictive Control (MPC) and Gaussian Processes. We use risk as a measure of the probability of success, centered around estimated future driver position. Several risk measures are discussed and CVaR is chosen as a robust measure for this problem. Second we address local route choice. This is route uncertainty for a single driver in some region of space. The primary challenge is the loss of gradient for the MPC controller. We extend the previous approach with a cross-entropy stochastic optimization algorithm that separates gradient-based from gradient-free optimization problems within the planner. We show that this approach is effective through a variety of numerical simulations. Lastly, we study a city-wide problem of estimating risk among several available drivers. We use real-world data combined with synthetic experiments and Deep Neural Networks (DNN) to produce an accurate estimator. The main challenges in this approach are threefold: DNN architecture, driver model, and data processing. We found that this learning problem suffers from vanishing gradients and numerous local minima, which we address with modern self-normalization techniques and mean-adjusted CVaR. We show the model's effectiveness in four scenarios of increasing complexity and propose ways of addressing its shortcomings

The influence of the sideslip target on the performance of vehicles with actively controlled handling

The influence of sideslip on the handling capability of a four wheeled vehicle is investigated. Both nonlinear, steady-state and linear, transient analyses are conducted on simple models in order to understand how the geometric and inertial effects of sideslip control influence the maneuvering capability of the vehicle. Nonlinear performance analyses confirm the findings of the literature, that constant sideslip angle at the centre of mass is required if it is desired to maintain consistent vehicle 'balance' with increasing lateral acceleration, and the reason for this is explained using simple mathematics. Analyses of energy flow between the power source and the various sinks of the vehicle show that for a typical modem vehicle, the power dissipated in a steady turn near the limiting lateral acceleration is approximately comparable in magnitude to that dissipated by aerodynamic drag near the maximum speed of the vehicle. Additionally, it shown that whenever brake control, rather than steering control, is employed to generate a yawing moment, the component of dissipated energy associated with this yaw demand is larger by at least an order of magnitude. It is concluded that whenever the required dynamic behaviour can be delivered by means of steering alone pure steering control should be preferred over the use of direct yaw control. This suggests that direct yaw control should only be used when the limit of the envelope of the steered vehicle has been reached. Transient analyses of sudden turn-in events are then undertaken. The assumption is that the driver wishes to maximise the lateral displacement of the vehicle as quickly as possible. Vehicle handling models with A WS are linearised and discretised, and Linear Progranuning is used to identifY the optimal turn-in maneuver. The objective is to understand how to make a vehicle perform well against such a target without any use of any energy-dissipating direct yaw control. It is observed that the optimal controls usually involve an immediate step to the limiting force that the front axle is able to deliver. It is shown that for vehicles with yaw dynamics where this input does not lead to saturation of the rear tyres, the transient performance is totally insensitive to changes in the enforced sideslip control. The form of this optimal force input is then used in a further mathematical analysis of the optimal obstacle avoidance maneuver. It is shown that in the case mentioned above, where sufficient friction is available at the rear axle, the time taken to build up lateral acceleration and yaw rate for a turn is a simple function of the geometric and inertial properties of the vehicle, and unrelated to rear tyre cornering stiffness, rear camber or rear steering control. It is shown also shown that for an equal level of limit over- or under-steer, 2WS vehicles that are limit over-steering are able to turn in more quickly than those which are limit under-steering, since the excess friction is available at the front axle, and can be used during the turn-in phase. Further, it is shown that both commonly adopted sideslip targets for 4WS vehicles and responses that often result from 2WS vehicles can easily be 'incompatible' with the handling envelope of a steered vehicle from an optimal obstacle avoidance point of view. This means that for some vehicles, strict enforcement of such sideslip targets directly increases the time taken to transfer such a vehicle to the limiting lateral acceleration. This limit of 'compatibility' of the sideslip target and vehicle envelope is confirmed analytically. It is then shown, that the zero sideslip target which is commonly adopted for A WS vehicles in the literature, and which was previously shown to be the ideal for consistent vehicle stability and 'balance', is only able to deliver the optimal turn-in behaviour when the underlying vehicle has a limit-neutral or limit under-steering balance. Further, the zero sideslip target requires a strongly limit under-steering balance if the sideslip target is to be maintained when the vehicle is rnaneuvered from turning quickly in one direction to turning quickly in the other without compromising the time taken to complete the maneuver. However, it is also shown that either a controlled front differential, or front axle direct yaw-moment control are each able to extend the envelope of the vehicle in the necessary direction that maintaining zero sideslip throughout such transients may become feasible, albeit at an energy cost that increases as the vehicle is maneuvered more rapidly. Additionally, an alternative sideslip target is presented, that allows optimal maneuvering to take place whilst the sideslip target is simultaneously maintained, without requiring the intervention of controlled differentials or direct yaw control.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

High-Performance Modelling and Simulation for Big Data Applications

This open access book was prepared as a Final Publication of the COST Action IC1406 “High-Performance Modelling and Simulation for Big Data Applications (cHiPSet)“ project. Long considered important pillars of the scientific method, Modelling and Simulation have evolved from traditional discrete numerical methods to complex data-intensive continuous analytical optimisations. Resolution, scale, and accuracy have become essential to predict and analyse natural and complex systems in science and engineering. When their level of abstraction raises to have a better discernment of the domain at hand, their representation gets increasingly demanding for computational and data resources. On the other hand, High Performance Computing typically entails the effective use of parallel and distributed processing units coupled with efficient storage, communication and visualisation systems to underpin complex data-intensive applications in distinct scientific and technical domains. It is then arguably required to have a seamless interaction of High Performance Computing with Modelling and Simulation in order to store, compute, analyse, and visualise large data sets in science and engineering. Funded by the European Commission, cHiPSet has provided a dynamic trans-European forum for their members and distinguished guests to openly discuss novel perspectives and topics of interests for these two communities. This cHiPSet compendium presents a set of selected case studies related to healthcare, biological data, computational advertising, multimedia, finance, bioinformatics, and telecommunications

The Nexus between Artificial Intelligence and Economics

This book is organized as follows. Section 2 introduces the notion of the Singularity, a stage in development in which technological progress and economic growth increase at a near-infinite rate. Section 3 describes what artificial intelligence is and how it has been applied. Section 4 considers artificial happiness and the likelihood that artificial intelligence might increase human happiness. Section 5 discusses some prominent related concepts and issues. Section 6 describes the use of artificial agents in economic modeling, and section 7 considers some ways in which economic analysis can offer some hints about what the advent of artificial intelligence might bring. Chapter 8 presents some thoughts about the current state of AI and its future prospects.