Numerical solution of a pursuit-evasion differential game involving two spacecraft in low earth orbit

This paper considers a spacecraft pursuit-evasion problem taking place in low earth orbit. The problem is formulated as a zero-sum differential game in which there are two players, a pursuing spacecraft that attempts to minimize a payoff, and an evading spacecraft that attempts to maximize the same payoff. We introduce two associated optimal control problems and show that a saddle point for the differential game exists if and only if the two optimal control problems have the same optimal value. Then, on the basis of this result, we propose two computational methods for determining a saddle point solution: a semi-direct control parameterization method (SDCP method), which is based on a piecewise-constant control approximation scheme, and a hybrid method, which combines the new SDCP method with the multiple shooting method. Simulation results show that the proposed SDCP and hybrid methodsare superior to the semi-direct collocation nonlinear programming method (SDCNLP method), which is widely used to solve pursuit-evasion problems in the aerospace field

Linear Regression Models Applied to Imperfect Information Spacecraft Pursuit-evasion Differential Games

Within satellite rendezvous and proximity operations lies pursuit-evasion differential games between two spacecraft. The extent of possible outcomes can be mathematically bounded by differential games where each player employs optimal strategies. A linear regression model is developed from a large data set of optimal control solutions. The model is shown to map pursuer relative starting positions to final capture positions and estimate capture time. The model is 3.8 times faster than the indirect heuristic method for arbitrary pursuer starting positions on an initial relative orbit about the evader. The linear regression model is shown to be well suited for on-board implementation for autonomous mission planning

Convexity Applications in Single and Multi-Agent Control

The focus of this dissertation is in the application of convexity for control problems; specifically, single-agent problems with linear or nonlinear dynamics and multi-agent problems with linear dynamics. A mixture of convex and non-convex constraints for optimal control problems is also considered. The main contributions of this dissertation include: 1) a convexification of single-agent problems with linear dynamics and annular control constraint, 2) a technique for controlling bounded nonlinear single-agent systems, and 3) a technique for solving multi-agent pursuit-evasion games with linear dynamics and convex control and state constraints. The first result shows that for annularly constrained linear systems, controllability is a sufficient condition for a free or fixed time problem to be solvable as a sequence of convex optimization problems. The second result shows that if a nonlinear system is bounded and “ordered”, it is possible to use a convex combination of bounding linear systems to design a control for the nonlinear system. The third result takes advantage of a convex reachable set computation for each agent in solving games using a geometrical approach. Altogether, the theoretical and computational results demonstrate the significance of convex analysis in solving non-convex control problems

Optimal and Robust Neural Network Controllers for Proximal Spacecraft Maneuvers

Recent successes in machine learning research, buoyed by advances in computational power, have revitalized interest in neural networks and demonstrated their potential in solving complex controls problems. In this research, the reinforcement learning framework is combined with traditional direct shooting methods to generate optimal proximal spacecraft maneuvers. Open-loop and closed-loop feedback controllers, parameterized by multi-layer feed-forward artificial neural networks, are developed with evolutionary and gradient-based optimization algorithms. Utilizing Clohessy- Wiltshire relative motion dynamics, terminally constrained fixed-time, fuel-optimal trajectories are solved for intercept, rendezvous, and natural motion circumnavigation transfer maneuvers using three different thrust models: impulsive, finite, and continuous. In addition to optimality, the neurocontroller performance robustness to parametric uncertainty and bounded initial conditions is assessed. By bridging the gap between existing optimal and nonlinear control techniques, this research demonstrates that neurocontrollers offer a flexible and robust alternative approach to the solution of complex controls problems in the space domain and present a promising path forward to more capable, autonomous spacecraft

A Series Solution Framework for Finite-time Optimal Feedback Control, H-infinity Control and Games

The Bolza-form of the finite-time constrained optimal control problem leads to the Hamilton-Jacobi-Bellman (HJB) equation with terminal boundary conditions and tobe- determined parameters. In general, it is a formidable task to obtain analytical and/or numerical solutions to the HJB equation. This dissertation presents two novel polynomial expansion methodologies for solving optimal feedback control problems for a class of polynomial nonlinear dynamical systems with terminal constraints. The first approach uses the concept of higher-order series expansion methods. Specifically, the Series Solution Method (SSM) utilizes a polynomial series expansion of the cost-to-go function with time-dependent coefficient gains that operate on the state variables and constraint Lagrange multipliers. A significant accomplishment of the dissertation is that the new approach allows for a systematic procedure to generate optimal feedback control laws that exactly satisfy various types of nonlinear terminal constraints. The second approach, based on modified Galerkin techniques for the solution of terminally constrained optimal control problems, is also developed in this dissertation. Depending on the time-interval, nonlinearity of the system, and the terminal constraints, the accuracy and the domain of convergence of the algorithm can be related to the order of truncation of the functional form of the optimal cost function. In order to limit the order of the expansion and still retain improved midcourse performance, a waypoint scheme is developed. The waypoint scheme has the dual advantages of reducing computational efforts and gain-storage requirements. This is especially true for autonomous systems. To illustrate the theoretical developments, several aerospace application-oriented examples are presented, including a minimum-fuel orbit transfer problem. Finally, the series solution method is applied to the solution of a class of partial differential equations that arise in robust control and differential games. Generally, these problems lead to the Hamilton-Jacobi-Isaacs (HJI) equation. A method is presented that allows this partial differential equation to be solved using the structured series solution approach. A detailed investigation, with several numerical examples, is presented on the Nash and Pareto-optimal nonlinear feedback solutions with a general terminal payoff. Other significant applications are also discussed for one-dimensional problems with control inequality constraints and parametric optimization

Guidance and control for defense systems against ballistic threats

A defense system against ballistic threat is a very complex system from the engineering point of view. It involves different kinds of subsystems and, at the same time, it presents very strict requirements. Technology evolution drives the need of constantly upgrading system’s capabilities. The guidance and control fields are two of the areas with the best progress possibilities. This thesis deals with the guidance and control problems involved in a defense system against ballistic threats. This study was undertaken by analyzing the mission of an intercontinental ballistic missile. Trajectory reconstruction from radar and satellite measurements was carried out with an estimation algorithm for nonlinear systems. Knowing the trajectory is a prerequisite for intercepting the ballistic missile. Interception takes place thanks to a dedicated tactical missile. The guidance and control of this missile were also studied in this work. Particular attention was paid on the estimation of engagement’s variables inside the homing loop. Interceptor missiles are usually equipped with a seeker that provides the angle under which the interceptor sees its target. This single measurement does not guarantee the observability of the variables required by advanced guidance laws such as APN, OGL, or differential games-based laws. A new guidance strategy was proposed, that solves the bad observability problems and returns satisfactory engagement performances. The thesis is concluded by a study of the interceptor most suitable aerodynamic configuration in order to implement the proposed strategy, and by the relative autopilot design. The autopilot implements the lateral acceleration commands from the guidance system. The design was carried out with linear control techniques, considering requirements on the rising time, actuators maximum effort, and response to a bang-bang guidance command. The analysis of the proposed solutions was carried on by means of numerical simulations, developed for each single case-study

Game Theoretic Strategies for Spacecraft Rendezvous and Motion Synchronization

Uno dei possibili sviluppi della guida e del controllo relativo nello spazio è quella di estendere gli algoritmi per operazioni di rendezvous e di docking autonome a più veicoli spaziali che collaborano tra di loro. Il problema del rendezvous tra due veicoli spaziali viene risolto utilizzando la teoria dei giochi differenziali lineari quadratici. La dinamica del gioco viene descritta in un sistema di riferimento cartesiano non inerziale. Per estendere l'utilizzo della teoria dei giochi differenziali lineari quadratici alle equazioni non lineari di moto relativo è stata utilizzata le tecnica di parametrizzazione in funzione dello stato o linearizzazione estesa. Nelle simulazioni è stato valutato il confronto tra le prestazioni e le traiettorie ottenute con l'equilibrio di Pareto e quello di Nash quando entrambi i veicoli spaziali agiscono sotto spinta continua. Una strategia sequenziale è stata utilizzata per estendere il gioco differenziali a più di due giocatori per avere la sincronizzazione del moto relativo durante operazioni di assemblaggio nello spazio. One of the main challenges for autonomous spacecraft relative guidance and control is extending the algorithms for autonomous rendezvous and docking (AR&D) operations to multiple collaborative spacecraft. In this thesis, the autonomous rendezvous problem, between two active spacecraft, is formulated as a two player nonzero-sum differential game. The local-vertical local-horizontal (LVLH) rotating reference frame is used to describe the dynamic of the game. The State-Dependent Riccati equation (SDRE) method is applied to extend the Linear Quadratic differential game theory to obtain a feedback control law for nonlinear equation of relative motion. In the simulations both the spacecraft use continuous thrust engines. A comparison among Pareto and Nash equilibrium has been performed. A multiplayer sequential game strategy is used to extend the control law to many spacecraft for relative motion synchronization in an on-orbit self assembly strategy

Game Theoretic Training Enabled Deep Learning Solutions for Rapid Discovery of Satellite Behaviors

The chapter presents a game theoretic training model enabling a deep learning solution for rapid discovery of satellite behaviors from collected sensor data. The solution has two parts, namely, Part 1 and Part 2. Part 1 is a PE game model that enables data augmentation method, and Part 2 uses convolutional neural networks (CNNs) for satellite behavior classification. The sensor data are propagated with the various maneuver strategies from the proposed space game models. Under the PE game theoretic framework, various satellite behaviors are simulated to generate synthetic datasets with labels for the training to detect space object behaviors. To evaluate the performance of the proposed PE model, a CNN model is designed and implemented for satellite behavior classification. Python 3 and TensorFlow are used in this implementation. The simulation results show that the trained machine learning model can efficiently and correctly classify the satellite behaviors up to 99.8%

Sensitivity Methods Applied to Orbital Pursuit-Evasion

In this work, sensitivity methods are examined as a means to solve and analyze the problem of orbital pursuit-evasion (PE). Orbital PE is a two-sided spacecraft trajectory optimization problem characterized by high dimensionality and nonlinearity. Modern methods for solving problems of this sort employ generic, computationally intensive techniques, including random search methods such as the genetic algorithm; collocation methods based on discrete approximation; or combinations of these methods. The advantages of these methods are relatively high degrees of robustness, straightforward implementation, and ease of handling state and control constraints. Yet we note the disadvantages: chiefly high computation load, as well as absence of insight into the problem, and accuracy of the result. Sensitivity methods provide corresponding strengths in each of these areas. We present novel sensitivity analysis techniques that may be useful in other optimization problems featuring high dimensionality, nonlinearity, and/or state and control constraints. The techniques shown include a novel solution method; a computationally efficient feedback control technique; a means of sketching barrier surfaces; and the use of hybrid one-sided/two-sided controllers for sophisticated emergent behavior. We also introduce a new formulation of the problem incorporating a minimum-altitude constraint, and we make an initial investigation of a sensitivity-based method of handling state constraints. Overall, our results suggest that sensitivity methods can provide useful augmentation to techniques that rely more heavily upon computational power, and may be particularly valuable for implementation in an onboard control algorithm

AFIT School of Engineering Contributions to Air Force Research and Technology. Calendar Year 1971

This report contains abstracts of Master of Science theses and Doctoral Dissertations completed during the 1971 calendar year at the School of Engineering, Air Force Institute of Technology