A Geometrical, Reachable Set Approach for Constrained Pursuit–Evasion Games With Multiple Pursuers and Evaders

This paper presents a solution strategy for deterministic time-optimal pursuit–evasion games with linear state constraints, convex control constraints, and linear dynamics that is consistent with linearized relative orbital motion models such as the Clohessy–Wiltshire equations and relative orbital elements. The strategy first generates polytopic inner approximations of the players’ reachable sets by solving a sequence of convex programs. A bisection method then computes the optimal termination time, which is the least time at which a set containment condition is satisfied. The pursuit–evasion games considered are games with (1) a single pursuer and single evader, (2) multiple pursuers and a single evader, and (3) a single pursuer and multiple evaders. Compared to variational methods, this reachable set strategy leads to a tractable formulation even when there are state and control constraints. The efficacy of the strategy is demonstrated in three numerical simulations for a constellation of satellites in close proximity in low earth orbit

Optimal Finite Thrust Guidance Methods for Constrained Satellite Proximity Operations Inspection Maneuvers

Algorithms are developed to find optimal guidance for an inspector satellite operating nearby a resident space object (RSO). For a non-maneuvering RSO, methods are first developed for a satellite subject to maximum slew rates to conduct an initial inspection of an RSO, where the control variables include the throttle level and direction of the thrust. Second, methods are developed to optimally maneuver a satellite with on/off thrusters into a natural motion circumnavigation or teardrop trajectory, subject to lighting and collision constraints. It is shown that for on/off thrusters, a control sequence can be parameterized to a relatively small amount of control variables and the relative states can be analytically propagated as a function of those control variables. For a maneuvering RSO, differential games are formulated and solved for an inspector satellite to achieve multiple inspection goals, such as aligning with the Sun vector or matching the RSO\u27s energy. The developed algorithms lead to fuel and time savings which can increase the mission life and capabilities of inspector satellites and thus improve space situational awareness for the U.S. Air Force

Multi-player pursuit–evasion games with one superior evader

Inspired by the hunting and foraging behaviors of group predators, this paper addresses a class of multi-player pursuit–evasion games with one superior evader, who moves faster than the pursuers. We are concerned with the conditions under which the pursuers can capture the evader, involving the minimum number and initial spatial distribution required as well as the cooperative strategies of the pursuers. We present some necessary or sufficient conditions to regularize the encirclement formed by the pursuers to the evader. Then we provide a cooperative scheme for the pursuers to maintain and shrink the encirclement until the evader is captured. Finally, we give some examples to illustrate the theoretical results

Two-stage pursuit strategy for incomplete-information impulsive space pursuit-evasion mission using reinforcement learning

This paper presents a novel and robust two-stage pursuit strategy for the incomplete-information impulsive space pursuit-evasion missions considering the J2 perturbation. The strategy firstly models the impulsive pursuit-evasion game problem into a far-distance rendezvous stage and a close-distance game stage according to the perception range of the evader. For the far-distance rendezvous stage, it is transformed into a rendezvous trajectory optimization problem and a new objective function is proposed to obtain the pursuit trajectory with the optimal terminal pursuit capability. For the close-distance game stage, a closed-loop pursuit approach is proposed using one of the reinforcement learning algorithms, i.e. the deep deterministic policy gradient algorithm, to solve and update the pursuit trajectory for the incomplete-information impulsive pursuit-evasion missions. The feasibility of this novel strategy and its robustness to different initial states of the pursuer and evader and to the evasion strategies are demonstrated for the sun-synchronous orbit pursuit-evasion game scenarios. The results of the Monte Carlo tests show that the successful pursuit ratio of the proposed method is over 91% for all the given scenario

Convexity Applications in Single and Multi-Agent Control

The focus of this dissertation is in the application of convexity for control problems; specifically, single-agent problems with linear or nonlinear dynamics and multi-agent problems with linear dynamics. A mixture of convex and non-convex constraints for optimal control problems is also considered. The main contributions of this dissertation include: 1) a convexification of single-agent problems with linear dynamics and annular control constraint, 2) a technique for controlling bounded nonlinear single-agent systems, and 3) a technique for solving multi-agent pursuit-evasion games with linear dynamics and convex control and state constraints. The first result shows that for annularly constrained linear systems, controllability is a sufficient condition for a free or fixed time problem to be solvable as a sequence of convex optimization problems. The second result shows that if a nonlinear system is bounded and “ordered”, it is possible to use a convex combination of bounding linear systems to design a control for the nonlinear system. The third result takes advantage of a convex reachable set computation for each agent in solving games using a geometrical approach. Altogether, the theoretical and computational results demonstrate the significance of convex analysis in solving non-convex control problems

Game Theoretic Training Enabled Deep Learning Solutions for Rapid Discovery of Satellite Behaviors

The chapter presents a game theoretic training model enabling a deep learning solution for rapid discovery of satellite behaviors from collected sensor data. The solution has two parts, namely, Part 1 and Part 2. Part 1 is a PE game model that enables data augmentation method, and Part 2 uses convolutional neural networks (CNNs) for satellite behavior classification. The sensor data are propagated with the various maneuver strategies from the proposed space game models. Under the PE game theoretic framework, various satellite behaviors are simulated to generate synthetic datasets with labels for the training to detect space object behaviors. To evaluate the performance of the proposed PE model, a CNN model is designed and implemented for satellite behavior classification. Python 3 and TensorFlow are used in this implementation. The simulation results show that the trained machine learning model can efficiently and correctly classify the satellite behaviors up to 99.8%

Air Force Institute of Technology Research Report 2020

This Research Report presents the FY20 research statistics and contributions of the Graduate School of Engineering and Management (EN) at AFIT. AFIT research interests and faculty expertise cover a broad spectrum of technical areas related to USAF needs, as reflected by the range of topics addressed in the faculty and student publications listed in this report. In most cases, the research work reported herein is directly sponsored by one or more USAF or DOD agencies. AFIT welcomes the opportunity to conduct research on additional topics of interest to the USAF, DOD, and other federal organizations when adequate manpower and financial resources are available and/or provided by a sponsor. In addition, AFIT provides research collaboration and technology transfer benefits to the public through Cooperative Research and Development Agreements (CRADAs). Interested individuals may discuss ideas for new research collaborations, potential CRADAs, or research proposals with individual faculty using the contact information in this document

ESTIMATION-BASED SOLUTIONS TO INCOMPLETE INFORMATION PURSUIT-EVASION GAMES

Differential games are a useful tool both for modeling conflict between autonomous systems and for synthesizing robust control solutions. The traditional study of games has assumed decision agents possess complete information about one another’s strategies and numerical weights. This dissertation relaxes this assumption. Instead, uncertainty in the opponent’s strategy is treated as a symptom of the inevitable gap between modeling assumptions and applications. By combining nonlinear estimation approaches with problem domain knowledge, procedures are developed for acting under uncertainty using established methods that are suitable for applications on embedded systems. The dissertation begins by using nonlinear estimation to account for parametric uncertainty in an opponent’s strategy. A solution is proposed for engagements in which both players use this approach simultaneously. This method is demonstrated on a numerical example of an orbital pursuit-evasion game, and the findings motivate additional developments. First, the solutions of the governing Riccati differential equations are approximated, using automatic differentiation to obtain high-degree Taylor series approximations. Second, constrained estimation is introduced to prevent estimator failures in near-singular engagements. Numerical conditions for nonsingularity are approximated using Chebyshev polynomial basis functions, and applied as constraints to a state estimate. Third and finally, multiple model estimation is suggested as a practical solution for time-critical engagements in which the form of the opponent’s strategy is uncertain. Deceptive opponent strategies are identified as a candidate approach to use against an adaptive player, and a procedure for designing such strategies is proposed. The new developments are demonstrated in a missile interception pursuit-evasion game in which the evader selects from a set of candidate strategies with unknown weights

Sensitivity Methods Applied to Orbital Pursuit-Evasion

In this work, sensitivity methods are examined as a means to solve and analyze the problem of orbital pursuit-evasion (PE). Orbital PE is a two-sided spacecraft trajectory optimization problem characterized by high dimensionality and nonlinearity. Modern methods for solving problems of this sort employ generic, computationally intensive techniques, including random search methods such as the genetic algorithm; collocation methods based on discrete approximation; or combinations of these methods. The advantages of these methods are relatively high degrees of robustness, straightforward implementation, and ease of handling state and control constraints. Yet we note the disadvantages: chiefly high computation load, as well as absence of insight into the problem, and accuracy of the result. Sensitivity methods provide corresponding strengths in each of these areas. We present novel sensitivity analysis techniques that may be useful in other optimization problems featuring high dimensionality, nonlinearity, and/or state and control constraints. The techniques shown include a novel solution method; a computationally efficient feedback control technique; a means of sketching barrier surfaces; and the use of hybrid one-sided/two-sided controllers for sophisticated emergent behavior. We also introduce a new formulation of the problem incorporating a minimum-altitude constraint, and we make an initial investigation of a sensitivity-based method of handling state constraints. Overall, our results suggest that sensitivity methods can provide useful augmentation to techniques that rely more heavily upon computational power, and may be particularly valuable for implementation in an onboard control algorithm