Advances in Zero-Sum Dynamic Games

International audienceThe survey presents recent results in the theory of two-person zero-sum repeated games and their connections with differential and continuous-time games. The emphasis is made on the following(1) A general model allows to deal simultaneously with stochastic and informational aspects.(2) All evaluations of the stage payoffs can be covered in the same framework (and not only the usual Cesàro and Abel means).(3) The model in discrete time can be seen and analyzed as a discretization of a continuous time game. Moreover, tools and ideas from repeated games are very fruitful for continuous time games and vice versa.(4) Numerous important conjectures have been answered (some in the negative).(5) New tools and original models have been proposed. As a consequence, the field (discrete versus continuous time, stochastic versus incomplete information models) has a much more unified structure, and research is extremely active

Reachable sets analysis in the cooperative control of pursuer vehicles

This thesis is concerned with the Pursuit-and-Evasion (PE) problem where the pursuer aims to minimize the time to capture the evader while the evader tries to prevent capture. In the problem, the evader has two advantages: a higher manoeuvrability and that the pursuer is uncertain about the evader's state. Cooperation among multiple pursuer vehicles can thus be used to overcome the evader’s advantages. The focus here is on the formulation and development of frameworks and algorithms for cooperation amongst pursuers, aiming at feasible implementation on real and autonomous vehicles. The thesis is split into Parts I and II. Part I considers the problem of capturing an evader of higher manoeuvrability in a deterministic PE game. The approach is the employment of Forward Reachable Set (FRS) analysis in the pursuers’ control. The analysis considers the coverage of the evader’s FRS, which is the set of reachable states at a future time, with the pursuer’s FRS and assumes that the chance of capturing the evader is dependent on the degree of the coverage. Using the union of multiple pursuers’ FRSs intuitively leads to more evader FRS coverage and this forms the mechanism of cooperation. A framework for cooperative control based on the FRS coverage, or FRS-based control, is proposed. Two control algorithms were developed within this framework. Part II additionally introduces the problem of evader state uncertainty due to noise and limited field-of-view of the pursuers’ sensors. A search-and-capture (SAC) problem is the result and a hybrid architecture, which includes multi-sensor estimation using the Particle Filter as well as FRS-based control, is proposed to accomplish the SAC task. The two control algorithms in Part I were tested in simulations against an optimal guidance algorithm. The results show that both algorithms yield a better performance in terms of time and miss distance. The results in Part II demonstrate the effectiveness of the hybrid architecture for the SAC task. The proposed frameworks and algorithms provide insights for the development of effective and more efficient control of pursuer vehicles and can be useful in the practical applications such as defence systems and civil law enforcement

AFIT School of Engineering Contributions to Air Force Research and Technology. Calendar Year 1971

This report contains abstracts of Master of Science theses and Doctoral Dissertations completed during the 1971 calendar year at the School of Engineering, Air Force Institute of Technology

Bounded Rationality, Heterogeneity and Learning

The dissertation deals with both a model in which agents with limited information conform to a learning rule, as well as boundedly rational consumers who follow simple rules of thumb. The first case is treated within a generic, illustrative model situation, the so-called Shower Temperature Problem, in which the agents possess either the same (homogeneous) or an individual (heterogeneous) action space. The latter case is treated for consumer markets and additionally requires the modeling of an appropriate, strategic pricing of firms. The analysis of the Shower Temperature Problem shows that action heterogeneity represents a robust solution for the agents with only few systematic deviations, but at the cost of a higher risk for the individual agent than in the homogeneous case. Regarding the market of boundedly rational consumers, interesting results are obtained how psychological and experimental results can be cast into a mathematical model with boundedly rational, habit-forming and imitative consumers; this consumer model is analyzed and consequences are investigated. From the firms' point of view, conditions under which firms operate profitably in the long-term are examined. Furthermore, it is shown that the considered market is in a sense well-behaved, since for a rising number of firms, the prices decrease, the prices of the weakest products converge against marginal costs, and the welfare rises. Additionally, it is proven for a monopoly that Nash equilibria are found in the strategy space of all time-constant price paths. Finally, advertising is shown to be an effective method to sustain demand

Toward a Framework for Systematically Categorizing Future UAS Threat Space

Title from PDF of title page, viewed September 21, 2022Dissertation advisor: Travis FieldsVitaIncludes bibliographical references (pages 241-270)Dissertation (Ph.D.)--Department of Civil and Mechanical Engineering, Department of Electrical and Computer Engineering. University of Missouri--Kansas City, 2021The development of unmanned aerial vehicles (UAVs) is occurring as fast or faster than any other innovation throughout the course of human history. Building an effective means of defending against threats posed by malicious applications of novel technology is imperative in the current global landscape. Gone are the days where the enemy and the threat it poses are well defined and understood. Defensive technologies have to be modular and able to adapt to a threat technology space which is likely to recycle several times over during the course of a single defense system acquisition cycle. This manuscript wrestles with understanding the unique threat posed by UAVs and related technologies. A thorough taxonomy of the problem is given including projections for how the defining characteristics of the problem are likely to change and grow in the near future. Next, a discussion of the importance of tactics related to the problem of a rapidly changing threat space is provided. A discussion of case studies related to lessons learned from military acquisition programs and pivotal technological innovations in the course of history are given. Multiple measures of success are proposed which are designed to allow for meaningful comparisons and honest evaluations of capabilities. These measures are designed to facilitate discussions by providing a common, and comprehensible language that accounts for the vast complexity of the problem space without getting bogged down by the details. Lastly, predictions for the future threat space comprising UAVs is given. The contributions of this work are thus threefold. Firstly, an analytic framework is presented including a detailed parameterization of the problem as well as various solution techniques borrowed from a variety of fields. Secondly, measures of success are presented which attempt to compare the effectiveness of various systems by converting to expected values in terms of effective range, or extending the popular concept of kill chain and collapsing effectiveness into units of time. A novel technique for measuring effectiveness is presented whereby effectiveness is composed of various individual probabilities. Probabilities and associated distributions can be combined according to the rules of joint probabilities and distributions and allows performance against a probabilistic threat to be measured succinctly and effectively. The third contribution concerns predictions made with respect to the UAS threat space in the future. These predictions are designed to allow for defensive systems to be developed with a high expected effectiveness against current and future threats. Essentially this work comprises a first attempt toward developing a complete framework related to engagement and mission level modeling of a generic defensive system (or combination of systems) in the face of current and future threats presented by UAS.Introduction -- Literature review -- War gaming -- Measures of success -- Conclusion

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic