Cycles in adversarial regularized learning

Regularized learning is a fundamental technique in online optimization, machine learning and many other fields of computer science. A natural question that arises in these settings is how regularized learning algorithms behave when faced against each other. We study a natural formulation of this problem by coupling regularized learning dynamics in zero-sum games. We show that the system's behavior is Poincar\'e recurrent, implying that almost every trajectory revisits any (arbitrarily small) neighborhood of its starting point infinitely often. This cycling behavior is robust to the agents' choice of regularization mechanism (each agent could be using a different regularizer), to positive-affine transformations of the agents' utilities, and it also persists in the case of networked competition, i.e., for zero-sum polymatrix games.Comment: 22 pages, 4 figure

Adversarial scheduling analysis of Game-Theoretic Models of Norm Diffusion.

In (Istrate et al. SODA 2001) we advocated the investigation of robustness of results in the theory of learning in games under adversarial scheduling models. We provide evidence that such an analysis is feasible and can lead to nontrivial results by investigating, in an adversarial scheduling setting, Peyton Young's model of diffusion of norms . In particular, our main result incorporates contagion into Peyton Young's model.evolutionary games, stochastic stability, adversarial scheduling

A double-edged sword: Benefits and pitfalls of heterogeneous punishment in evolutionary inspection games

As a simple model for criminal behavior, the traditional two-strategy inspection game yields counterintuitive results that fail to describe empirical data. The latter shows that crime is often recurrent, and that crime rates do not respond linearly to mitigation attempts. A more apt model entails ordinary people who neither commit nor sanction crime as the third strategy besides the criminals and punishers. Since ordinary people free-ride on the sanctioning efforts of punishers, they may introduce cyclic dominance that enables the coexistence of all three competing strategies. In this setup ordinary individuals become the biggest impediment to crime abatement. We therefore also consider heterogeneous punisher strategies, which seek to reduce their investment into fighting crime in order to attain a more competitive payoff. We show that this diversity of punishment leads to an explosion of complexity in the system, where the benefits and pitfalls of criminal behavior are revealed in the most unexpected ways. Due to the raise and fall of different alliances no less than six consecutive phase transitions occur in dependence on solely the temptation to succumb to criminal behavior, leading the population from ordinary people-dominated across punisher-dominated to crime-dominated phases, yet always failing to abolish crime completely.Comment: 9 two-column pages, 5 figures; accepted for publication in Scientific Report

Machine Learning for Fluid Mechanics

The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from field measurements, experiments and large-scale simulations at multiple spatiotemporal scales. Machine learning offers a wealth of techniques to extract information from data that could be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. It outlines fundamental machine learning methodologies and discusses their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that considers data as an inherent part of modeling, experimentation, and simulation. Machine learning provides a powerful information processing framework that can enrich, and possibly even transform, current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202

Allocating Limited Resources to Protect a Massive Number of Targets using a Game Theoretic Model

Resource allocation is the process of optimizing the rare resources. In the area of security, how to allocate limited resources to protect a massive number of targets is especially challenging. This paper addresses this resource allocation issue by constructing a game theoretic model. A defender and an attacker are players and the interaction is formulated as a trade-off between protecting targets and consuming resources. The action cost which is a necessary role of consuming resource, is considered in the proposed model. Additionally, a bounded rational behavior model (Quantal Response, QR), which simulates a human attacker of the adversarial nature, is introduced to improve the proposed model. To validate the proposed model, we compare the different utility functions and resource allocation strategies. The comparison results suggest that the proposed resource allocation strategy performs better than others in the perspective of utility and resource effectiveness.Comment: 14 pages, 12 figures, 41 reference

Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

An important challenge for safety in machine learning and artificial intelligence systems is a~set of related failures involving specification gaming, reward hacking, fragility to distributional shifts, and Goodhart's or Campbell's law. This paper presents additional failure modes for interactions within multi-agent systems that are closely related. These multi-agent failure modes are more complex, more problematic, and less well understood than the single-agent case, and are also already occurring, largely unnoticed. After motivating the discussion with examples from poker-playing artificial intelligence (AI), the paper explains why these failure modes are in some senses unavoidable. Following this, the paper categorizes failure modes, provides definitions, and cites examples for each of the modes: accidental steering, coordination failures, adversarial misalignment, input spoofing and filtering, and goal co-option or direct hacking. The paper then discusses how extant literature on multi-agent AI fails to address these failure modes, and identifies work which may be useful for the mitigation of these failure modes.Comment: 12 Pages, This version re-submitted to Big Data and Cognitive Computing, Special Issue "Artificial Superintelligence: Coordination & Strategy