Flows and Decompositions of Games: Harmonic and Potential Games

In this paper we introduce a novel flow representation for finite games in strategic form. This representation allows us to develop a canonical direct sum decomposition of an arbitrary game into three components, which we refer to as the potential, harmonic and nonstrategic components. We analyze natural classes of games that are induced by this decomposition, and in particular, focus on games with no harmonic component and games with no potential component. We show that the first class corresponds to the well-known potential games. We refer to the second class of games as harmonic games, and study the structural and equilibrium properties of this new class of games. Intuitively, the potential component of a game captures interactions that can equivalently be represented as a common interest game, while the harmonic part represents the conflicts between the interests of the players. We make this intuition precise, by studying the properties of these two classes, and show that indeed they have quite distinct and remarkable characteristics. For instance, while finite potential games always have pure Nash equilibria, harmonic games generically never do. Moreover, we show that the nonstrategic component does not affect the equilibria of a game, but plays a fundamental role in their efficiency properties, thus decoupling the location of equilibria and their payoff-related properties. Exploiting the properties of the decomposition framework, we obtain explicit expressions for the projections of games onto the subspaces of potential and harmonic games. This enables an extension of the properties of potential and harmonic games to "nearby" games. We exemplify this point by showing that the set of approximate equilibria of an arbitrary game can be characterized through the equilibria of its projection onto the set of potential games

Tackling bovine TB

On 18 December Defra revealed that during 2018, 32,601 badgers were killed, bringing the total number slaughtered under licence since 2013 to almost 67,000.1 ‘Effectiveness’ claims relate not to the impact on cattle TB, but rather to the ability of the contracted shooters to kill sufficient badgers to satisfy their licence requirements, which they can hardly fail to reach given that target numbers are ‘adjusted’ by Natural England part-way through the culls according to the contractors’ progress. Sixty per cent of the badgers have been killed by ‘controlled shooting’, a method rejected by both the government’s Independent Expert Panel2 and the BVA3 because of animal welfare concerns. During 2018 Natural England directly monitored just 89 (0.43 per cent) of controlled shooting events. It is deplorable that the chief veterinary officer (CVO) continues to support the roll-out of a policy that permits controlled shooting, when veterinary organisations have condemned the method on animal welfare grounds. It is particularly concerning that the CVO appears to have deflected responsibility for humaneness to Natural England’s chief scientist who, as far as we are aware, has no background in animal welfare science. It is also unacceptable for government to attribute reductions in herd bovine TB (bTB) incidents over the first four years of culling in the original ‘pilot’ cull zones to its badger culling policy.4 Independent analysis of this and more recent data from the Gloucestershire pilot cull zone5 indicate that new herd incidence is rising sharply, with 22 herd breakdowns in the 12 months to September 2017 (an increase of 29.4 per cent when compared to the 17 breakdowns reported by APHA for the previous 12 months). Analysis of the 2018 figures indicates that both incidence and prevalence are now rising even faster, with a further 24 herd breakdowns occurring between 1 January and 5 December 2018. To date, the government and its officials cite data that are two years out of date, but have declined to comment on this emerging evidence that, far from resulting in a substantial disease control benefit, badger culls may be leading to a sharp increase in bTB in cattle. Natural England’s chief scientist and the UK’s CVO continue to endorse a failing and inhumane policy, bringing their offices into serious disrepute. We urge them, and the BVA, to reconsider their support for further badger culling, and instead focus on the actual cause of bTB’s epidemic spread – a cattle skin test with a sensitivity of only 50 per cent,6,7 and the ongoing problems associated with cattle movements and on-farm biosecurity

Approximate policy iteration: A survey and some new methods

We consider the classical policy iteration method of dynamic programming (DP), where approximations and simulation are used to deal with the curse of dimensionality. We survey a number of issues: convergence and rate of convergence of approximate policy evaluation methods, singularity and susceptibility to simulation noise of policy evaluation, exploration issues, constrained and enhanced policy iteration, policy oscillation and chattering, and optimistic and distributed policy iteration. Our discussion of policy evaluation is couched in general terms and aims to unify the available methods in the light of recent research developments and to compare the two main policy evaluation approaches: projected equations and temporal differences (TD), and aggregation. In the context of these approaches, we survey two different types of simulation-based algorithms: matrix inversion methods, such as least-squares temporal difference (LSTD), and iterative methods, such as least-squares policy evaluation (LSPE) and TD (λ), and their scaled variants. We discuss a recent method, based on regression and regularization, which rectifies the unreliability of LSTD for nearly singular projected Bellman equations. An iterative version of this method belongs to the LSPE class of methods and provides the connecting link between LSTD and LSPE. Our discussion of policy improvement focuses on the role of policy oscillation and its effect on performance guarantees. We illustrate that policy evaluation when done by the projected equation/TD approach may lead to policy oscillation, but when done by aggregation it does not. This implies better error bounds and more regular performance for aggregation, at the expense of some loss of generality in cost function representation capability. Hard aggregation provides the connecting link between projected equation/TD-based and aggregation-based policy evaluation, and is characterized by favorable error bounds.National Science Foundation (U.S.) (No.ECCS-0801549)Los Alamos National Laboratory. Information Science and Technology InstituteUnited States. Air Force (No.FA9550-10-1-0412