Cheung and Piliouras (2020) recently showed that two variants of the
Multiplicative Weights Update method - OMWU and MWU - display opposite
convergence properties depending on whether the game is zero-sum or
cooperative. Inspired by this work and the recent literature on learning to
optimize for single functions, we introduce a new framework for learning
last-iterate convergence to Nash Equilibria in games, where the update rule's
coefficients (learning rates) along a trajectory are learnt by a reinforcement
learning policy that is conditioned on the nature of the game: \textit{the game
signature}. We construct the latter using a new decomposition of two-player
games into eight components corresponding to commutative projection operators,
generalizing and unifying recent game concepts studied in the literature. We
compare the performance of various update rules when their coefficients are
learnt, and show that the RL policy is able to exploit the game signature
across a wide range of game types. In doing so, we introduce CMWU, a new
algorithm that extends consensus optimization to the constrained case, has
local convergence guarantees for zero-sum bimatrix games, and show that it
enjoys competitive performance on both zero-sum games with constant
coefficients and across a spectrum of games when its coefficients are learnt