3 research outputs found
On Local Regret
Online learning aims to perform nearly as well as the best hypothesis in
hindsight. For some hypothesis classes, though, even finding the best
hypothesis offline is challenging. In such offline cases, local search
techniques are often employed and only local optimality guaranteed. For online
decision-making with such hypothesis classes, we introduce local regret, a
generalization of regret that aims to perform nearly as well as only nearby
hypotheses. We then present a general algorithm to minimize local regret with
arbitrary locality graphs. We also show how the graph structure can be
exploited to drastically speed learning. These algorithms are then demonstrated
on a diverse set of online problems: online disjunct learning, online Max-SAT,
and online decision tree learning.Comment: This is the longer version of the same-titled paper appearing in the
Proceedings of the Twenty-Ninth International Conference on Machine Learning
(ICML), 201
Efficient Regret Minimization in Non-Convex Games
We consider regret minimization in repeated games with non-convex loss
functions. Minimizing the standard notion of regret is computationally
intractable. Thus, we define a natural notion of regret which permits efficient
optimization and generalizes offline guarantees for convergence to an
approximate local optimum. We give gradient-based methods that achieve optimal
regret, which in turn guarantee convergence to equilibrium in this framework.Comment: Published as a conference paper at ICML 201
Online Convex Optimization for Sequential Decision Processes and Extensive-Form Games
Regret minimization is a powerful tool for solving large-scale extensive-form
games. State-of-the-art methods rely on minimizing regret locally at each
decision point. In this work we derive a new framework for regret minimization
on sequential decision problems and extensive-form games with general compact
convex sets at each decision point and general convex losses, as opposed to
prior work which has been for simplex decision points and linear losses. We
call our framework laminar regret decomposition. It generalizes the CFR
algorithm to this more general setting. Furthermore, our framework enables a
new proof of CFR even in the known setting, which is derived from a perspective
of decomposing polytope regret, thereby leading to an arguably simpler
interpretation of the algorithm. Our generalization to convex compact sets and
convex losses allows us to develop new algorithms for several problems:
regularized sequential decision making, regularized Nash equilibria in
extensive-form games, and computing approximate extensive-form perfect
equilibria. Our generalization also leads to the first regret-minimization
algorithm for computing reduced-normal-form quantal response equilibria based
on minimizing local regrets. Experiments show that our framework leads to
algorithms that scale at a rate comparable to the fastest variants of
counterfactual regret minimization for computing Nash equilibrium, and
therefore our approach leads to the first algorithm for computing quantal
response equilibria in extremely large games. Finally we show that our
framework enables a new kind of scalable opponent exploitation approach