The Extended Regularized Dual Averaging Method for Composite Optimization

We present a new algorithm, extended regularized dual averaging (XRDA), for solving composite optimization problems, which are a generalization of the regularized dual averaging (RDA) method. The main novelty of the method is that it allows more flexible control of the backward step size. For instance, the backward step size for RDA grows without bound, while XRDA the backward step size can be kept bounded

Echoes of chaos from string theory black holes

The strongly coupled D1-D5 conformal field theory is a microscopic model of black holes which is expected to have chaotic dynamics. Here, we study the weak coupling limit of the theory where it is integrable rather than chaotic. In this limit, the operators creating microstates of the lowest mass black hole are known exactly. We consider the time-ordered two-point function of light probes in these microstates, normalized by the same two-point function in vacuum. These correlators display a universal early-time decay followed by late-time sporadic behavior. To find a prescription for temporal coarse-graining of these late fluctuations we appeal to random matrix theory, where we show that a progressive time-average smooths the spectral form factor (a proxy for the 2-point function) in a typical draw of a random matrix. This coarse-grained quantity reproduces the matrix ensemble average to a good approximation. Employing this coarse-graining in the D1-D5 system, we find that the early-time decay is followed by a dip, a ramp and a plateau, in remarkable qualitative agreement with recent studies of the Sachdev-Ye-Kitaev (SYK) model. We study the timescales involved, comment on similarities and differences between our integrable model and the chaotic SYK model, and suggest ways to extend our results away from the integrable limit.Comment: 26 pages, 9 figures, v3: discussion of dip time adde

Online Convex Optimization for Sequential Decision Processes and Extensive-Form Games

Regret minimization is a powerful tool for solving large-scale extensive-form games. State-of-the-art methods rely on minimizing regret locally at each decision point. In this work we derive a new framework for regret minimization on sequential decision problems and extensive-form games with general compact convex sets at each decision point and general convex losses, as opposed to prior work which has been for simplex decision points and linear losses. We call our framework laminar regret decomposition. It generalizes the CFR algorithm to this more general setting. Furthermore, our framework enables a new proof of CFR even in the known setting, which is derived from a perspective of decomposing polytope regret, thereby leading to an arguably simpler interpretation of the algorithm. Our generalization to convex compact sets and convex losses allows us to develop new algorithms for several problems: regularized sequential decision making, regularized Nash equilibria in extensive-form games, and computing approximate extensive-form perfect equilibria. Our generalization also leads to the first regret-minimization algorithm for computing reduced-normal-form quantal response equilibria based on minimizing local regrets. Experiments show that our framework leads to algorithms that scale at a rate comparable to the fastest variants of counterfactual regret minimization for computing Nash equilibrium, and therefore our approach leads to the first algorithm for computing quantal response equilibria in extremely large games. Finally we show that our framework enables a new kind of scalable opponent exploitation approach