Exponential Convergence Bounds using Integral Quadratic Constraints

The theory of integral quadratic constraints (IQCs) allows verification of stability and gain-bound properties of systems containing nonlinear or uncertain elements. Gain bounds often imply exponential stability, but it can be challenging to compute useful numerical bounds on the exponential decay rate. In this work, we present a modification of the classical IQC results of Megretski and Rantzer that leads to a tractable computational procedure for finding exponential rate certificates

Query-Efficient Algorithms to Find the Unique Nash Equilibrium in a Two-Player Zero-Sum Matrix Game

We study the query complexity of identifying Nash equilibria in two-player zero-sum matrix games. Grigoriadis and Khachiyan (1995) showed that any deterministic algorithm needs to query $\Omega(n^2)$ entries in worst case from an $n\times n$ input matrix in order to compute an $\varepsilon$-approximate Nash equilibrium, where $\varepsilon<\frac{1}{2}$. Moreover, they designed a randomized algorithm that queries $\mathcal O(\frac{n\log n}{\varepsilon^2})$ entries from the input matrix in expectation and returns an $\varepsilon$-approximate Nash equilibrium when the entries of the matrix are bounded between $-1$ and $1$. However, these two results do not completely characterize the query complexity of finding an exact Nash equilibrium in two-player zero-sum matrix games. In this work, we characterize the query complexity of finding an exact Nash equilibrium for two-player zero-sum matrix games that have a unique Nash equilibrium $(x_\star,y_\star)$. We first show that any randomized algorithm needs to query $\Omega(nk)$ entries of the input matrix $A\in\mathbb{R}^{n\times n}$ in expectation in order to find the unique Nash equilibrium where $k=|\text{supp}(x_\star)|$. We complement this lower bound by presenting a simple randomized algorithm that, with probability $1-\delta$, returns the unique Nash equilibrium by querying at most $\mathcal O(nk^4\cdot \text{polylog}(\frac{n}{\delta}))$ entries of the input matrix $A\in\mathbb{R}^{n\times n}$. In the special case when the unique Nash Equilibrium is a pure-strategy Nash equilibrium (PSNE), we design a simple deterministic algorithm that finds the PSNE by querying at most $\mathcal O(n)$ entries of the input matrix.Comment: 17 page

Near-Optimal Pure Exploration in Matrix Games: A Generalization of Stochastic Bandits & Dueling Bandits

We study the sample complexity of identifying the pure strategy Nash equilibrium (PSNE) in a two-player zero-sum matrix game with noise. Formally, we are given a stochastic model where any learner can sample an entry $(i,j)$ of the input matrix $A\in[-1,1]^{n\times m}$ and observe $A_{i,j}+\eta$ where $\eta$ is a zero-mean 1-sub-Gaussian noise. The aim of the learner is to identify the PSNE of $A$, whenever it exists, with high probability while taking as few samples as possible. Zhou et al. (2017) presents an instance-dependent sample complexity lower bound that depends only on the entries in the row and column in which the PSNE lies. We design a near-optimal algorithm whose sample complexity matches the lower bound, up to log factors. The problem of identifying the PSNE also generalizes the problem of pure exploration in stochastic multi-armed bandits and dueling bandits, and our result matches the optimal bounds, up to log factors, in both the settings.Comment: 22 pages, 5 figure

Performance Guarantees in Learning and Robust Control

As the systems we control become more complex, first-principle modeling becomes either impossible or intractable, motivating the use of machine learning techniques for the control of systems with continuous action spaces. As impressive as the empirical success of these methods have been, strong theoretical guarantees of performance, safety, or robustness are few and far between. This manuscript takes a step towards such providing such guarantees by establishing finite-data performance guarantees for identifying and controlling fully- or partially-unknown dynamical systems.In this manuscript, we explore three different viewpoints that each provide different quantitative guarantees of performance. First, we present a generalization of the classical theory of integral quadratic constraints. This generalization leads to a tractable computational procedure for finding exponential stability certificates for partially-unknown feedback systems. Second, we present non-asymptotic lower and upper bounds for core problems in the field of system identification. Finally, using the recently developed system-level synthesis framework and tools from high-dimensional statistics, we establish finite-sample performance guarantees for robust output-feedback control of an unknown dynamical system

Performance Guarantees in Learning and Robust Control

As the systems we control become more complex, first-principle modeling becomes either impossible or intractable, motivating the use of machine learning techniques for the control of systems with continuous action spaces. As impressive as the empirical success of these methods have been, strong theoretical guarantees of performance, safety, or robustness are few and far between. This manuscript takes a step towards such providing such guarantees by establishing finite-data performance guarantees for identifying and controlling fully- or partially-unknown dynamical systems.In this manuscript, we explore three different viewpoints that each provide different quantitative guarantees of performance. First, we present a generalization of the classical theory of integral quadratic constraints. This generalization leads to a tractable computational procedure for finding exponential stability certificates for partially-unknown feedback systems. Second, we present non-asymptotic lower and upper bounds for core problems in the field of system identification. Finally, using the recently developed system-level synthesis framework and tools from high-dimensional statistics, we establish finite-sample performance guarantees for robust output-feedback control of an unknown dynamical system