85 research outputs found
On the resolution of misspecification in stochastic optimization, variational inequality, and game-theoretic problems
Traditionally, much of the research in the field of optimization algorithms has assumed that problem parameters are correctly specified. Recent efforts under the robust optimization framework have relaxed this assumption by allowing unknown parameters to vary in a prescribed uncertainty set and by subsequently solving for a worst-case solution. This dissertation considers a rather different approach in which the unknown or misspecified parameter is a solution to a suitably defined (stochastic) learning problem based on
having access to a set of samples. Practical approaches in resolving such a set of coupled problems have been either sequential or direct variational approaches. In the case of the former, this entails the following steps: (i) a solution to the learning problem for parameters is first obtained; and (ii) a solution is obtained to the associated parametrized computational problem by using (i). Such avenues prove difficult to adopt particularly since the learning process has to be terminated finitely and consequently, in large-scale or stochastic instances, sequential approaches may often be corrupted by error. On the other hand, a variational approach requires that the problem may be recast as a possibly non-monotone stochastic variational inequality problem; but there are no known first-order (stochastic) schemes currently available for the solution of such
problems. Motivated by these challenges, this thesis focuses on studying joint schemes of optimization and learning in three settings: (i) misspecified stochastic optimization and variational inequality problems, (ii)
misspecified stochastic Nash games, (iii) misspecified Markov decision processes.
In the first part of this thesis, we present a coupled stochastic approximation scheme which simultaneously solves both the optimization and the learning problems. The obtained schemes are shown to be equipped with almost sure convergence properties in regimes when the function is either strongly convex as well as merely convex. Importantly, the scheme displays the optimal rate for strongly convex problems while in merely convex regimes, through an averaging approach, we quantify the degradation associated with learning
by noting that the error in function value after steps is , rather than when is available. Notably, when the averaging window is modified suitably, it can be see that the original rate
of is recovered. Additionally, we consider an online counterpart of the misspecified optimization problem and provide a non-asymptotic bound on the average regret with respect to an offline counterpart. We also extend these statements to a class of stochastic variational inequality problems, an object that unifies stochastic convex optimization problems and a range of stochastic equilibrium problems. Analogous almost-sure convergence statements are provided in strongly monotone and merely monotone regimes, the latter facilitated by using an iterative Tikhonov regularization. In the merely monotone regime, under a
weak-sharpness requirement, we quantify the degradation associated with learning and show that expected
error associated with is .
In the second part of this thesis, we present schemes for computing equilibria to two classes of convex
stochastic Nash games complicated by a parametric misspecification, a natural concern in the control of large-
scale networked engineered system. In both schemes, players learn the equilibrium strategy while resolving
the misspecification: (1) Stochastic Nash games: We present a set of coupled stochastic approximation
distributed schemes distributed across agents in which the first scheme updates each agent’s strategy via a projected (stochastic) gradient step while the second scheme updates every agent’s belief regarding its misspecified parameter using an independently specified learning problem. We proceed to show that the produced sequences converge to the true equilibrium strategy and the true parameter in an almost sure sense. Surprisingly, convergence in the equilibrium strategy achieves the optimal rate of convergence in a mean-squared sense with a quantifiable degradation in the rate constant; (2) Stochastic Nash-Cournot games with unobservable aggregate output: We refine (1) to a Cournot setting where we assume that the tuple of strategies is unobservable while payoff functions and strategy sets are public knowledge through a common knowledge assumption. By utilizing observations of noise-corrupted prices, iterative fixed-point schemes are developed, allowing for simultaneously learning the equilibrium strategies and the misspecified parameter in an almost-sure sense.
In the third part of this thesis, we consider the solution of a finite-state infinite horizon Markov Decision Process (MDP) in which both the transition matrix and the cost function are misspecified, the latter in a parametric sense. We consider a data-driven regime in which the learning problem is a stochastic
convex optimization problem that resolves misspecification. Via such a framework, we make the following
contributions: (1) We first show that a misspecified value iteration scheme converges almost surely to its
true counterpart and the mean-squared error after iterations is ; (2) An analogous asymptotic almost-sure convergence statement is provided for misspecified policy iteration; and (3) Finally, we present a constant steplength misspecified Q-learning scheme and show that a suitable error metric is + after K iterations where δ is a bound on the steplength
Sequential Sampling Equilibrium
This paper introduces an equilibrium framework based on sequential sampling
in which players face strategic uncertainty over their opponents' behavior and
acquire informative signals to resolve it. Sequential sampling equilibrium
delivers a disciplined model featuring an endogenous distribution of choices,
beliefs, and decision times, that not only rationalizes well-known deviations
from Nash equilibrium, but also makes novel predictions supported by existing
data. It grounds a relationship between empirical learning and strategic
sophistication, and generates stochastic choice through randomness inherent to
sampling, without relying on indifference or choice mistakes. Further, it
provides a rationale for Nash equilibrium when sampling costs vanish
Robust Model Misspecification and Paradigm Shifts
This paper studies the forms of model misspecification that are likely to
persist when compared with competing models. I consider an agent using a
subjective model to learn about an action-dependent outcome distribution. Aware
of potential model misspecification, she uses a threshold rule to switch
between models according to how well they fit the data. A model is globally
robust if it can persist against every finite set of competing models and is
locally robust if it can persist against every finite set of nearby competing
models. The main result provides simple characterizations of globally robust
and locally robust models based on the set of Berk-Nash equilibria they induce.
I then apply the results to examples including risk underestimation,
overconfidence, and incorrect beliefs about market demand
- …