Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature

This paper studies model-based bandit and reinforcement learning (RL) with nonlinear function approximations. We propose to study convergence to approximate local maxima because we show that global convergence is statistically intractable even for one-layer neural net bandit with a deterministic reward. For both nonlinear bandit and RL, the paper presents a model-based algorithm, Virtual Ascent with Online Model Learner (ViOL), which provably converges to a local maximum with sample complexity that only depends on the sequential Rademacher complexity of the model class. Our results imply novel global or local regret bounds on several concrete settings such as linear bandit with finite or sparse model class, and two-layer neural net bandit. A key algorithmic insight is that optimism may lead to over-exploration even for two-layer neural net model class. On the other hand, for convergence to local maxima, it suffices to maximize the virtual return if the model can also reasonably predict the size of the gradient and Hessian of the real return.Comment: Added an instantiation (Example 4.3) of the RL theorem (Theorem 4.4) and more reference

Maximal characterization of Hardy-Sobolev spaces on manifolds

International audienceLet $M$ be a complete non-compact Riemannian manifold with a doubling measure and admitting a Poincaré inequality. In the present paper, we identify the Sobolev space $\dot{M}^1_1$, introduced by Haj{\l}asz, with a new Hardy-Sobolev space defined by requiring the integrability of a certain maximal function of the gradient. This completes the circle of ideas begun in \cite{badrdafni}, where $\dot{M}^1_1$ was identified with a Hardy-Sobolev space via atomic decomposition

Stability of rank-3 Lazarsfeld-Mukai bundles on K3 surfaces

Given an ample line bundle L on a K3 surface S, we study the slope stability with respect to L of rank-3 Lazarsfeld-Mukai bundles associated with complete, base point free nets of type g^2_d on curves C in the linear system |L|. When d is large enough and C is general, we obtain a dimensional statement for the variety W^2_d(C). If the Brill-Noether number is negative, we prove that any g^2_d on any smooth, irreducible curve in |L| is contained in a g^r_e which is induced from a line bundle on S, thus answering a conjecture of Donagi and Morrison. Applications towards transversality of Brill-Noether loci and higher rank Brill-Noether theory are then discussed.Comment: 29 pages, final version, to appear in Proc. Lon. Math. So

Galois Correspondence and Fourier Analysis on Local Discrete Subfactors

Discrete subfactors include a particular class of infinite index subfactors and all finite index ones. A discrete subfactor is called local when it is braided and it fulfills a commutativity condition motivated by the study of inclusion of Quantum Field Theories in the algebraic Haag–Kastler setting. In Bischoff et al. (J Funct Anal 281(1):109004, 2021), we proved that every irreducible local discrete subfactor arises as the fixed point subfactor under the action of a canonical compact hypergroup. In this work, we prove a Galois correspondence between intermediate von Neumann algebras and closed subhypergroups, and we study the subfactor theoretical Fourier transform in this context. Along the way, we extend the main results concerning α-induction and σ-restriction for braided subfactors previously known in the finite index case

Spectral analysis of the transfer operator for the Lorentz gas

We study the billiard map associated with both the finite and infinite horizon Lorentz gases having smooth scatterers with strictly positive curvature. We introduce generalized function spaces (Banach spaces of distributions) on which the transfer operator is quasi-compact. The mixing properties of the billiard map then imply the existence of a spectral gap and related statistical properties such as exponential decay of correlations and the central limit theorem. Finer statistical properties of the map such as the identification of Ruelle resonances, large deviation estimates and an almost-sure invariance principle follow immediately once the spectral picture is established

Deep Sufficient Representation Learning via Mutual Information

We propose a mutual information-based sufficient representation learning (MSRL) approach, which uses the variational formulation of the mutual information and leverages the approximation power of deep neural networks. MSRL learns a sufficient representation with the maximum mutual information with the response and a user-selected distribution. It can easily handle multi-dimensional continuous or categorical response variables. MSRL is shown to be consistent in the sense that the conditional probability density function of the response variable given the learned representation converges to the conditional probability density function of the response variable given the predictor. Non-asymptotic error bounds for MSRL are also established under suitable conditions. To establish the error bounds, we derive a generalized Dudley's inequality for an order-two U-process indexed by deep neural networks, which may be of independent interest. We discuss how to determine the intrinsic dimension of the underlying data distribution. Moreover, we evaluate the performance of MSRL via extensive numerical experiments and real data analysis and demonstrate that MSRL outperforms some existing nonlinear sufficient dimension reduction methods.Comment: 43 pages, 6 figures and 5 table

Global Convergence and Stability of Stochastic Gradient Descent

In machine learning, stochastic gradient descent (SGD) is widely deployed to train models using highly non-convex objectives with equally complex noise models. Unfortunately, SGD theory often makes restrictive assumptions that fail to capture the non-convexity of real problems, and almost entirely ignore the complex noise models that exist in practice. In this work, we make substantial progress on this shortcoming. First, we establish that SGD's iterates will either globally converge to a stationary point or diverge under nearly arbitrary nonconvexity and noise models. Under a slightly more restrictive assumption on the joint behavior of the non-convexity and noise model that generalizes current assumptions in the literature, we show that the objective function cannot diverge, even if the iterates diverge. As a consequence of our results, SGD can be applied to a greater range of stochastic optimization problems with confidence about its global convergence behavior and stability