Convex Q Learning in a Stochastic Environment: Extended Version

The paper introduces the first formulation of convex Q-learning for Markov decision processes with function approximation. The algorithms and theory rest on a relaxation of a dual of Manne's celebrated linear programming characterization of optimal control. The main contributions firstly concern properties of the relaxation, described as a deterministic convex program: we identify conditions for a bounded solution, and a significant relationship between the solution to the new convex program, and the solution to standard Q-learning. The second set of contributions concern algorithm design and analysis: (i) A direct model-free method for approximating the convex program for Q-learning shares properties with its ideal. In particular, a bounded solution is ensured subject to a simple property of the basis functions; (ii) The proposed algorithms are convergent and new techniques are introduced to obtain the rate of convergence in a mean-square sense; (iii) The approach can be generalized to a range of performance criteria, and it is found that variance can be reduced by considering ``relative'' dynamic programming equations; (iv) The theory is illustrated with an application to a classical inventory control problem.Comment: Extended version of "Convex Q-learning in a stochastic environment", IEEE Conference on Decision and Control, 2023 (to appear

S-Duality and Exact Type IIB Superstring Backgrounds

A geometrical approach in the non-symmetric connection framework is employed to examine the issue of higher order $\alpha'$ corrections to D=10 type IIB superstring backgrounds with a covariantly constant null Killing isometry and non-zero Ramond-Ramond field content. These describe generalized supersymmetric string waves and were obtained recently by us through the S-duality transformation of purely NS-NS plane wave backgrounds. We find that the backgrounds are exact subject to the existence of certain field redefinitions and provided certain restrictive conditions are satisfied.Comment: 21 Pages, LATEX fil

Strong Coupling Theory for Interacting Lattice Models

We develop a strong coupling approach for a general lattice problem. We argue that this strong coupling perspective represents the natural framework for a generalization of the dynamical mean field theory (DMFT). The main result of this analysis is twofold: 1) It provides the tools for a unified treatment of any non-local contribution to the Hamiltonian. Within our scheme, non-local terms such as hopping terms, spin-spin interactions, or non-local Coulomb interactions are treated on equal footing. 2) By performing a detailed strong-coupling analysis of a generalized lattice problem, we establish the basis for possible clean and systematic extensions beyond DMFT. To this end, we study the problem using three different perspectives. First, we develop a generalized expansion around the atomic limit in terms of the coupling constants for the non-local contributions to the Hamiltonian. By analyzing the diagrammatics associated with this expansion, we establish the equations for a generalized dynamical mean-field theory (G-DMFT). Second, we formulate the theory in terms of a generalized strong coupling version of the Baym-Kadanoff functional. Third, following Pairault, Senechal, and Tremblay, we present our scheme in the language of a perturbation theory for canonical fermionic and bosonic fields and we establish the interpretation of various strong coupling quantities within a standard perturbative picture.Comment: Revised Version, 17 pages, 5 figure

Unified Einstein-Virasoro Master Equation in the General Non-Linear Sigma Model

The Virasoro master equation (VME) describes the general affine-Virasoro construction T=L^{ab}J_aJ_b+iD^a \dif J_a in the operator algebra of the WZW model, where $L^{ab}$ is the inverse inertia tensor and $D^a$ is the improvement vector. In this paper, we generalize this construction to find the general (one-loop) Virasoro construction in the operator algebra of the general non-linear sigma model. The result is a unified Einstein-Virasoro master equation which couples the spacetime spin-two field $L^{ab}$ to the background fields of the sigma model. For a particular solution $L_G^{ab}$, the unified system reduces to the canonical stress tensors and conventional Einstein equations of the sigma model, and the system reduces to the general affine-Virasoro construction and the VME when the sigma model is taken to be the WZW action. More generally, the unified system describes a space of conformal field theories which is presumably much larger than the sum of the general affine-Virasoro construction and the sigma model with its canonical stress tensors. We also discuss a number of algebraic and geometrical properties of the system, including its relation to an unsolved problem in the theory of $G$-structures on manifolds with torsion.Comment: LaTeX, 55 pages, one postscript figure, uses epsfig.sty. contains a few minor corrections; version to be published in Int. J. Mod. Phys.

Generalized conformal structure, dilaton gravity and SYK

A theory admits generalized conformal structure if the only scale in the quantum theory is set by a dimensionful coupling. SYK is an example of a theory with generalized conformal structure and in this paper we investigate the consequences of this structure for correlation functions and for the holographic realization of SYK. The Ward identities associated with the generalized conformal structure of SYK are implemented holographically in gravity/multiple scalar theories, which always have a parent AdS$_3$ origin. For questions involving only the graviton/running scalar sector, one can always describe the bulk running in terms of a single scalar but multiple running scalars are in general needed once one includes the bulk fields corresponding to all SYK operators. We then explore chaos in holographic theories with generalized conformal structure. The four point function explored by Maldacena, Shenker and Stanford exhibits exactly the same chaotic behaviour in any such theory as in holographic realizations of conformal theories i.e. the dimensionful coupling scale does not affect the chaotic exponential growth.Comment: 42 pages, 3 figure

Finite Temperature Models of Bose-Einstein Condensation

The theoretical description of trapped weakly-interacting Bose-Einstein condensates is characterized by a large number of seemingly very different approaches which have been developed over the course of time by researchers with very distinct backgrounds. Newcomers to this field, experimentalists and young researchers all face a considerable challenge in navigating through the `maze' of abundant theoretical models, and simple correspondences between existing approaches are not always very transparent. This Tutorial provides a generic introduction to such theories, in an attempt to single out common features and deficiencies of certain `classes of approaches' identified by their physical content, rather than their particular mathematical implementation. This Tutorial is structured in a manner accessible to a non-specialist with a good working knowledge of quantum mechanics. Although some familiarity with concepts of quantum field theory would be an advantage, key notions such as the occupation number representation of second quantization are nonetheless briefly reviewed. Following a general introduction, the complexity of models is gradually built up, starting from the basic zero-temperature formalism of the Gross-Pitaevskii equation. This structure enables readers to probe different levels of theoretical developments (mean-field, number-conserving and stochastic) according to their particular needs. In addition to its `training element', we hope that this Tutorial will prove useful to active researchers in this field, both in terms of the correspondences made between different theoretical models, and as a source of reference for existing and developing finite-temperature theoretical models.Comment: Detailed Review Article on finite temperature theoretical techniques for studying weakly-interacting atomic Bose-Einstein condensates written at an elementary level suitable for non-experts in this area (e.g. starting PhD students). Now includes table of content