AD in Fortran, Part 1: Design

We propose extensions to Fortran which integrate forward and reverse Automatic Differentiation (AD) directly into the programming model. Irrespective of implementation technology, embedding AD constructs directly into the language extends the reach and convenience of AD while allowing abstraction of concepts of interest to scientific-computing practice, such as root finding, optimization, and finding equilibria of continuous games. Multiple different subprograms for these tasks can share common interfaces, regardless of whether and how they use AD internally. A programmer can maximize a function F by calling a library maximizer, XSTAR=ARGMAX(F,X0), which internally constructs derivatives of F by AD, without having to learn how to use any particular AD tool. We illustrate the utility of these extensions by example: programs become much more concise and closer to traditional mathematical notation. A companion paper describes how these extensions can be implemented by a program that generates input to existing Fortran-based AD tools

AD in Fortran, Part 2: Implementation via Prepreprocessor

We describe an implementation of the Farfel Fortran AD extensions. These extensions integrate forward and reverse AD directly into the programming model, with attendant benefits to flexibility, modularity, and ease of use. The implementation we describe is a "prepreprocessor" that generates input to existing Fortran-based AD tools. In essence, blocks of code which are targeted for AD by Farfel constructs are put into subprograms which capture their lexical variable context, and these are closure-converted into top-level subprograms and specialized to eliminate EXTERNAL arguments, rendering them amenable to existing AD preprocessors, which are then invoked, possibly repeatedly if the AD is nested

Automatic differentiation in machine learning: a survey

Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "autodiff", is a family of techniques similar to but more general than backpropagation for efficiently and accurately evaluating derivatives of numeric functions expressed as computer programs. AD is a small but established field with applications in areas including computational fluid dynamics, atmospheric sciences, and engineering design optimization. Until very recently, the fields of machine learning and AD have largely been unaware of each other and, in some cases, have independently discovered each other's results. Despite its relevance, general-purpose AD has been missing from the machine learning toolbox, a situation slowly changing with its ongoing adoption under the names "dynamic computational graphs" and "differentiable programming". We survey the intersection of AD and machine learning, cover applications where AD has direct relevance, and address the main implementation techniques. By precisely defining the main differentiation techniques and their interrelationships, we aim to bring clarity to the usage of the terms "autodiff", "automatic differentiation", and "symbolic differentiation" as these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure

Algebraic Structures of Quantum Projective Field Theory Related to Fusion and Braiding. Hidden Additive Weight

The interaction of various algebraic structures describing fusion, braiding and group symmetries in quantum projective field theory is an object of an investigation in the paper. Structures of projective Zamolodchikov al- gebras, their represntations, spherical correlation functions, correlation characters and envelopping QPFT-operator algebras, projective \"W-algebras, shift algebras, braiding admissible QPFT-operator algebras and projective G-hypermultiplets are explored. It is proved (in the formalism of shift algebras) that sl(2,C)-primary fields are characterized by their projective weights and by the hidden additive weight, a hidden quantum number discovered in the paper (some discussions on this fact and its possible relation to a hidden 4-dimensional QFT maybe found in the note by S.Bychkov, S.Plotnikov and D.Juriev, Uspekhi Matem. Nauk 47(3) (1992)[in Russian]). The special attention is paid to various constructions of projective G-hyper- multiplets (QPFT-operator algebras with G-symmetries).Comment: AMS-TEX, amsppt style, 16 pages, accepted for a publication in J.MATH.PHYS. (Typographical errors are excluded

Wodzicki residue and anomalies of current algebras

The commutator anomalies (Schwinger terms) of current algebras in $3+1$ dimensions are computed in terms of the Wodzicki residue of pseudodifferential operators; the result can be written as a (twisted) Radul 2-cocycle for the Lie algebra of PSDO's. The construction of the (second quantized) current algebra is closely related to a geometric renormalization of the interaction Hamiltonian $H_I=j_{\mu} A^{\mu}$ in gauge theory.Comment: 15 pages, updated version of a talk at the Baltic School in Field Theory, September 199

W_{1+\infty} and W(gl_N) with central charge N

We study representations of the central extension of the Lie algebra of differential operators on the circle, the W-infinity algebra. We obtain complete and specialized character formulas for a large class of representations, which we call primitive; these include all quasi-finite irreducible unitary representations. We show that any primitive representation with central charge N has a canonical structure of an irreducible representation of the W-algebra W(gl_N) with the same central charge and that all irreducible representations of W(gl_N) with central charge N arise in this way. We also establish a duality between "integral" modules of W(gl_N) and finite-dimensional irreducible modules of gl_N, and conjecture their fusion rules.Comment: 29 pages, Latex, uses file amssym.def (a few remarks added, typos corrected

Dispersionful analogues of Benney's equations and NN-wave systems

We recall Krichever's construction of additional flows to Benney's hierarchy, attached to poles at finite distance of the Lax operator. Then we construct a ``dispersionful'' analogue of this hierarchy, in which the role of poles at finite distance is played by Miura fields. We connect this hierarchy with $N$-wave systems, and prove several facts about the latter (Lax representation, Chern-Simons-type Lagrangian, connection with Liouville equation, $\tau$-functions).Comment: 12 pages, latex, no figure

Schwinger Terms and Cohomology of Pseudodifferential Operators

We study the cohomology of the Schwinger term arising in second quantization of the class of observables belonging to the restricted general linear algebra. We prove that, for all pseudodifferential operators in 3+1 dimensions of this type, the Schwinger term is equivalent to the ``twisted'' Radul cocycle, a modified version of the Radul cocycle arising in non-commutative differential geometry. In the process we also show how the ordinary Radul cocycle for any pair of pseudodifferential operators in any dimension can be written as the phase space integral of the star commutator of their symbols projected to the appropriate asymptotic component.Comment: 19 pages, plain te