70 research outputs found
AD in Fortran, Part 1: Design
We propose extensions to Fortran which integrate forward and reverse
Automatic Differentiation (AD) directly into the programming model.
Irrespective of implementation technology, embedding AD constructs directly
into the language extends the reach and convenience of AD while allowing
abstraction of concepts of interest to scientific-computing practice, such as
root finding, optimization, and finding equilibria of continuous games.
Multiple different subprograms for these tasks can share common interfaces,
regardless of whether and how they use AD internally. A programmer can maximize
a function F by calling a library maximizer, XSTAR=ARGMAX(F,X0), which
internally constructs derivatives of F by AD, without having to learn how to
use any particular AD tool. We illustrate the utility of these extensions by
example: programs become much more concise and closer to traditional
mathematical notation. A companion paper describes how these extensions can be
implemented by a program that generates input to existing Fortran-based AD
tools
AD in Fortran, Part 2: Implementation via Prepreprocessor
We describe an implementation of the Farfel Fortran AD extensions. These
extensions integrate forward and reverse AD directly into the programming
model, with attendant benefits to flexibility, modularity, and ease of use. The
implementation we describe is a "prepreprocessor" that generates input to
existing Fortran-based AD tools. In essence, blocks of code which are targeted
for AD by Farfel constructs are put into subprograms which capture their
lexical variable context, and these are closure-converted into top-level
subprograms and specialized to eliminate EXTERNAL arguments, rendering them
amenable to existing AD preprocessors, which are then invoked, possibly
repeatedly if the AD is nested
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in
machine learning. Automatic differentiation (AD), also called algorithmic
differentiation or simply "autodiff", is a family of techniques similar to but
more general than backpropagation for efficiently and accurately evaluating
derivatives of numeric functions expressed as computer programs. AD is a small
but established field with applications in areas including computational fluid
dynamics, atmospheric sciences, and engineering design optimization. Until very
recently, the fields of machine learning and AD have largely been unaware of
each other and, in some cases, have independently discovered each other's
results. Despite its relevance, general-purpose AD has been missing from the
machine learning toolbox, a situation slowly changing with its ongoing adoption
under the names "dynamic computational graphs" and "differentiable
programming". We survey the intersection of AD and machine learning, cover
applications where AD has direct relevance, and address the main implementation
techniques. By precisely defining the main differentiation techniques and their
interrelationships, we aim to bring clarity to the usage of the terms
"autodiff", "automatic differentiation", and "symbolic differentiation" as
these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure
Algebraic Structures of Quantum Projective Field Theory Related to Fusion and Braiding. Hidden Additive Weight
The interaction of various algebraic structures describing fusion, braiding
and group symmetries in quantum projective field theory is an object of an
investigation in the paper. Structures of projective Zamolodchikov al- gebras,
their represntations, spherical correlation functions, correlation characters
and envelopping QPFT-operator algebras, projective \"W-algebras, shift
algebras, braiding admissible QPFT-operator algebras and projective
G-hypermultiplets are explored. It is proved (in the formalism of shift
algebras) that sl(2,C)-primary fields are characterized by their projective
weights and by the hidden additive weight, a hidden quantum number discovered
in the paper (some discussions on this fact and its possible relation to a
hidden 4-dimensional QFT maybe found in the note by S.Bychkov, S.Plotnikov and
D.Juriev, Uspekhi Matem. Nauk 47(3) (1992)[in Russian]). The special attention
is paid to various constructions of projective G-hyper- multiplets
(QPFT-operator algebras with G-symmetries).Comment: AMS-TEX, amsppt style, 16 pages, accepted for a publication in
J.MATH.PHYS. (Typographical errors are excluded
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD) is a technique for calculating derivatives of numeric functions expressed as computer programs efficiently and accurately, used in fields such as computational fluid dynamics, nuclear engineering, and atmospheric sciences. Despite its advantages and use in other fields, machine learning practitioners have been little influenced by AD and make scant use of available tools. We survey the intersection of AD and machine learning, cover applications where AD has the potential to make a big impact, and report on some recent developments in the adoption of this technique. We aim to dispel some misconceptions that we contend have impeded the use of AD within the machine learning community
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD) is a technique for calculating derivatives of numeric functions expressed as computer programs efficiently and accurately, used in fields such as computational fluid dynamics, nuclear engineering, and atmospheric sciences. Despite its advantages and use in other fields, machine learning practitioners have been little influenced by AD and make scant use of available tools. We survey the intersection of AD and machine learning, cover applications where AD has the potential to make a big impact, and report on some recent developments in the adoption of this technique. We aim to dispel some misconceptions that we contend have impeded the use of AD within the machine learning community
- …