Lazy Multivariate Higher-Order Forward-Mode AD

A method is presented for computing all higher-order partial derivatives of a multivariate function Rn → R. This method works by evaluating the function under a nonstandard interpretation, lifting reals to multivariate power series. Multivariate power series, with potentially an infinite number of terms with nonzero coefficients, are represented using a lazy data structure constructed out of linear terms. A complete implementation of this method in SCHEME is presented, along with a straightforward exposition, based on Taylor expansions, of the method’s correctness

A Succinct Multivariate Lazy Multivariate Tower AD for Weil Algebra Computation (Computer Algebra - Theory and its Applications)

We propose a functional implementation of multivariate Tower Automatic Differentiation. Our implementation is intended to be used in implementing C^∞-structure computation of an arbitrary Weil algebra, which we discussed in [5]

Efficient Implementation of a Higher-Order Language with Built-In AD

We show that Automatic Differentiation (AD) operators can be provided in a dynamic language without sacrificing numeric performance. To achieve this, general forward and reverse AD functions are added to a simple high-level dynamic language, and support for them is included in an aggressive optimizing compiler. Novel technical mechanisms are discussed, which have the ability to migrate the AD transformations from run-time to compile-time. The resulting system, although only a research prototype, exhibits startlingly good performance. In fact, despite the potential inefficiencies entailed by support of a functional-programming language and a first-class AD operator, performance is competitive with the fastest available preprocessor-based Fortran AD systems. On benchmarks involving nested use of the AD operators, it can even dramatically exceed their performance