Fast automatic differentiation Jacobians by compact LU factorization

For a vector function coded without branches or loops, a code for the Jacobian is generated by interpreting Griewank and Reese's vertex elimination as Gaussian elimination and implementing this as compact LU factorization. Tests on several platforms show such a code is typically 4 to 20 times faster than that produced by tools such as Adifor, Tamc, or Tapenade, on average significantly faster than vertex elimination code produced by the EliAD tool [Tadjouddine et al., in Proceedings of ICCS (2), Lecture Notes in Comput. Sci. 2330, Springer, New York, 2002] and can outperform a hand-coded Jacobian. The LU approach is promising, e.g., for CFD flux functions that are central to assembling Jacobians in finite element or finite volume calculations and, in general, for any inner-loop basic block whose Jacobian is crucial to an overall computation involving derivatives

Foundational Certification of Code Transformations Using Automatic Differentiation

Automatic Differentiation (AD) is concerned with the semantics augmentation of an input program representing a function to form a transformed program that computes the function's derivatives. To ensure the correctness  of the AD transformed code, particularly for safety critical applications, we aim at certifying the algebraic manipulations at the heart of the AD process. We have considered a WHILE-language and have shown how such proofs can be constructed by using an appropriate relational Hoare logic.In particular, we have shown how such inference rules can be constructed for both the forward and reverse mode AD by using an abductive logical reasoning

Jacobian Code Generated by Source Transformation and Vertex Elimination can be as Efficient as Hand-Coding

This article presents the first extended set of results from ELIAD, a source- transformation implementation of the vertex-elimination Automatic Differentiation approach to calculating the Jacobians of functions defined by Fortran code (Griewank and Reese, Automatic Differentiation of Algorithms: Theory, Implementation, and Application, 1991, pp. 126-135). We introduce the necessary theory in terms of well known algorithms of numerical linear algebra applied to the linear, extended Jacobian system that prescribes the relationship between the derivatives of all variables in the function code. Using an example, we highlight the potential for numerical instability in vertex-elimination. We describe the source transformation implementation of our tool ELIAD and present results from five test cases, four of which are taken from the MINPACK- 2 collection (Averick et al, Report ANL/MCS-TM-150, 1692) and for which hand- coded Jacobian codes are available. On five computer/compiler platforms, we show that the Jacobian code obtained by ELIAD is as efficient as hand-coded Jacobian code. It is also between 2 to 20 times more efficient than that produced by current, state of the art, Automatic Differentiation tools even when such tools make use of sophisticated techniques such as sparse Jacobian compression. We demonstrate the effectiveness of reverse-ordered pre-elimination from the (successively updated) extended Jacobian system of all intermediate variables used once. Thereafter, the monotonic forward/reverse ordered eliminations of all other intermediates is shown to be very efficient. On only one test case were orderings determined by the Markowitz or related VLR heuristics found superior. A re-ordering of the statements of the Jacobian code, with the aim of reducing reads and writes of data from cache to registers, was found to have mixed effects but could be very beneficial

Adjoint differentiation of a structural dynamics solver

The design of a satellite boom using passive vibration control by Keane [J. of Sound and Vibration, 1995, 185(3),441-453] has previously been carried out using an energy function of the design geometry aimed at minimising mechanical noise and vibrations. To minimise this cost function, a Genetic Algorithm (GA) was used, enabling modification of the initial geometry for a better design. To improve efficiency, it is proposed to couple the GA with a local search method involving the gradient of the cost function. In this paper, we detail the generation of an adjoint solver by automatic differentiation via ADIFOR. This has resulted in a gradient code that runs in 7.4 times the time of the function evaluation. This should reduce the rather time-consuming process (over 10 CPU days by using parallel processing) of the GA optimiser for this problem