Advanced Concepts for Automatic Differentiation based on Operator Overloading

Mit Hilfe der Technik des Automatischen Differenzierens (AD) lassen sich für Funktionen, die als Programmquellcode gegeben sind, Ableitungsinformationen rechentechnisch effizient und mit geringem Aufwand für den Nutzer bereitstellen. Eine Variante der Implementierung von AD basiert auf der Überladung von Operatoren und Funktionen, die von vielen modernen Programmiersprachen ermöglicht wird. Durch Ausnutzung des Konzepts der Überladung wird eine interne Funktions-Repräsentation (Tape) generiert, die anschließend für die Ableitungsberechnung herangezogen wird. In der Dissertation werden neue Techniken erarbeitet, die eine effizientere Tape-Erstellung und die parallele Tape-Auswertung ermöglichen. Anhand von Laufzeituntersuchungen für numerische Beispiele werden die Möglichkeiten der neuen Techniken verdeutlicht.Using the technique of Automatic Differentiation (AD), derivative information can be computed efficiently for any function that is given as source code in a supported programming languages. One basic implementation strategy is based on the concept of operator overloading that is available for many programming languages. Due the overloading of operators, an internal representation of the function can be generated at runtime. This so-called tape can then be used for computing derivatives. In the thesis, new techniques are introduced that allow a more efficient tape creation and the parallel evaluation of tapes. Advantages of the new techniques are demonstrated by means of runtime analyses for numerical examples

Sensitivity computations by algorithmic differentiation of a high-­order cfd code based on spectral differences

We compute flow sensitivities by differentiating a high-­order computational fluid dynamics code. Our fully discrete approach relies on automatic differentiation (AD) of the original source code. We obtain two transformed codes by using the AD tool Tapenade (INRIA), one for each differentiation mode: tangent and adjoint. Both differentiated codes are tested against each other by computing sensitivities in an unsteady test case. The results from both codes agree to within machine accuracy, and compare well with those approximated by finite differences. We compare execution times and discuss the encountered technical difficulties due to 1) the code parallelism and 2) the memory overhead caused by unsteady problems

Adjoint computations by algorithmic differentiation of a parallel solver for time-dependent PDEs

A computational fluid dynamics code is differentiated using algorithmic differentiation (AD) in both tangent and adjoint modes. The two novelties of the present approach are 1) the adjoint code is obtained by letting the AD tool Tapenade invert the complete layer of message passing interface (MPI) communications, and 2) the adjoint code integrates time-dependent, non-linear and dissipative (hence physically irreversible) PDEs with an explicit time integration loop running for ca. $10^{6}$ time steps. The approach relies on using the Adjoinable MPI library to reverse the non-blocking communication patterns in the original code, and by controlling the memory overhead induced by the time-stepping loop with binomial checkpointing. A description of the necessary code modifications is provided along with the validation of the computed derivatives and a performance comparison of the tangent and adjoint codes.Comment: Submitted to Journal of Computational Scienc

Discrete Adjoints: Theoretical Analysis, Efficient Computation, and Applications

The technique of automatic differentiation provides directional derivatives and discrete adjoints with working accuracy. A complete complexity analysis of the basic modes of automatic differentiation is available. Therefore, the research activities are focused now on different aspects of the derivative calculation, as for example the efficient implementation by exploitation of structural information, studies of the theoretical properties of the provided derivatives in the context of optimization problems, and the development and analysis of new mathematical algorithms based on discrete adjoint information. According to this motivation, this habilitation presents an analysis of different checkpointing strategies to reduce the memory requirement of the discrete adjoint computation. Additionally, a new algorithm for computing sparse Hessian matrices is presented including a complexity analysis and a report on practical experiments. Hence, the first two contributions of this thesis are dedicated to an efficient computation of discrete adjoints. The analysis of discrete adjoints with respect to their theoretical properties is another important research topic. The third and fourth contribution of this thesis focus on the relation of discrete adjoint information and continuous adjoint information for optimal control problems. Here, differences resulting from different discretization strategies as well as convergence properties of the discrete adjoints are analyzed comprehensively. In the fifth contribution, checkpointing approaches that are successfully applied for the computation of discrete adjoints, are adapted such that they can be used also for the computation of continuous adjoints. Additionally, the fifth contributions presents a new proof of optimality for the binomial checkpointing that is based on new theoretical results. Discrete adjoint information can be applied for example for the approximation of dense Jacobian matrices. The development and analysis of new mathematical algorithms based on these approximate Jacobians is the topic of the sixth contribution. Is was possible to show global convergence to first-order critical points for a whole class of trust-region methods. Here, the usage of inexact Jacobian matrices allows a considerable reduction of the computational complexity

Unsteady adjoint computations by algorithmic differentiation of parallel code

International audienceA computational fluid dynamics code relying on a high-order spatial discretization is differentiated using algorithmic differentiation (AD). Two unsteady test cases are considered: a decaying incompressible viscous shear layer and an inviscid compressible flow around a NACA 0012 airfoil. Both tangent and adjoint modes of AD are explored in the viscous case, while only the tangent mode is applied to the inviscid case. The layer of message passing interface (MPI) communications was handled by the AD tool (Tapenade) through the Adjoinable MPI library, with fully automatic inversion of the MPI communications in adjoint mode. A description of the necessary code modifications is provided along with the validation of the computed derivatives and a comparison of the performance of the different codes. The explicit time integration loop of the viscous problem required of the order of 10^6 time steps, which could be inverted in the backward sweep of the adjoint code by means of binomial checkpointing

Towards Reduced-order Model Accelerated Optimization for Aerodynamic Design

The adoption of mathematically formal simulation-based optimization approaches within aerodynamic design depends upon a delicate balance of affordability and accessibility. Techniques are needed to accelerate the simulation-based optimization process, but they must remain approachable enough for the implementation time to not eliminate the cost savings or act as a barrier to adoption. This dissertation introduces a reduced-order model technique for accelerating fixed-point iterative solvers (e.g. such as those employed to solve primal equations, sensitivity equations, design equations, and their combination). The reduced-order model-based acceleration technique collects snapshots of early iteration (pre-convergent) solutions and residuals and then uses them to project to significantly more accurate solutions, i.e. smaller residual. The technique can be combined with other convergence schemes like multigrid and adaptive timestepping. The technique is generalizable and in this work is demonstrated to accelerate steady and unsteady flow solutions; continuous and discrete adjoint sensitivity solutions; and one-shot design optimization solutions. This final application, reduced-order model accelerated one-shot optimization approach, in particular represents a step towards more efficient aerodynamic design optimization. Through this series of applications, different basis vectors were considered and best practices for snapshot collection procedures were outlined. The major outcome of this dissertation is the development and demonstration of this reduced-order model acceleration technique. This work includes the first application of the reduced-order model-based acceleration method to an explicit one-shot iterative optimization process

The Tapenade Automatic Differentiation tool: principles, model, and specification

International audienceTapenade is an Automatic Differentiation tool which, given a Fortran or C code that computes a function, creates a new code that computes its tangent or adjoint derivatives. Tapenade puts particular emphasis on adjoint differentiation, which computes gradients at a remarkably low cost. This paper describes the principles of Tapenade, a subset of the general principles of AD. We motivate and illustrate on examples the AD model of Tapenade, i.e. the structure of differentiated codes and the strategies used to make them more efficient. Along with this informal description, we formally specify this model by means of Data-Flow equations and rules of Operational Semantics, making this the reference specification of the tangent and adjoint modes of Tapenade. One benefit we expect from this formal specification is the capacity to study formally the AD model itself, especially for the adjoint mode and its sophisticated strategies. This paper also describes the architectural choices of the implementation of Tapenade. We describe the current performances of Tapenade on a set of codes that include industrial-size applications. We present the extensions of the tool that are planned in a foreseeable future, deriving from our ongoing research on AD

The automation of PDE-constrained optimisation and its applications

This thesis is concerned with the automation of solving optimisation problems constrained by partial differential equations (PDEs). Gradient-based optimisation algorithms are the key to solve optimisation problems of practical interest. The required derivatives can be efficiently computed with the adjoint approach. However, current methods for the development of adjoint models often require a significant amount of effort and expertise, in particular for non-linear time-dependent problems. This work presents a new high-level reinterpretation of algorithmic differentiation to develop adjoint models. This reinterpretation considers the discrete system as a sequence of equation solves. Applying this approach to a general finite-element framework results in an automatic and robust way of deriving and solving adjoint models. This drastically reduces the development effort compared to traditional methods. Based on this result, a new framework for rapidly defining and solving optimisation problems constrained by PDEs is developed. The user specifies the discrete optimisation problem in a compact high-level language that resembles the mathematical structure of the underlying system. All remaining steps, including parameter updates, PDE solves and derivative computations, are performed without user intervention. The framework can be applied to a wide range of governing PDEs, and interfaces to various gradient-free and gradient-based optimisation algorithms. The capabilities of this framework are demonstrated through the application to two PDE-constrained optimisation problems. The first is concerned with the optimal layout of turbines in tidal stream farms; this optimisation problem is one of the main challenges facing the marine renewable energy industry. The second application applies data assimilation to reconstruct the profile of tsunami waves based on inundation observations. This provides the first step towards the general reconstruction of tsunami signals from satellite information