Discrete Second Order Adjoints in Atmospheric Chemical Transport Modeling

Atmospheric chemical transport models (CTMs) are essential tools for the study of air pollution, for environmental policy decisions, for the interpretation of observational data, and for producing air quality forecasts. Many air quality studies require sensitivity analyses, i.e., the computation of derivatives of the model output with respect to model parameters. The derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through adjoint sensitivity analysis. While the traditional (first order) adjoint models give the gradient of the cost functional with respect to parameters, second order adjoint models give second derivative information in the form of products between the Hessian of the cost functional and a user defined vector. In this paper we discuss the mathematical foundations of the discrete second order adjoint sensitivity method and present a complete set of computational tools for performing second order sensitivity studies in three-dimensional atmospheric CTMs. The tools include discrete second order adjoints of Runge Kutta and of Rosenbrock time stepping methods for stiff equations together with efficient implementation strategies. Numerical examples illustrate the use of these computational tools in important applications like sensitivity analysis, optimization, uncertainty quantification, and the calculation of directions of maximal error growth in three-dimensional atmospheric CTMs

Automatic differentiation in machine learning: a survey

Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "autodiff", is a family of techniques similar to but more general than backpropagation for efficiently and accurately evaluating derivatives of numeric functions expressed as computer programs. AD is a small but established field with applications in areas including computational fluid dynamics, atmospheric sciences, and engineering design optimization. Until very recently, the fields of machine learning and AD have largely been unaware of each other and, in some cases, have independently discovered each other's results. Despite its relevance, general-purpose AD has been missing from the machine learning toolbox, a situation slowly changing with its ongoing adoption under the names "dynamic computational graphs" and "differentiable programming". We survey the intersection of AD and machine learning, cover applications where AD has direct relevance, and address the main implementation techniques. By precisely defining the main differentiation techniques and their interrelationships, we aim to bring clarity to the usage of the terms "autodiff", "automatic differentiation", and "symbolic differentiation" as these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure

Automatic Differentiation of Algorithms for Machine Learning

Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age. Reverse mode automatic differentiation both antedates and generalizes the method of backwards propagation of errors used in machine learning. Despite this, practitioners in a variety of fields, including machine learning, have been little influenced by automatic differentiation, and make scant use of available tools. Here we review the technique of automatic differentiation, describe its two main modes, and explain how it can benefit machine learning practitioners. To reach the widest possible audience our treatment assumes only elementary differential calculus, and does not assume any knowledge of linear algebra.Comment: 7 pages, 1 figur

A coupled discrete adjoint method for optimal design with dynamic non-linear fluid structure interactions

Incorporating high-fidelity analysis methods in multidisciplinary design optimization necessitates efficient sensitivity evaluation, which is particularly important for time-accurate problems. This thesis presents a new discrete adjoint formulation suitable for fully coupled, non-linear, dynamic FSI problems. The solution includes time-dependent adjoint variables that arise from grid motion and chosen time integration methods for both the fluid and structural domains. Implemented as a generic multizone discrete adjoint solver for time-accurate analysis in the open-source multiphysics solver SU2, this provides a flexible framework for a wide range of applications. Design optimization of aerodynamic structures need accurate characterization of the coupled fluid-structure interactions (FSI). Incorporating high-fidelity analysis methods in the multidisciplinary design optimization (MDO) necessitates efficient sensitivity evaluation, which is particularly important for time-accurate problems. Adjoint methods are well established for sensitivity analysis when large number of design variables are needed. The use of discrete adjoint method through algorithmic differentiation enables the evaluation of sensitivities using an approximation of the Jacobian of the coupled problem, thus enabling this approach to be applied for multidisciplinary analysis. This thesis presents a new discrete adjoint formulation suitable for fully coupled, non-linear, dynamic FSI problems. A partitioned approach is considered with finite volume for the fluid and finite elements for the solid domains. The solution includes the time-dependent adjoint variables that arise from the grid motion and chosen time integration methods for both the fluid and structural domains. Implemented as a generic multizone discrete adjoint solver for timeaccurate analysis in the open-source multiphysics solver SU2, this provides a flexible framework for a wide range of applications. The partitioned FSI solver approach has been leveraged to extend the dynamic FSI capabilities to low speed flows through the introduction of a densitybased unsteady incompressible flow solver. The developed methodology and implementation are demonstrated using a range of numerical test cases. Optimal design for steady, coupled FSI problems are firstly presented before moving to the building blocks of dynamic coupled problems using single domain analysis, for both structural and fluid domains in turn. The new unsteady incompressible fluid solver, for both the primal and adjoint analysis, are verified against a range of well-known benchmark test cases, including problems with grid motion. Finally, applications of coupled dynamic problems are presented to verify both the unsteady incompressible solver for FSI as well as the successful verification of the discrete adjoint sensitivities for the transient response of a transonic compliant airfoil for a variety of both aerodynamic and structural objective functions.Open Acces

DENSERKS: Fortran sensitivity solvers using continuous, explicit Runge-Kutta schemes

DENSERKS is a Fortran sensitivity equation solver package designed for integrating models whose evolution can be described by ordinary differential equations (ODEs). A salient feature of DENSERKS is its support for both forward and adjoint sensitivity analyses, with built-in integrators for both first and second order continuous adjoint models. The software implements explicit Runge-Kutta methods with adaptive timestepping and high-order dense output schemes for the forward and the tangent linear model trajectory interpolation. Implementations of six Runge-Kutta methods are provided, with orders of accuracy ranging from two to eight. This makes DENSERKS suitable for a wide range of practical applications. The use of dense output, a novel approach in adjoint sensitivity analysis solvers, allows for a high-order cost-effective interpolation. This is a necessary feature when solving adjoints of nonlinear systems using highly accurate Runge-Kutta methods (order five and above). To minimize memory requirements and make long-time integrations computationally efficient, DENSERKS implements a two-level checkpointing mechanism. The code is tested on a selection of problems illustrating first and second order sensitivity analysis with respect to initial model conditions. The resulting derivative information is also used in a gradient-based optimization algorithm to minimize cost functionals dependent on a given set of model parameters

Development and Applications of Adjoint-Based Aerodynamic and Aeroacoustic Multidisciplinary Optimization for Rotorcraft

Urban Air Mobility (UAM) is one of the most popular proposed solutions for alleviating traffic problems in populated areas. In this context, the proposed types of vehicles mainly consist of rotors and propellers powered by electric motors. However, those rotary-wing components can contribute excessively to noise generation. Therefore, a significant noise concern emerges due to urban air vehicles in or around residential areas. Reducing noise emitted by air vehicles is critically important to improve public acceptance of such vehicles for operations in densely populated areas. Two main objectives of the present dissertation are: (1) to expand the multidisciplinary optimization to utilize adjoint-based aeroacoustic and aerodynamic sensitivities; (2) to optimize the shape of proprotor blades to improve the overall performance of selected rotorcraft from both aerodynamic and aeroacoustic perspectives. This dissertation reports on the development and application of an unsteady discrete adjoint solver for aerodynamic and aeroacoustic coupling to obtain an improved design for quieter rotorcraft. The optimization framework developed through this dissertation can be utilized for multiple flight conditions, multiple receivers, and multiple optimization objectives within the same design process. SU2-based code development involves the implementation of aeroacoustic analysis, adjoint computations, and integrations into a multidisciplinary rotorcraft optimization suite. A computational aeroacoustics tool is embedded into the SU2-suite to predict the propagation of the emitted noise from the moving sources with high fidelity. Capabilities of the developed computational aeroacoustics tool are demonstrated for a range of rotor, propeller, and proprotor applications, and they are verified by comparing with wind tunnel data whenever it is available. The aeroacoustic tool also computes sensitivities with respect to the conserved variables and grid coordinates by employing the algorithmic differentiation method. Integration of an acoustic solver into the discrete adjoint solver and related modifications enable the code to compute aeroacoustic sensitivities with respect to the design variables. Applying the developed optimization framework for a proprotor aims to reduce the noise radiation without sacrificing the required aerodynamic performance value. As an outcome of the optimization during forward-flight and hover, the reshaped blade design emits and propagates lower noise levels as perceived by multiple observers. The major contributions are: (1) a multidisciplinary optimization framework that presents an optimized rotorcraft design for better aeroacoustics and aerodynamics; (2) a novel adjoint-based formulation for aeroacoustic sensitivities with respect to design variables; (3) single acoustic objective function including multiple flight conditions and multiple microphone positions; (4) implementation of Farassat 1A formulation into opensource software, SU2, to compute noise propagation emitted from moving sources. In summary, this dissertation provides the results with high fidelity, a well-integrated and rapidly converging optimization tool to improve the rotorcraft\u27s aeroacoustic performance while retaining or improving the aerodynamic performance. Among the conclusions are the following: (1) Computational fluid dynamics analyses (SU2-CFD) can produce accurate results for various rotorcraft applications. (2) The developed aeroacoustic code predicts noise propagation emitted from propellers, rotors, and proprotors with high-fidelity. (3) The acoustic interaction between propeller and wing components can be assessed by employing the aeroacoustic solver. (4) The multidisciplinary optimization framework successively reduces noise level emitted by a proprotor in multiple flight configurations. (5) The optimized design improves emitted noise radiation while satisfying the given aerodynamic constraint(s)