10,021 research outputs found
Automating embedded analysis capabilities and managing software complexity in multiphysics simulation part I: template-based generic programming
An approach for incorporating embedded simulation and analysis capabilities
in complex simulation codes through template-based generic programming is
presented. This approach relies on templating and operator overloading within
the C++ language to transform a given calculation into one that can compute a
variety of additional quantities that are necessary for many state-of-the-art
simulation and analysis algorithms. An approach for incorporating these ideas
into complex simulation codes through general graph-based assembly is also
presented. These ideas have been implemented within a set of packages in the
Trilinos framework and are demonstrated on a simple problem from chemical
engineering
Forward-Mode Automatic Differentiation in Julia
We present ForwardDiff, a Julia package for forward-mode automatic
differentiation (AD) featuring performance competitive with low-level languages
like C++. Unlike recently developed AD tools in other popular high-level
languages such as Python and MATLAB, ForwardDiff takes advantage of
just-in-time (JIT) compilation to transparently recompile AD-unaware user code,
enabling efficient support for higher-order differentiation and differentiation
using custom number types (including complex numbers). For gradient and
Jacobian calculations, ForwardDiff provides a variant of vector-forward mode
that avoids expensive heap allocation and makes better use of memory bandwidth
than traditional vector mode. In our numerical experiments, we demonstrate that
for nontrivially large dimensions, ForwardDiff's gradient computations can be
faster than a reverse-mode implementation from the Python-based autograd
package. We also illustrate how ForwardDiff is used effectively within JuMP, a
modeling language for optimization. According to our usage statistics, 41
unique repositories on GitHub depend on ForwardDiff, with users from diverse
fields such as astronomy, optimization, finite element analysis, and
statistics.
This document is an extended abstract that has been accepted for presentation
at the AD2016 7th International Conference on Algorithmic Differentiation.Comment: 4 page
TMB: Automatic Differentiation and Laplace Approximation
TMB is an open source R package that enables quick implementation of complex
nonlinear random effect (latent variable) models in a manner similar to the
established AD Model Builder package (ADMB, admb-project.org). In addition, it
offers easy access to parallel computations. The user defines the joint
likelihood for the data and the random effects as a C++ template function,
while all the other operations are done in R; e.g., reading in the data. The
package evaluates and maximizes the Laplace approximation of the marginal
likelihood where the random effects are automatically integrated out. This
approximation, and its derivatives, are obtained using automatic
differentiation (up to order three) of the joint likelihood. The computations
are designed to be fast for problems with many random effects (~10^6) and
parameters (~10^3). Computation times using ADMB and TMB are compared on a
suite of examples ranging from simple models to large spatial models where the
random effects are a Gaussian random field. Speedups ranging from 1.5 to about
100 are obtained with increasing gains for large problems. The package and
examples are available at http://tmb-project.org
Automatic implementation of material laws: Jacobian calculation in a finite element code with TAPENADE
In an effort to increase the versatility of finite element codes, we explore
the possibility of automatically creating the Jacobian matrix necessary for the
gradient-based solution of nonlinear systems of equations. Particularly, we aim
to assess the feasibility of employing the automatic differentiation tool
TAPENADE for this purpose on a large Fortran codebase that is the result of
many years of continuous development. As a starting point we will describe the
special structure of finite element codes and the implications that this code
design carries for an efficient calculation of the Jacobian matrix. We will
also propose a first approach towards improving the efficiency of such a
method. Finally, we will present a functioning method for the automatic
implementation of the Jacobian calculation in a finite element software, but
will also point out important shortcomings that will have to be addressed in
the future.Comment: 17 pages, 9 figure
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in
machine learning. Automatic differentiation (AD), also called algorithmic
differentiation or simply "autodiff", is a family of techniques similar to but
more general than backpropagation for efficiently and accurately evaluating
derivatives of numeric functions expressed as computer programs. AD is a small
but established field with applications in areas including computational fluid
dynamics, atmospheric sciences, and engineering design optimization. Until very
recently, the fields of machine learning and AD have largely been unaware of
each other and, in some cases, have independently discovered each other's
results. Despite its relevance, general-purpose AD has been missing from the
machine learning toolbox, a situation slowly changing with its ongoing adoption
under the names "dynamic computational graphs" and "differentiable
programming". We survey the intersection of AD and machine learning, cover
applications where AD has direct relevance, and address the main implementation
techniques. By precisely defining the main differentiation techniques and their
interrelationships, we aim to bring clarity to the usage of the terms
"autodiff", "automatic differentiation", and "symbolic differentiation" as
these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure
The chebop system for automatic solution of differential equations
In MATLAB, it would be good to be able to solve a linear differential equation by typing u = L\f, where f, u, and L are representations of the right-hand side, the solution, and the differential operator with boundary conditions. Similarly it would be good to be able to exponentiate an operator with expm(L) or determine eigenvalues and eigenfunctions with eigs(L). A system is described in which such calculations are indeed possible, based on the previously developed chebfun system in object-oriented MATLAB. The algorithms involved amount to spectral collocation methods on Chebyshev grids of automatically determined resolution
- …