603 research outputs found
Second order adjoints for solving PDE-constrained optimization problems
Inverse problems are of utmost importance in many fields of science and engineering. In the
variational approach inverse problems are formulated as PDE-constrained optimization problems,
where the optimal estimate of the uncertain parameters is the minimizer of a certain cost
functional subject to the constraints posed by the model equations. The numerical solution
of such optimization problems requires the computation of derivatives of the model output
with respect to model parameters. The first order derivatives of a cost functional (defined
on the model output) with respect to a large number of model parameters can be calculated
efficiently through first order adjoint sensitivity analysis. Second order adjoint models
give second derivative information in the form of matrix-vector products between the Hessian
of the cost functional and user defined vectors. Traditionally, the construction of second
order derivatives for large scale models has been considered too costly. Consequently, data
assimilation applications employ optimization algorithms that use only first order derivative
information, like nonlinear conjugate gradients and quasi-Newton methods.
In this paper we discuss the mathematical foundations of second order adjoint sensitivity
analysis and show that it provides an efficient approach to obtain Hessian-vector products. We
study the benefits of using of second order information in the numerical optimization process
for data assimilation applications. The numerical studies are performed in a twin experiment
setting with a two-dimensional shallow water model. Different scenarios are considered with
different discretization approaches, observation sets, and noise levels. Optimization algorithms
that employ second order derivatives are tested against widely used methods that require
only first order derivatives. Conclusions are drawn regarding the potential benefits and the
limitations of using high-order information in large scale data assimilation problems
FATODE: A Library for Forward, Adjoint, and Tangent Linear Integration of ODEs
FATODE is a FORTRAN library for the integration of ordinary differential equations with direct and adjoint sensitivity analysis capabilities.
The paper describes the capabilities, implementation, code organization, and usage of this package.
FATODE implements four families of methods -- explicit Runge-Kutta for nonstiff problems and fully implicit Runge-Kutta, singly diagonally implicit Runge-Kutta, and Rosenbrock for stiff problems.
Each family contains several methods with different orders of accuracy; users can add new methods by simply providing their coefficients.
For each family the forward, adjoint, and tangent linear models are implemented.
General purpose solvers for dense and sparse linear algebra are used; users can easily incorporate problem-tailored linear algebra routines.
The performance of the package is demonstrated on several test problems.
To the best of our knowledge FATODE is the first publicly available general purpose package that offers forward and adjoint sensitivity
analysis capabilities in the context of Runge Kutta methods. A wide range of applications are expected to benefit from its use; examples include parameter estimation,
data assimilation, optimal control, and uncertainty quantification
Different Approaches to Proof Systems
The classical approach to proof complexity perceives proof systems as deterministic, uniform, surjective, polynomial-time computable functions that map strings to (propositional) tautologies. This approach has been intensively studied since the late 70’s and a lot of progress has been made. During the last years research was started investigating alternative notions of proof systems. There are interesting results stemming from dropping the uniformity requirement, allowing oracle access, using quantum computations, or employing probabilism. These lead to different notions of proof systems for which we survey recent results in this paper
Recommended from our members
Computability Theory
Computability and computable enumerability are two of the fundamental notions of mathematics. Interest in effectiveness is already apparent in the famous Hilbert problems, in particular the second and tenth, and in early 20th century work of Dehn, initiating the study of word problems in group theory. The last decade has seen both completely new subareas develop as well as remarkable growth in two-way interactions between classical computability theory and areas of applications. There is also a great deal of work on algorithmic randomness, reverse mathematics, computable analysis, and in computable structure theory/computable model theory. The goal of this workshop is to bring together researchers representing different aspects of computability theory to discuss recent advances, and to stimulate future work
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in
machine learning. Automatic differentiation (AD), also called algorithmic
differentiation or simply "autodiff", is a family of techniques similar to but
more general than backpropagation for efficiently and accurately evaluating
derivatives of numeric functions expressed as computer programs. AD is a small
but established field with applications in areas including computational fluid
dynamics, atmospheric sciences, and engineering design optimization. Until very
recently, the fields of machine learning and AD have largely been unaware of
each other and, in some cases, have independently discovered each other's
results. Despite its relevance, general-purpose AD has been missing from the
machine learning toolbox, a situation slowly changing with its ongoing adoption
under the names "dynamic computational graphs" and "differentiable
programming". We survey the intersection of AD and machine learning, cover
applications where AD has direct relevance, and address the main implementation
techniques. By precisely defining the main differentiation techniques and their
interrelationships, we aim to bring clarity to the usage of the terms
"autodiff", "automatic differentiation", and "symbolic differentiation" as
these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure
- …