Efficient Higher Order Derivatives of Objective Functions Composed of Matrix Operations

This paper is concerned with the efficient evaluation of higher-order derivatives of functions $f$ that are composed of matrix operations. I.e., we want to compute the $D$-th derivative tensor $\nabla^D f(X) \in \mathbb R^{N^D}$, where $f:\mathbb R^{N} \to \mathbb R$ is given as an algorithm that consists of many matrix operations. We propose a method that is a combination of two well-known techniques from Algorithmic Differentiation (AD): univariate Taylor propagation on scalars (UTPS) and first-order forward and reverse on matrices. The combination leads to a technique that we would like to call univariate Taylor propagation on matrices (UTPM). The method inherits many desirable properties: It is easy to implement, it is very efficient and it returns not only $\nabla^D f$ but yields in the process also the derivatives $\nabla^d f$ for $d \leq D$. As performance test we compute the gradient $\nabla f(X)$ % and the Hessian $\nabla_A^2 f(A)$ by a combination of forward and reverse mode of f(X) = \trace (X^{-1}) in the reverse mode of AD for $X \in \mathbb R^{n \times n}$. We observe a speedup of about 100 compared to UTPS. Due to the nature of the method, the memory footprint is also small and therefore can be used to differentiate functions that are not accessible by standard methods due to limited physical memory

On the efficient computation of high-order derivatives for implicitly defined functions

Scientific studies often require the precise calculation of derivatives. In many cases an analytical calculation is not feasible and one resorts to evaluating derivatives numerically. These are error-prone, especially for higher-order derivatives. A technique based on algorithmic differentiation is presented which allows for a precise calculation of higher-order derivatives. The method can be widely applied even for the case of only numerically solvable, implicit dependencies which totally hamper a semi-analytical calculation of the derivatives. As a demonstration the method is applied to a quantum field theoretical physical model. The results are compared with standard numerical derivative methods.Comment: 11 pages, 4 figures, to appear in Comput. Phys. Commu

Validated Computation of the Local Truncation Error of Runge-Kutta Methods with Automatic Differentiation

International audienceIn this paper, we propose a novel approach to bound the local truncation error based on the order condition which is usable for explicit and implicit Runge-Kutta methods

Total Generalized Variation for Manifold-valued Data

In this paper we introduce the notion of second-order total generalized variation (TGV) regularization for manifold-valued data in a discrete setting. We provide an axiomatic approach to formalize reasonable generalizations of TGV to the manifold setting and present two possible concrete instances that fulfill the proposed axioms. We provide well-posedness results and present algorithms for a numerical realization of these generalizations to the manifold setup. Further, we provide experimental results for synthetic and real data to further underpin the proposed generalization numerically and show its potential for applications with manifold-valued data

Automating embedded analysis capabilities and managing software complexity in multiphysics simulation part I: template-based generic programming

An approach for incorporating embedded simulation and analysis capabilities in complex simulation codes through template-based generic programming is presented. This approach relies on templating and operator overloading within the C++ language to transform a given calculation into one that can compute a variety of additional quantities that are necessary for many state-of-the-art simulation and analysis algorithms. An approach for incorporating these ideas into complex simulation codes through general graph-based assembly is also presented. These ideas have been implemented within a set of packages in the Trilinos framework and are demonstrated on a simple problem from chemical engineering