Automatic Storage Optimization for Arrays

International audienceEfficient memory allocation is crucial for data-intensive applications as a smaller memory footprint ensures better cache performance and allows one to run a larger problem size given a fixed amount of main memory. In this paper, we describe a new automatic storage optimization technique to minimize the dimensionality and storage requirements of arrays used in sequences of loop nests with a predetermined schedule. We formulate the problem of intra-array storage optimization as one of finding the right storage partitioning hyperplanes: each storage partition corresponds to a single storage location. Our heuristic is driven by a dual objective function that minimizes both, the dimensionality of the mapping and the extents along those dimensions. The technique is dimension optimal for most codes encountered in practice. The storage requirements of the mappings obtained also are asymptotically better than those obtained by any existing schedule-dependent technique. Storage reduction factors and other results we report from an implementation of our technique demonstrate its effectiveness on several real-world examples drawn from the domains of image processing, stencil computations, high-performance computing, and the class of tiled codes in general

Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods

Automatic methods for model calibration seek to take advantage of the speed and power of digital computers, while being objective and relatively easy to implement. However, they do not provide parameter estimates and hydrograph simulations that are considered acceptable by the hydrologists responsible for operational forecasting and have therefore not entered into widespread use. In contrast, the manual approach which has been developed and refined over the years to result in excellent model calibrations is complicated and highly labor-intensive, and the expertise acquired by one individual with a specific model is not easily transferred to another person (or model). In this paper, we propose a hybrid approach that combines the strengths of each. A multicriteria formulation is used to "model" the evaluation techniques and strategies used in manual calibration, and the resulting optimization problem is solved by means of a computerized algorithm. The new approach provides a stronger test of model performance than methods that use a single overall statistic to aggregate model errors over a large range of hydrologic behaviors. The power of the new approach is illustrated by means of a case study using the Sacramento Soil Moisture Accounting model

Cheap Newton steps for optimal control problems: automatic differentiation and Pantoja's algorithm

Original article can be found at: http://www.informaworld.com/smpp/title~content=t713645924~db=all Copyright Taylor and Francis / Informa.In this paper we discuss Pantoja's construction of the Newton direction for discrete time optimal control problems. We show that automatic differentiation (AD) techniques can be used to calculate the Newton direction accurately, without requiring extensive re-writing of user code, and at a surprisingly low computational cost: for an N-step problem with p control variables and q state variables at each step, the worst case cost is 6(p + q + 1) times the computational cost of a single target function evaluation, independent of N, together with at most p3/3 + p2(q + 1) + 2p(q + 1)2 + (q + l)3, i.e. less than (p + q + l)3, floating point multiply-and-add operations per time step. These costs may be considerably reduced if there is significant structural sparsity in the problem dynamics. The systematic use of checkpointing roughly doubles the operation counts, but reduces the total space cost to the order of 4pN floating point stores. A naive approach to finding the Newton step would require the solution of an Np Np system of equations together with a number of function evaluations proportional to Np, so this approach to Pantoja's construction is extremely attractive, especially if q is very small relative to N. Straightforward modifications of the AD algorithms proposed here can be used to implement other discrete time optimal control solution techniques, such as differential dynamic programming (DDP), which use state-control feedback. The same techniques also can be used to determine with certainty, at the cost of a single Newton direction calculation, whether or not the Hessian of the target function is sufficiently positive definite at a point of interest. This allows computationally cheap post-hoc verification that a second-order minimum has been reached to a given accuracy, regardless of what method has been used to obtain it.Peer reviewe

Automatic Differentiation of Algorithms for Machine Learning

Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age. Reverse mode automatic differentiation both antedates and generalizes the method of backwards propagation of errors used in machine learning. Despite this, practitioners in a variety of fields, including machine learning, have been little influenced by automatic differentiation, and make scant use of available tools. Here we review the technique of automatic differentiation, describe its two main modes, and explain how it can benefit machine learning practitioners. To reach the widest possible audience our treatment assumes only elementary differential calculus, and does not assume any knowledge of linear algebra.Comment: 7 pages, 1 figur

Automatic tuning of Free Electron Lasers

Existing FEL facilities often suffer from stability issues: so electron orbit, transverse electron optics, electron bunch compression and other parameters have to be readjusted often to account for drifts in performance of various components. The tuning procedures typically employed in operation are often manual and lengthy. We have been developing a combination of model-free and model-based automatic tuning methods to meet the needs of present and upcoming XFEL facilities. Our approach has been implemented at FLASH \cite{flash} to achieve automatic SASE tuning using empirical control of orbit, electron optics and bunch compression. In this paper we describe our approach to empirical tuning, the software which implements it, and the results of using it at FLASH. We also discuss the potential of using machine learning and model-based techniques in tuning methods