Neural Nets with a Newton Conjugate Gradient Method on Multiple GPUs

Training deep neural networks consumes increasing computational resource shares in many compute centers. Often, a brute force approach to obtain hyperparameter values is employed. Our goal is (1) to enhance this by enabling second-order optimization methods with fewer hyperparameters for large-scale neural networks and (2) to perform a survey of the performance optimizers for specific tasks to suggest users the best one for their problem. We introduce a novel second-order optimization method that requires the effect of the Hessian on a vector only and avoids the huge cost of explicitly setting up the Hessian for large-scale networks. We compare the proposed second-order method with two state-of-the-art optimizers on five representative neural network problems, including regression and very deep networks from computer vision or variational autoencoders. For the largest setup, we efficiently parallelized the optimizers with Horovod and applied it to a 8 GPU NVIDIA P100 (DGX-1) machine.Comment: Accepted to PPAM conferenc

Efficient Quantification of Model Uncertainties When De-boarding a Train

It is difficult to provide live simulation systems for decision support. Time is limited and uncertainty quantification requires many simulation runs. We combine a surrogate model with the stochastic collocation method to overcome time and storage restrictions and show a proof of concept for a de-boarding scenario of a train

Context-aware learning of hierarchies of low-fidelity models for multi-fidelity uncertainty quantification

Multi-fidelity Monte Carlo methods leverage low-fidelity and surrogate models for variance reduction to make tractable uncertainty quantification even when numerically simulating the physical systems of interest with high-fidelity models is computationally expensive. This work proposes a context-aware multi-fidelity Monte Carlo method that optimally balances the costs of training low-fidelity models with the costs of Monte Carlo sampling. It generalizes the previously developed context-aware bi-fidelity Monte Carlo method to hierarchies of multiple models and to more general types of low-fidelity models. When training low-fidelity models, the proposed approach takes into account the context in which the learned low-fidelity models will be used, namely for variance reduction in Monte Carlo estimation, which allows it to find optimal trade-offs between training and sampling to minimize upper bounds of the mean-squared errors of the estimators for given computational budgets. This is in stark contrast to traditional surrogate modeling and model reduction techniques that construct low-fidelity models with the primary goal of approximating well the high-fidelity model outputs and typically ignore the context in which the learned models will be used in upstream tasks. The proposed context-aware multi-fidelity Monte Carlo method applies to hierarchies of a wide range of types of low-fidelity models such as sparse-grid and deep-network models. Numerical experiments with the gyrokinetic simulation code \textsc{Gene} show speedups of up to two orders of magnitude compared to standard estimators when quantifying uncertainties in small-scale fluctuations in confined plasma in fusion reactors. This corresponds to a runtime reduction from 72 days to about four hours on one node of the Lonestar6 supercomputer at the Texas Advanced Computing Center.Comment: 25 pages, 12 figures, 3 table

High Mountain Asian glacier response to climate revealed by multi-temporal satellite observations since the 1960s

Funding: This study was supported by the Strategic Priority Research Program of Chinese Academy of Sciences (XDA20100300) and the Swiss National Science Foundation (200021E_177652/1). NN received funding from the European Union’s Horizon 2020 programme (No. 689443).Knowledge about the long-term response of High Mountain Asian glaciers to climatic variations is paramount because of their important role in sustaining Asian river flow. Here, a satellite-based time series of glacier mass balance for seven climatically different regions across High Mountain Asia since the 1960s shows that glacier mass loss rates have persistently increased at most sites. Regional glacier mass budgets ranged from −0.40 ± 0.07 m w.e.a−1 in Central and Northern Tien Shan to −0.06 ± 0.07 m w.e.a−1 in Eastern Pamir, with considerable temporal and spatial variability. Highest rates of mass loss occurred in Central Himalaya and Northern Tien Shan after 2015 and even in regions where glaciers were previously in balance with climate, such as Eastern Pamir, mass losses prevailed in recent years. An increase in summer temperature explains the long-term trend in mass loss and now appears to drive mass loss even in regions formerly sensitive to both temperature and precipitation.Publisher PDFPeer reviewe

Ice flow and the conditions of the ice-bed interface at the onset of the Northeast Greenland Ice Stream

The Northeast Greenland Ice Stream (NEGIS) is an important dynamic component contributing to the total mass balance of the Greenland ice sheet, as it reaches up to the central divide and drains 12% of the ice sheet. The ice stream geometry and surface velocities in the onset region of the NEGIS are not yet sufficiently well reproduced by ice sheet models. We present an assessment of the basal conditions of the onset region in a systematic analysis of airborne ultra-wideband radar data. Our data yield a new detailed model of ice-thickness distribution and basal topography in the upstream part of the ice stream. We observe a change from a smooth to a rougher bed where the ice stream widens from 10 to 60 km, and a distinct roughness anisotropy, indicating a preferred orientation of subglacial structures. The observation of off-nadir reflections that are symmetrical to the bed reflection in the radargrams suggests that these structures are elongated subglacial landforms, which in turn indicate potential streamlining of the bed. Together with basal water routing pathways, our observations hint to two different zones in this part of the NEGIS: an accelerating and smooth upstream region, which is collecting water, with reduced basal traction, and in the further downstream part, where the ice stream is slowing down and is widening, with a distribution of basal water towards the shear margins. Our findings support the hypothesis that the NEGIS is strongly interconnected to the subglacial water system in its onset region, but also to the subglacial substrate and morphology

3D-Structure ofNEGIS shearmarginsfromradarstratigraphy

The North East Greenland Ice Stream (NEGIS) is delineated by well-defined shear margins, which are evident in the gradient of surface velocity field as well as in the surface topography, where they form troughs up to ten meters deep. In the upper part of the ice stream the margins appear not to be linked to bedrock topography. To understand this efficient system of mass transport towards the ocean it is essential to investigate the nature of the shear margins, as here very localized deformation decouples the inner ice stream from the slower flowing surrounding ice sheet. This process is influenced by several factors and feedback mechanisms, including the crystal fabric orientation, strain heating and localization of meltwater. In summary, the shear margins are area-wise a small part of the ice stream itself, but the processes leading to the localization of deformation are of similar importance for ice discharge as the processes enabling fast flow of the main trunk over the bed. We present results from an airborne radar survey with the AWI Ultra-Wide Band Radar system, covering an area 150 km upstream and 100 km downstream of the deep drilling site on the ice stream (EGRIP). Over the survey area the ice stream accelerates from 12 m/a to 75 m/a. We focus on the signatures of the shear margins in the radar data. In the regions of localized shear, the internal reflections in the radargrams show disturbances in the form of steep undulations, or chevron folds, which are intensified with ongoing shear. As the ice stream has been covered with 36 flow-perpendicular radar sections we are able to show the evolution of these characteristic signatures over the survey area, and thus, as an analog, over time. 3D-representations of the folded stratigraphic layers reveal how new folds are formed when the ice stream widens and how older structures are preserved in the outer part of the main trunk, where they are no longer subject to shear. Furthermore, we link the change of the shape of the internal reflections in the shear zones to a strain rate field calculated from high resolution flow velocities derived by TerraSAR-X data

The new political economy of higher education: between distributional conflicts and discursive stratification

The higher education sector has been undergoing a far-reaching institutional re-orientation during the past two decades. Many adjustments appear to have strengthened the role of competition in the governance of higher education, but the character of the sector?s emerging new political economy has frequently remained unclear. Serving as the introduction for the special issue, this article makes the case for a multidimensional strategy to probe higher education?s competitive transformation. In terms of conceptualizing the major empirical shifts, we argue for analyzing three core phenomena: varieties of academic capitalism, the discursive construction of inequality, and the transformation of hierarchies in competitive settings. With respect to theoretical tools, we emphasize the complementary contributions of institutional, class-oriented, and discourse analytical approaches. As this introduction elaborates and the contributions to the special issue demonstrate, critical dialog among different analytical traditions over the interpretation of change is crucial for improving established understandings. Arguably, it is essential for clarifying the respective roles of capitalist power and hierarchical rule in the construction of the sector?s new order

Concepts for an Efficient Implementation of Domain Decomposition Approaches for Fluid-Structure Interactions

In the context of fluid-structure interactions, there are several natural starting points for domain decomposition. First, the decomposition into the fluid and the structure domain, second, the further decomposition of those domains into subdomains for parallelisation, and, third, the decomposition of the solver grid into finer and coarse