Integrating Grid Services into the Cray XT4 Environment

The 38640 core Cray XT4"Franklin" system at the National Energy Research Scientific Computing Center (NERSC) is a massively parallel resource available to Department of Energy researchers that also provides on-demand grid computing to the Open Science Grid. The integration of grid services on Franklin presented various challenges, including fundamental differences between the interactive and compute nodes, a stripped down compute-node operating system without dynamic library support, a shared-root environment and idiosyncratic application launching. Inour work, we describe how we resolved these challenges on a running, general-purpose production system to provide on-demand compute, storage, accounting and monitoring services through generic grid interfaces that mask the underlying system-specific details for the end user

The Repack Challenge

Physics data stored in CERN tapes is quickly reaching the 100 PB milestone. Tape is an ever-changing technology that is still following Moore's law in terms of capacity. This means we can store every year more and more data in the same amount of tapes. However this doesn't come for free: the first obvious cost is the new higher capacity media. The second less known cost is related to moving the data from the old tapes to the new ones. This activity is what we call repack. Repack is vital for any large tape user: without it, one would have to buy more tape libraries and more floor space and, eventually, data on old non supported tapes would become unreadable and be lost forever. In this paper we describe the challenge of repacking 115 PB before LHC data taking starts in the beginning of 2015. This process will have to run concurrently with the existing experiment tape activities, and therefore needs to be as transparent as possible for users. Making sure that this works out seamlessly implies careful planning of the resources and the various policies for sharing them fairly and conveniently. To tackle this problem we need to fully exploit the speed and throughput of our modern tape drives. This involves proper dimensioning and configuration of the disk arrays and all the links between them and the tape servers, i.e the machines responsible for managing the tape drives. It is also equally important to provide tools to improve the efficiency with which we use our tape libraries. The new repack setup we deployed has on average increased tape drive throughput by 80%, allowing them to perform closer to their design specifications. This improvement in turn means a 48% decrease in the number of drives needed to achieve the required throughput to complete the full repack on time

Investigating the relationship between volume transport and sea surface height in a numerical ocean model

The Agulhas Current Time-series Experiment mooring array (ACT) measured transport of the Agulhas Current at 34∘ S for a period of 3 years. Using along-track satellite altimetry data directly above the array, a proxy of Agulhas Current transport was developed based on the relationship between cross-current sea surface height (SSH) gradients and the measured transports. In this study, the robustness of the proxy is tested within a numerical modelling framework using a 34-year-long regional hindcast simulation from the Hybrid Coordinate Ocean Model (HYCOM). The model specifically tested the sensitivity of the transport proxy to (1) changes in the vertical structure of the current and to (2) different sampling periods used to calculate the proxy. Two reference proxies were created using HYCOM data from 2010 to 2013 by extracting model data at the mooring positions and along the satellite altimeter track for the box (net) transport and the jet (southwestward) transport. Sensitivity tests were performed where the proxy was recalculated from HYCOM for (1) a period where the modelled vertical stratification was different compared to the reference proxy and (2) different lengths of time periods: 1, 3, 6, 12, 18, and 34 years. Compared to the simulated (native) transports, it was found that the HYCOM proxy was more capable of estimating the box transport of the Agulhas Current compared to the jet transport. This was because the model is unable to resolve the dynamics associated with meander events, for which the jet transport algorithm was developed. The HYCOM configuration in this study contained exaggerated levels of offshore variability in the form of frequently impinging baroclinic anticyclonic eddies. These eddies consequently broke down the linear relationship between SSH slope and vertically integrated transport. Lastly, results showed that calculating the proxy over shorter or longer time periods in the model did not significantly impact the skill of the Agulhas transport proxy. Modelling studies of this kind provide useful information towards advancing our understanding of the sensitivities and limitations of transport proxies that are needed to improve long-term ocean monitoring approaches

CryoSat pre-launch calibration and validation studies of sea ice thickness

This report provides an overview of snow and ice measurements form several field experiments providing useful data for CryoSat pre-launch studies related to sea ice thickness. The most extensive experiments were performed during the Polarstern cruises in the Barents Sea / Storfjorden area in March 2003 and in the area north of Svalbard in April 2003. In addition, summer experiments were conducted during expeditions with the Lance in the Fram Strait in 2003 and 2004. The experiment used a number of different observing techniques from satellites, aircraft, helicopter and in situ observing systems. Several new and unique measurements techniques for sea ice properties were tested and used in combination with established methods. Comparison of methods used simultaneously over the same area is an important part of the work because more established methods can be used to validate new methods. For example, use of the helicopter EM system which has been developed over many years, was successfully used to validate the airborne laser-derived thickness estimates. In situ measurements obtained by drilling holes and using ground penetrating radar give detailed data on local scale. But spatial coverage of these data is very limited and it is challenge to obtain representative measurements for large parts of the Arctic sea ice.NERSC Technical Report no. 357.Funded by the European Space Agency through PRODEX Contract no. C9014

Resource-Efficient, Hierarchical Auto-Tuning of a Hybrid Lattice Boltzmann Computation on the Cray XT4

We apply auto-tuning to a hybrid MPI-pthreads lattice Boltzmann computation running on the Cray XT4 at National Energy Research Scientific Computing Center (NERSC). Previous work showed that multicore-specific auto-tuning can improve the performance of lattice Boltzmann magnetohydrodynamics (LBMHD) by a factor of 4x when running on dual- and quad-core Opteron dual-socket SMPs. We extend these studies to the distributed memory arena via a hybrid MPI/pthreads implementation. In addition to conventional auto-tuning at the local SMP node, we tune at the message-passing level to determine the optimal aspect ratio as well as the correct balance between MPI tasks and threads per MPI task. Our study presents a detailed performance analysis when moving along an isocurve of constant hardware usage: fixed total memory, total cores, and total nodes. Overall, our work points to approaches for improving intra- and inter-node efficiency on large-scale multicore systems for demanding scientific applications

Resource-Efficient, Hierarchical Auto-Tuning of a Hybrid Lattice Boltzmann Computation on the Cray XT4

We apply auto-tuning to a hybrid MPI-pthreads lattice Boltzmann computation running on the Cray XT4 at National Energy Research Scientific Computing Center (NERSC). Previous work showed that multicore-specific auto-tuning can improve the performance of lattice Boltzmann magnetohydrodynamics (LBMHD) by a factor of 4x when running on dual- and quad-core Opteron dual-socket SMPs. We extend these studies to the distributed memory arena via a hybrid MPI/pthreads implementation. In addition to conventional auto-tuning at the local SMP node, we tune at the message-passing level to determine the optimal aspect ratio as well as the correct balance between MPI tasks and threads per MPI task. Our study presents a detailed performance analysis when moving along an isocurve of constant hardware usage: fixed total memory, total cores, and total nodes. Overall, our work points to approaches for improving intra- and inter-node efficiency on large-scale multicore systems for demanding scientific applications