Analyzing Parallel Applications for Unnecessary I/O Semantics That Inhibit File System Performance

Scalability and performance of I/O intensive parallel applications are major concerns in modern High Performance Computing (HPC) environments. Almost all applications use POSIX I/O explicitly or implicitly through third party libraries like MPI-IO to perform I/O operations on the file system. POSIX I/O is known to be one of the lead causes of poor I/O performance due to its restrictive access semantics and consistency requirements. Some file systems therefore relax specific POSIX semantics to alleviate I/O performance penalties. In order to make the most effective use of the offered file systems features it is required to know what kind of POSIX semantics an application requires. Existing tools can analyze parallel I/O performance to report type and duration of executed I/O operations. There are even tools that analyse the consistency requirements of data operations, but none that also consider perfromance critical patterns of metadata operations. In this paper, we present a novel, systematic approach that groups parallel I/O operations and analyzes their I/O semantics with respect to POSIX I/O. We provide the tool rabbitxx that identifies concurrent overlapping accesses to the same file but also identifies metadata accesses such as concurrent create operations in the same directory. Our work indicates that POSIX defined I/O access semantics, in its current form, are often not necessary for parallel applications

OTF2: Open Trace Format Version 2 (v3.0.2)

The Open Trace Format Version 2 (OTF2) is a highly scalable, memory efficient event trace data format plus support library. It is the standard trace format for Scalasca, Vampir, and Tau and is open for other tools.OTF2 is available under the 3-clause BSD Open Source license.OTF2 is the common successor format for the Open Trace Format (OTF) and the Epilog trace format. It preserves the essential features as well as most record types of both and introduces new features such as support for multiple read/write substrates, in-place time stamp manipulation, and on-the-fly token translation. In particular, it will avoid copying during unification of parallel event streams

Score-P: Scalable performance measurement infrastructure for parallel codes (v8.0)

The Score-P measurement infrastructure is a highly scalable and easy-to-use tool suite for profiling, event tracing, and online analysis of HPC applications. Score-P offers the user a maximum of convenience by supporting a number of analysis tools. Currently, it works with CubeGUI, Scalasca trace tools, Vampir, Tau, and Extra-P and is open for other tools. Score-P comes together with the new Open Trace Format Version 2, the Cube4 profiling format and the Opari2 instrumenter. Score-P is available under the 3-clause BSD Open Source license

Score-P: Scalable performance measurement infrastructure for parallel codes (v8.3)

The instrumentation and measurement framework Score-P, together with analysis tools build on top of its output formats, provides insight into massively parallel HPC applications, their communication, synchronization, I/O, and scaling behaviour to pinpoint performance bottlenecks and their causes. Score-P is a highly scalable and easy-to-use tool suite for profiling (summarizing program execution) and event tracing (capturing events in chronological order) of HPC applications. The scorep instrumentation command adds instrumentation hooks into a user's application by either prepending or replacing the compile and link commands. C, C++, Fortran, and Python codes as well as contemporary HPC programming models (MPI, threading, GPUs, I/O) are supported. When running an instrumented application, measurement event data is provided by the instrumentation hooks to the measurement core. There, the events are augmented with high-accuracy timestamps and potentially hardware counters (a plugin-API allows querying additional metric sources). The augmented events are then passed to one or both of the built-in event consumers, profiling and tracing (a plugin-API allows creation of additional event consumers) which finally provide output in the formats CUBE4 and OTF2, respectively. Score-P is available under the 3-clause BSD Open Source license

Erratum to: Guidelines for the use and interpretation of assays for monitoring autophagy (3rd edition) (Autophagy, 12, 1, 1-222, 10.1080/15548627.2015.1100356

non present