GekkoFS: A temporary distributed file system for HPC applications

We present GekkoFS, a temporary, highly-scalable burst buffer file system which has been specifically optimized for new access patterns of data-intensive High-Performance Computing (HPC) applications. The file system provides relaxed POSIX semantics, only offering features which are actually required by most (not all) applications. It is able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of general-purpose parallel file systems.The work has been funded by the German Research Foundation (DFG) through the ADA-FS project as part of the Priority Programme 1648. It is also supported by the Spanish Ministry of Science and Innovation (TIN2015–65316), the Generalitat de Catalunya (2014–SGR–1051), as well as the European Union’s Horizon 2020 Research and Innovation Programme (NEXTGenIO, 671951) and the European Comission’s BigStorage project (H2020-MSCA-ITN-2014-642963). This research was conducted using the supercomputer MOGON II and services offered by the Johannes Gutenberg University Mainz.Peer ReviewedPostprint (author's final draft

Distributed computing practice for large-scale science and engineering applications

It is generally accepted that the ability to develop large-scale distributed applications has lagged seriously behind other developments in cyberinfrastructure. In this paper, we provide insight into how such applications have been developed and an understanding of why developing applications for distributed infrastructure is hard. Our approach is unique in the sense that it is centered around half a dozen existing scientific applications; we posit that these scientific applications are representative of the characteristics, requirements, as well as the challenges of the bulk of current distributed applications on production cyberinfrastructure (such as the US TeraGrid). We provide a novel and comprehensive analysis of such distributed scientific applications. Specifically, we survey existing models and methods for large-scale distributed applications and identify commonalities, recurring structures, patterns and abstractions. We find that there are many ad hoc solutions employed to develop and execute distributed applications, which result in a lack of generality and the inability of distributed applications to be extensible and independent of infrastructure details. In our analysis, we introduce the notion of application vectors: a novel way of understanding the structure of distributed applications. Important contributions of this paper include identifying patterns that are derived from a wide range of real distributed applications, as well as an integrated approach to analyzing applications, programming systems and patterns, resulting in the ability to provide a critical assessment of the current practice of developing, deploying and executing distributed applications. Gaps and omissions in the state of the art are identified, and directions for future research are outlined

GekkoFS: A temporary burst buffer file system for HPC applications

Many scientific fields increasingly use high-performance computing (HPC) to process and analyze massive amounts of experimental data while storage systems in today’s HPC environments have to cope with new access patterns. These patterns include many metadata operations, small I/O requests, or randomized file I/O, while general-purpose parallel file systems have been optimized for sequential shared access to large files. Burst buffer file systems create a separate file system that applications can use to store temporary data. They aggregate node-local storage available within the compute nodes or use dedicated SSD clusters and offer a peak bandwidth higher than that of the backend parallel file system without interfering with it. However, burst buffer file systems typically offer many features that a scientific application, running in isolation for a limited amount of time, does not require. We present GekkoFS, a temporary, highly-scalable file system which has been specifically optimized for the aforementioned use cases. GekkoFS provides relaxed POSIX semantics which only offers features which are actually required by most (not all) applications. GekkoFS is, therefore, able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of common parallel file systems.Peer ReviewedPostprint (author's final draft

X-Composer: Enabling Cross-Environments In-SituWorkflows between HPC and Cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time

Morphometric, Behavioral, and Genomic Evidence for a New Orangutan Species

Six extant species of non-human great apes are currently recognized: Sumatran and Bornean orangutans, eastern and western gorillas, and chimpanzees and bonobos [1]. However, large gaps remain in our knowledge of fine-scale variation in hominoid morphology, behavior, and genetics, and aspects of great ape taxonomy remain in flux. This is particularly true for orangutans (genus: Pongo), the only Asian great apes and phylogenetically our most distant relatives among extant hominids [1]. Designation of Bornean and Sumatran orangutans, P. pygmaeus (Linnaeus 1760) and P. abelii (Lesson 1827), as distinct species occurred in 2001 [1, 2]. Here, we show that an isolated population from Batang Toru, at the southernmost range limit of extant Sumatran orangutans south of Lake Toba, is distinct from other northern Sumatran and Bornean populations. By comparing cranio-mandibular and dental characters of an orangutan killed in a human-animal conflict to those of 33 adult male orangutans of a similar developmental stage, we found consistent differences between the Batang Toru individual and other extant Ponginae. Our analyses of 37 orangutan genomes provided a second line of evidence. Model-based approaches revealed that the deepest split in the evolutionary history of extant orangutans occurred ∼3.38 mya between the Batang Toru population and those to the north of Lake Toba, whereas both currently recognized species separated much later, about 674 kya. Our combined analyses support a new classification of orangutans into three extant species. The new species, Pongo tapanuliensis, encompasses the Batang Toru population, of which fewer than 800 individuals survive

Doctor of Philosophy

dissertationThe increase in computational power of supercomputers is enabling complex scientific phenomena to be simulated at ever-increasing resolution and fidelity. With these simulations routinely producing large volumes of data, performing efficient I/O at this scale has become a very difficult task. Large-scale parallel writes are challenging due to the complex interdependencies between I/O middleware and hardware. Analytic-appropriate reads are traditionally hindered by bottlenecks in I/O access. Moreover, the two components of I/O, data generation from simulations (writes) and data exploration for analysis and visualization (reads), have substantially different data access requirements. Parallel writes, performed on supercomputers, often deploy aggregation strategies to permit large-sized contiguous access. Analysis and visualization tasks, usually performed on computationally modest resources, require fast access to localized subsets or multiresolution representations of the data. This dissertation tackles the problem of parallel I/O while bridging the gap between large-scale writes and analytics-appropriate reads. The focus of this work is to develop an end-to-end adaptive-resolution data movement framework that provides efficient I/O, while supporting the full spectrum of modern HPC hardware. This is achieved by developing technology for highly scalable and tunable parallel I/O, applicable to both traditional parallel data formats and multiresolution data formats, which are directly appropriate for analysis and visualization. To demonstrate the efficacy of the approach, a novel library (PIDX) is developed that is highly tunable and capable of adaptive-resolution parallel I/O to a multiresolution data format. Adaptive resolution storage and I/O, which allows subsets of a simulation to be accessed at varying spatial resolutions, can yield significant improvements to both the storage performance and I/O time. The library provides a set of parameters that controls the storage format and the nature of data aggregation across he network; further, a machine learning-based model is constructed that tunes these parameters for the maximum throughput. This work is empirically demonstrated by showing parallel I/O scaling up to 768K cores within a framework flexible enough to handle adaptive resolution I/O

XSEDE: eXtreme Science and Engineering Discovery Environment Third Quarter 2012 Report

The Extreme Science and Engineering Discovery Environment (XSEDE) is the most advanced, powerful, and robust collection of integrated digital resources and services in the world. It is an integrated cyberinfrastructure ecosystem with singular interfaces for allocations, support, and other key services that researchers can use to interactively share computing resources, data, and expertise.This a report of project activities and highlights from the third quarter of 2012.National Science Foundation, OCI-105357

Temporal Lossy In-Situ Compression for Computational Fluid Dynamics Simulations

Während CFD Simulationen für Metallschmelze im Rahmen des SFB920 fallen auf dem Taurus HPC Cluster in Dresden sehr große Datenmengen an, deren Handhabung den wissenschaftlichen Arbeitsablauf stark verlangsamen. Zum einen ist der Transfer in Visualisierungssysteme nur unter hohem Zeitaufwand möglich. Zum anderen ist interaktive Analyse von zeitlich abhängigen Prozessen auf Grund des Speicherflaschenhalses nahezu unmöglich. Aus diesen Gründen beschäftigt sich die vorliegende Dissertation mit der Entwicklung sog. Temporaler In-Situ Kompression für wissenschaftliche Daten direkt innerhalb von CFD Simulationen. Dabei werden mittels neuer Quantisierungsverfahren die Daten auf ~10% komprimiert, wobei dekomprimierte Daten einen Fehler von maximal 1% aufweisen. Im Gegensatz zu nicht-temporaler Kompression, wird bei temporaler Kompression der Unterschied zwischen Zeitschritten komprimiert, um den Kompressionsgrad zu erhöhen. Da die Datenmenge um ein Vielfaches kleiner ist, werden Kosten für die Speicherung und die Übertragung gesenkt. Da Kompression, Transfer und Dekompression bis zu 4 mal schneller ablaufen als der Transfer von unkomprimierten Daten, wird der wissenschaftliche Arbeitsablauf beschleunigt

Software for Exascale Computing - SPPEXA 2016-2019

This open access book summarizes the research done and results obtained in the second funding phase of the Priority Program 1648 "Software for Exascale Computing" (SPPEXA) of the German Research Foundation (DFG) presented at the SPPEXA Symposium in Dresden during October 21-23, 2019. In that respect, it both represents a continuation of Vol. 113 in Springer’s series Lecture Notes in Computational Science and Engineering, the corresponding report of SPPEXA’s first funding phase, and provides an overview of SPPEXA’s contributions towards exascale computing in today's sumpercomputer technology. The individual chapters address one or more of the research directions (1) computational algorithms, (2) system software, (3) application software, (4) data management and exploration, (5) programming, and (6) software tools. The book has an interdisciplinary appeal: scholars from computational sub-fields in computer science, mathematics, physics, or engineering will find it of particular interest

Conception et implémentation distribuée d'un outil de simulation d'écoulement de fluide basé sur les méthodes de Lattice Boltzmann

Lattice Boltzmann-based (LB) simulations are well suited to the simulation offluid flows in complex structures encountered in chemical engineering likeporous media or structured packing used in distillation and reactivedistillation columns. These simulations require large amounts ofmemory (around 10 gigabytes) and would require very long execution times(around 2 years) if executed on a single powerful desktop computer.The execution of LB simulations in a distributed way (for example, usingcluster computing) can decrease the execution time and reduces the memoryrequirements for each computer. Dynamic Heterogeneous Clusters (DHC) is a classof clusters involving computers inter-connected by a local area network; thesecomputers are potentially unreliable and do not share the same architecture,operating system, computational power, etc. However, DHCs are easy to setup and extend,and are made of affordable computers.The design and development of a software system which organizes large scaleDHCs in an efficient, scalable and robust way for implementing very large scaleLB simulations is challenging.In order to avoid that some computers are overloaded and slowdown the overall execution, the heterogeneity of computational power should betaken into account. In addition, the failure of one or several computers duringthe execution of a simulation should not prevent its completion.In the context of this thesis, a simulation tool called LaBoGrid wasdesigned. It uses existing static load balancing tools and implements anoriginal dynamic load balancing method in order to distribute the simulation ina way that minimizes its execution time. In addition, a distributed and scalablefault-tolerance mechanism based on the regular saving of simulation's stateis proposed. Finally, LaBoGrid is based on a distributed master-slavemodel that is robust and potentially scalable.Les simulations basées sur les méthodes de Lattice Boltzmann sont bienadaptées aux simulations d'écoulements de fluides à l'intérieur de structurescomplexes rencontrées en génie chimique, telles que les milieux poreux ou lesempilements structurés utilisés dans des colonnes de distillation et dedistillation réactive. Elles requièrent toutefois de grandes quantités demémoire (environ 10 gigaoctets). Par ailleurs, leur exécution sur un seulordinateur de bureau puissant nécessiterait un temps très long (environ deuxans).Il est possible de réduire à la fois le temps d'exécution et la quantité demémoire requise par ordinateur en exécutant les simulations LB de manièredistribuée, par exemple en utilisant un cluster. Un Cluster HétérogèneDynamique (CHD) est une classe de clusters impliquant des ordinateursqui sont interconnectés au moyen d'un réseau local, qui ne sont pasnécessairement fiables et qui ne partagent pas la même architecture, lemême système d'exploitation, la même puissance de calcul, etc. En revanche, lesCHD sont faciles à installer, à étendre et peu coûteux.Concevoir et développer un logiciel capable de gérer des CHD à grande échellede façon efficace, extensible et robuste et capable d'effectuer des simulationsLB à très grande échelle constitue un défi. L'hétérogénéité de la puissance decalcul doit être prise en compte afin d'éviter que certains ordinateurs soientdébordés et ralentissent le temps global d'exécution. En outre, une panne d'unou de plusieurs ordinateurs pendant l'exécution d'une simulation ne devrait pasempêcher son achèvement.Dans le contexte de cette thèse, un outil de simulation appelé LaBoGrid a étéconçu. LaBoGrid utilise des outils existants derépartition statique de la charge et implémente une méthode originale derépartition dynamique de la charge, ce qui lui permet de distribuer unesimulation LB de manière à minimiser son temps d'exécution. De plus, un mécanisme distribuéet extensible de tolérance aux pannes,fondé sur une sauvegarde régulière de l'état de simulation, est proposé. Enfin,LaBoGrid se base sur un modèle distribué de type « maître-esclaves » qui estrobuste et potentiellement extensible