CoRD: Converged RDMA Dataplane for High-Performance Clouds

High-performance networking is often characterized by kernel bypass which is considered mandatory in high-performance parallel and distributed applications. But kernel bypass comes at a price because it breaks the traditional OS architecture, requiring applications to use special APIs and limiting the OS control over existing network connections. We make the case, that kernel bypass is not mandatory. Rather, high-performance networking relies on multiple performance-improving techniques, with kernel bypass being the least effective. CoRD removes kernel bypass from RDMA networks, enabling efficient OS-level control over RDMA dataplane.Comment: 11 page

Workflow models for heterogeneous distributed systems

The role of data in modern scientific workflows becomes more and more crucial. The unprecedented amount of data available in the digital era, combined with the recent advancements in Machine Learning and High-Performance Computing (HPC), let computers surpass human performances in a wide range of fields, such as Computer Vision, Natural Language Processing and Bioinformatics. However, a solid data management strategy becomes crucial for key aspects like performance optimisation, privacy preservation and security. Most modern programming paradigms for Big Data analysis adhere to the principle of data locality: moving computation closer to the data to remove transfer-related overheads and risks. Still, there are scenarios in which it is worth, or even unavoidable, to transfer data between different steps of a complex workflow. The contribution of this dissertation is twofold. First, it defines a novel methodology for distributed modular applications, allowing topology-aware scheduling and data management while separating business logic, data dependencies, parallel patterns and execution environments. In addition, it introduces computational notebooks as a high-level and user-friendly interface to this new kind of workflow, aiming to flatten the learning curve and improve the adoption of such methodology. Each of these contributions is accompanied by a full-fledged, Open Source implementation, which has been used for evaluation purposes and allows the interested reader to experience the related methodology first-hand. The validity of the proposed approaches has been demonstrated on a total of five real scientific applications in the domains of Deep Learning, Bioinformatics and Molecular Dynamics Simulation, executing them on large-scale mixed cloud-High-Performance Computing (HPC) infrastructures

Assessment, Usability, and Sociocultural Impacts of DataONE

DataONE, funded from 2009-2019 by the U.S. National Science Foundation, is an early example of a large-scale project that built both a cyberinfrastructure and culture of data discovery, sharing, and reuse. DataONE used a Working Group model, where a diverse group of participants collaborated on targeted research and development activities to achieve broader project goals. This article summarizes the work carried out by two of DataONE’s working groups: Usability & Assessment (2009-2019) and Sociocultural Issues (2009-2014). The activities of these working groups provide a unique longitudinal look at how scientists, librarians, and other key stakeholders engaged in convergence research to identify and analyze practices around research data management through the development of boundary objects, an iterative assessment program, and reflection. Members of the working groups disseminated their findings widely in papers, presentations, and datasets, reaching international audiences through publications in 25 different journals and presentations to over 5,000 people at interdisciplinary venues. The working groups helped inform the DataONE cyberinfrastructure and influenced the evolving data management landscape. By studying working groups over time, the paper also presents lessons learned about the working group model for global large-scale projects that bring together participants from multiple disciplines and communities in convergence research

No Provisioned Concurrency: Fast RDMA-codesigned Remote Fork for Serverless Computing

Serverless platforms essentially face a tradeoff between container startup time and provisioned concurrency (i.e., cached instances), which is further exaggerated by the frequent need for remote container initialization. This paper presents MITOSIS, an operating system primitive that provides fast remote fork, which exploits a deep codesign of the OS kernel with RDMA. By leveraging the fast remote read capability of RDMA and partial state transfer across serverless containers, MITOSIS bridges the performance gap between local and remote container initialization. MITOSIS is the first to fork over 10,000 new containers from one instance across multiple machines within a second, while allowing the new containers to efficiently transfer the pre-materialized states of the forked one. We have implemented MITOSIS on Linux and integrated it with FN, a popular serverless platform. Under load spikes in real-world serverless workloads, MITOSIS reduces the function tail latency by 89% with orders of magnitude lower memory usage. For serverless workflow that requires state transfer, MITOSIS improves its execution time by 86%.Comment: To appear in OSDI'2

Survey of storage systems for high-performance computing

In current supercomputers, storage is typically provided by parallel distributed file systems for hot data and tape archives for cold data. These file systems are often compatible with local file systems due to their use of the POSIX interface and semantics, which eases development and debugging because applications can easily run both on workstations and supercomputers. There is a wide variety of file systems to choose from, each tuned for different use cases and implementing different optimizations. However, the overall application performance is often held back by I/O bottlenecks due to insufficient performance of file systems or I/O libraries for highly parallel workloads. Performance problems are dealt with using novel storage hardware technologies as well as alternative I/O semantics and interfaces. These approaches have to be integrated into the storage stack seamlessly to make them convenient to use. Upcoming storage systems abandon the traditional POSIX interface and semantics in favor of alternative concepts such as object and key-value storage; moreover, they heavily rely on technologies such as NVM and burst buffers to improve performance. Additional tiers of storage hardware will increase the importance of hierarchical storage management. Many of these changes will be disruptive and require application developers to rethink their approaches to data management and I/O. A thorough understanding of today's storage infrastructures, including their strengths and weaknesses, is crucially important for designing and implementing scalable storage systems suitable for demands of exascale computing

Annual Report, 2013-2014

Beginning in 2004/2005- issued in online format onl

Design Principles of Mobile Information Systems in the Digital Transformation of the Workplace - Utilization of Smartwatch-based Information Systems in the Corporate Context

During the last decades, smartwatches emerged as an innovative and promising technology and hit the consumer market due to the accessibility of affordable devices and predominant acceptance caused by the considerable similarity to common wristwatches. With the unique characteristics of permanent availability, unobtrusiveness, and hands-free operation, they can provide additional value in the corporate context. Thus, this thesis analyzes use cases for smartwatches in companies, elaborates on the design of smartwatch-based information systems, and covers the usability of smartwatch applications during the development of smartwatch-based information systems. It is composed of three research complexes. The first research complex focuses on the digital assistance of (mobile) employees who have to execute manual work and have been excluded so far from the benefits of the digitalization since they cannot operate hand-held devices. The objective is to design smartwatch-based information systems to support workflows in the corporate context, facilitate the daily work of numerous employees, and make processes more efficient for companies. During a design science research approach, smartwatch-based software artifacts are designed and evaluated in use cases of production, support, security service, as well as logistics, and a nascent design theory is proposed to complement theory according to mobile information system research. The evaluation shows that, on the one hand, smartwatches have enormous potential to assist employees with a fast and ubiquitous exchange of information, instant notifications, collaboration, and workflow guidance while they can be operated incidentally during manual work. On the other hand, the design of smartwatch-based information systems is a crucial factor for successful long-term deployment in companies, and especially limitations according to the small form-factor, general conditions, acceptance of the employees, and legal regulations have to be addressed appropriately. The second research complex addresses smartwatch-based information systems at the office workplace. This broadens and complements the view on the utilization of smartwatches in the corporate context in addition to the mobile context described in the first research complex. Though smartwatches are devices constructed for mobile use, the utilization in low mobile or stationary scenarios also has benefits due they exhibit the characteristic of a wearable computer and are directly connected to the employee’s body. Various sensors can perceive employee-, environment- and therefore context-related information and demand the employees’ attention with proactive notifications that are accompanied by a vibration. Thus, a smartwatch-based and gamified information system for health promotion at the office workplace is designed and evaluated. Research complex three provides a closer look at the topic of usability concerning applications running on smartwatches since it is a crucial factor during the development cycle. As a supporting element for the studies within the first and second research complex, a framework for the usability analysis of smartwatch applications is developed. For research, this thesis contributes a systemization of the state-of-the-art of smartwatch utilization in the corporate context, enabling and inhibiting influence factors of the smartwatch adoption in companies, and design principles as well as a nascent design theory for smartwatch-based information systems to support mobile employees executing manual work. For practice, this thesis contributes possible use cases for smartwatches in companies, assistance in decision-making for the introduction of smartwatch-based information systems in the corporate context with the Smartwatch Applicability Framework, situated implementations of a smartwatch-based information system for typical use cases, design recommendations for smartwatch-based information systems, an implementation of a smartwatch-based information system for the support of mobile employees executing manual work, and a usability-framework for smartwatches to automatically access usability of existing applications providing suggestions for usability improvement