Movement and placement of non-contiguous data in distributed GPU computing

A steady increase in accelerator performance has driven demand for faster interconnects to avert the memory bandwidth wall. This has resulted in the wide adoption of heterogeneous systems with varying underlying interconnects, and has delegated the task of understanding and copying data to the system or application developer. Data transfer performance on these systems is now impacted by many factors including data transfer modality, system interconnects hardware details, CPU caching state, CPU power management state, driver policies, virtual memory paging efficiency, and data placement. This work finds that empirical communication measurements can be used to automatically schedule and execute intra- and inter-node communication in a modern heterogeneous system, providing ``hand-tuned'' performance without the need for complex or error-prone communication development at the application level. Empirical measurements are provided by a set of microbenchmarks designed for system and application developers to understand memory transfer behavior across different data placement and exchange scenarios. These benchmarks are the first comprehensive evaluation of all GPU communication primitives. For communication-heavy applications, optimally using communication capabilities is challenging and essential for performance. Two different approaches are examined. The first is a high-level 3D stencil communication library, which can automatically create a static communication plan based on the stencil and system parameters. This library is able to reduce the iteration time of a state-of-the-art stencil code by 1.45x at 3072 GPUs and 512 nodes. The second is a more general MPI interposer library, with novel non-contiguous data handling and runtime implementation selection for MPI communication primitives. A portable pure-MPI halo exchange is brought to within half the speed of the stencil-specific library, supported by a five order-of-magnitude improvement in MPI communication latency for non-contiguous data

Heterogeneous system and application communication modeling

With the end of Dennard scaling, high-performance computing increasingly relies on heterogeneous systems with specialized hardware to improve application performance. This trend has driven up the complexity of high-performance software development, as developers must manage multiple programming systems and develop system-tuned code to utilize specialized hardware. In addition, it has exacerbated existing challenges of data placement as the specialized hardware often has local memories to fuel its computational demands. In addition to using appropriate software resources to target application computation at the best hardware for the job, application developers now must manage data movement and placement within their application, which also must be specifically tuned to the target system. Instead of relying on the application developer to have specialized knowledge of system characteristics and specialized expertise in multiple programming systems, this work proposes a heterogeneous system communication library that automatically chooses data location and data movement for high-performance application development and execution on heterogeneous systems. This work presents the foundational components of that library: a systematic approach for characterization of system communication links and application communication demands

Heterogeneous system and application communication modeling

With the end of Dennard scaling, high-performance computing increasingly relies on heterogeneous systems with specialized hardware to improve application performance. This trend has driven up the complexity of high-performance software development, as developers must manage multiple programming systems and develop system-tuned code to utilize specialized hardware. In addition, it has exacerbated existing challenges of data placement as the specialized hardware often has local memories to fuel its computational demands. In addition to using appropriate software resources to target application computation at the best hardware for the job, application developers now must manage data movement and placement within their application, which also must be specifically tuned to the target system. Instead of relying on the application developer to have specialized knowledge of system characteristics and specialized expertise in multiple programming systems, this work proposes a heterogeneous system communication library that automatically chooses data location and data movement for high-performance application development and execution on heterogeneous systems. This work presents the foundational components of that library: a systematic approach for characterization of system communication links and application communication demands

Faith and democracy: political transformations at the German Protestant Kirchentag, 1949-1969

In the decades following World War II, German Protestants worked to transform their religious tradition. While this tradition had been previously characterized by rigidly hierarchical institutional structures, strong nationalist leanings, and authoritarian political tendencies, the experiences of dictatorship and defeat caused many Protestants to question their earlier beliefs. Motivated by the desire to overcome the burden of the Nazi past and by the opportunity to play a major role in postwar rebuilding efforts, several groups within the churches worked to reform Protestant social and political attitudes. As a result of their efforts, the churches came to play an important role in the ultimate success and stability of West German democracy. This study examines this transformation at the meetings of the German Protestant Kirchentag, one of the largest and most diverse postwar gatherings of Protestant laity. After situating the Kirchentag within the theological and political debates of the immediate postwar years, it focuses on changing understandings of the role of the church in society, the pluralization of Protestant political attitudes, and the shift from national to international self-understandings within the churches. It closes with the challenges posed to this new consensus by the youth revolt and the rise of New Left politics in the late 1960s. By examining the important role of the Kirchentag and of the Protestant churches in the democratization and political transformation of West Germany, this study asserts the continued relevance of religious categories of analysis in the Federal Republic. Focusing on the churches’ landmark contributions—including the promotion of democratic political activity, work toward East-West reconciliation, and the peace movement—it also argues for a broader, more complex conceptualization of postwar political transformations. In particular, rather than focusing on the work of any one faction or movement within the churches, it highlights the constructive roles played by different groups with different priorities and motivations