GPU peer-to-peer techniques applied to a cluster interconnect

Modern GPUs support special protocols to exchange data directly across the PCI Express bus. While these protocols could be used to reduce GPU data transmission times, basically by avoiding staging to host memory, they require specific hardware features which are not available on current generation network adapters. In this paper we describe the architectural modifications required to implement peer-to-peer access to NVIDIA Fermi- and Kepler-class GPUs on an FPGA-based cluster interconnect. Besides, the current software implementation, which integrates this feature by minimally extending the RDMA programming model, is discussed, as well as some issues raised while employing it in a higher level API like MPI. Finally, the current limits of the technique are studied by analyzing the performance improvements on low-level benchmarks and on two GPU-accelerated applications, showing when and how they seem to benefit from the GPU peer-to-peer method.Comment: paper accepted to CASS 201

Control System for the LEDA 6.7-MeV Proton Beam Halo Experiment

Measurement of high-power proton beam-halo formation is the ongoing scientific experiment for the Low Energy Demonstration Accelerator (LEDA) facility. To attain this measurement goal, a 52-magnet beam line containing several types of beam diagnostic instrumentation is being installed. The Experimental Physics and Industrial Control System (EPICS) and commercial software applications are presently being integrated to provide a real-time, synchronous data acquisition and control system. This system is comprised of magnet control, vacuum control, motor control, data acquisition, and data analysis. Unique requirements led to the development and integration of customized software and hardware. EPICS real-time databases, Interactive Data Language (IDL) programs, LabVIEW Virtual Instruments (VI), and State Notation Language (SNL) sequences are hosted on VXI, PC, and UNIX-based platforms which interact using the EPICS Channel Access (CA) communication protocol. Acquisition and control hardware technology ranges from DSP-based diagnostic instrumentation to the PLC-controlled vacuum system. This paper describes the control system hardware and software design, and implementation.Comment: LINAC2000 Conference, 4 pg