de Cuveland, J.

Hutter, D.

Lindenstruth, V.

GSI Repository

CBM First-level Event Selector Data Management Developments∗J. de Cuveland1, D. Hutter1, and V. Lindenstruth11FIAS Frankfurt Institute for Advanced Studies, Goethe-Universita¨t Frankfurt am Main, GermanyThe First-level Event Selector (FLES) is the centralevent selection system in the CBM experiment. Its task is toselect data for storage based on online analyses including acomplete event reconstruction. To do so, the FLES times-lice building has to combine data from all input links totime intervals and distribute them via a high-performancenetwork to the compute nodes. Data rates at this point areexpected to exceed 1 TByte/s.The FLES system consists of a scalable supercomputerwith custom FPGA-based input interface cards and a fastevent-building network (Fig. 1). Constructed largely fromstandard components, it will be situated at the new FAIRdata center.Timeslice BuildingA timeslice is the fundamental data structure that man-ages access to all detector raw data of a given time inter-val. It provides random access to microslices, each con-taining data from a small amount of time and a single inputdetector link (Fig. 2). By duplicating a small number ofmicroslices at the border of a timeslice (overlap region),differences in detector readout timing performance can beaccomodated, and it is ensured that every timeslice can beprocessed independently.These data structures and a timeslice-based API for thefollowing online data processing have been developed andoptimized such that computer memory access (e. g., re-quired by additional copy operations) can be minimized.For data transfer between the FLES nodes, it has provenpractical to use an InfiniBand network. A prototype soft-ware for efficient timeslice building based on InfiniBandVerbs has been developed, especially addressing questionsregarding data structures and buffer management. It imple-ments the full data structures as intended for the final setup.The feasibility of employing a more high-level interface tothe network hardware, like MPI, is currently under investi-gation.Efficient Timeslice building over a state-of-the-art Infini-Band FDR network has been successfully demonstrated ona small scale on the Micro-FLES cluster installed at GSI.FLES Interface Board (FLIB)The CBM detector data enters the FLES system throughcustom add-on cards in the FLES PCs. These FLES In-terface Boards (FLIBs) require high-speed optical inputs toreceive the data from the CBM readout electronics, a high-performance interface to the host PC, and a large buffermemory. A specially developed FPGA-based card with a∗Work supported by BMBF (05P12RFFCP) and HIC for FAIRFEBROC  HUB / OPTOFLIBMessage StreamsMicroslice  ContainersDataControlTimingDetector Subsystem FLESDAQPC Input NodePC Compute NodeConf iguration/ControlDPBFigure 1: The First-level event selector (FLES) in the CBMread-out data path.TimesliceDESC MC 0DESC MC 1CONTENT MC 0CONTENT MC 1. . .Component 0DESC MC 99CONTENT MC 99DESC MC 100CONTENT MC 100DESC MC 0DESC MC 1CONTENT MC 0CONTENT MC 1. . .Component 1DESC MC 99CONTENT MC 99DESC MC 100CONTENT MC 100. . .DESC MC 101CONTENT MC 101DESC MC 102CONTENT MC 102Core region Overlap regionTimeFigure 2: The FLES timeslice data structure provides in-dexed access to detector raw data packaged in microslices.PCIe interface at the maximum achievable rate will be em-ployed here.The FPGA design implements the DAQ protocol for re-ceiving the data, manages the buffer memory, and controlsthe PCIe transfer. In contrast to the final system, early pro-totype setups lack the DPB layer (cf. Fig. 1) and do notsupport the creation of microslices. The current prototypefirmware therefore includes a mockup of the foreseen DPBdesign and is capable of directly receiving CBM-Net mes-sages as delivered by the CBM front-end electronics.The FPGA design has been ported to the Xilinx Kintex-7 family of FPGAs, as the current standard readout chainfor CBM detector development employs a prototype boardwith this device. The prototype firmware, implemented in atotal of 9700 lines of VHDL code, has been completed andsuccessfully tested in a readout setup. In the future, furtheroptimization of the DMA architecture including the soft-ware device driver will have to be performed to minimizememory access operations on the host PC.Both aspects, input interface and timeslice building overInfiniband, have been intergrated in a common data man-agement software project. This allowed to set up and suc-cessfully demonstrate the FLES data chain from data gen-erating boards connected to the FLIB optical inputs to sub-stitute analysis code on a FLES compute node after com-plete timeslice building.NQM-CBM-36 GSI SCIENTIFIC REPORT 201352 doi:10.15120/GR-2014-1-NQM-CBM-36

CBM First-level Event Selector Data Management  Developments

http://repository.gsi.de/record/67782/files/NQM-CBM-36.pdf

CBM First-level Event Selector Data Management Developments

Abstract

Similar works

Full text

Available Versions

GSI Repository