The Level-1 trigger Data Scouting (L1DS) is a novel data acquisition system under development for the Phase-2 CMS detector at the High-Luminosity LHC (HL-LHC). Its purpose is to capture and process Level-1 trigger (L1T) information at the bunch crossing collision frequency of the LHC preceding the standard L1T selections. Referred to as 40 MHz Scouting, this system has the potential for filterless diagnostics for the detector, luminosity studies and investigations into 

signatures and processes that would be otherwise inaccessible or constrained due to the bias introduced by the trigger. An outline of Phase-2 L1DS is provided alongside a description of the Run-3 demonstrator’s architecture and the preliminary results obtained

Giorgetti, S.

Scientific Open-access Literature Archive and Repository

DOI 10.1393/ncc/i2024-24122-0Colloquia: IFAE 2023IL NUOVO CIMENTO 47 C (2024) 12240 MHz Scouting for the CMS Level-1 Trigger(∗)S. Giorgetti(1)(2) on behalf of the CMS Collaboration(1) INFN, Sezione di Padova - Padova, Italy(2) Dipartimento di Fisica, Università degli Studi di Padova - Padova, Italyreceived 13 February 2024Summary. — The Level-1 trigger Data Scouting (L1DS) is a novel data acquisitionsystem under development for the Phase-2 CMS detector at the High-LuminosityLHC (HL-LHC). Its purpose is to capture and process Level-1 trigger (L1T) infor-mation at the bunch crossing collision frequency of the LHC preceding the standardL1T selections. Referred to as 40 MHz Scouting, this system has the potential forfilterless diagnostics for the detector, luminosity studies and investigations into sig-natures and processes that would be otherwise inaccessible or constrained due to thebias introduced by the trigger. An outline of Phase-2 L1DS is provided alongsidea description of the Run-3 demonstrator’s architecture and the preliminary resultsobtained.1. – IntroductionThe proton-proton collisions that the Large Hadron Collider (LHC) delivers to theCMS experiment generate a raw data stream of the order of TB/s. The CMS experimentdesigned a two-level trigger system that ensures excellent sensitivity to most physicssignatures while reducing the event rate from 40 MHz to around 100 kHz at the Level-1trigger (L1T) and then to around 1 kHz through the High Level Trigger (HLT) [1].Introduced in 2011 at the HLT [2], the concept of scouting implies the acquisition ofreduced event-content data at considerably higher rates than the standard accept rate.The 40 MHz Scouting system records the L1T primitives at the LHC bunch crossingfrequency to carry out quasi-online studies on these data with limited resolution. TheLevel-1 trigger Data Scouting (L1DS) acquisition system will be fully integrated into theCMS Phase-2 L1T for the High-Luminosity phase of the LHC (HL-LHC); a demonstratorfor the LHC Run-3 is currently being developed as a proof of concept [3]. The L1DScan provide a high-statistics dataset for detector and trigger diagnostics or luminosity(∗) IFAE 2023 - “Poster” sessionc© CERN on behalf of the CMS CollaborationCreative Commons Attribution 4.0 License (http://creativecommons.org/licenses/by/4.0) 12 S. GIORGETTI on behalf of the CMS COLLABORATIONFig. 1. – Design of CMS Phase-2 L1T (left) and the L1DS (right) [3].measurements. Furthermore, it enables the study of events in unexplored phase space forwhich the trigger may be inefficient due to bandwidth and/or latency limitations. Sev-eral physics processes can be enhanced, for instance, particles spanning multiple bunchcrossing, multiple soft jets or rare decay processes like W → πππ.2. – The Level-1 trigger Data Scouting for CMS Phase-2For the high-luminosity phase, the HL-LHC will deliver an instantaneous luminosity ofup to 7.5 ·1034 cm−2s−1, nearly a fourfold increase to the current condition; additionally,the average number of proton-proton collisions per bunch crossing (pileup) is expectedto reach approximately 200. The upgraded Phase-2 CMS detector will continue to use atwo-level trigger system, with the Phase-2 L1T expected to achieve a 750 kHz rate andaccess, for the first time, the tracker and high-granularity calorimeter data [3]. The L1DSwill benefit from the upgraded design of the Phase-2 L1T, the architecture is presented infig. 1. The L1DS will read several L1 subsystems via spare output links and process themonline with a heterogeneous computing farm adopting a scalable architecture based ontwo stages. Stage 1, which includes the scouting Decision System (sDS) and the scoutingGlobal System (sGS), will mainly provide trigger diagnostic functionality for the GlobalTrigger (GT). The sDS will capture the output of the GT Final-OR and information onprescaling. The sGS will receive the output from the Global Calorimeter Trigger, theGlobal Track Trigger, the Global Muon Trigger, and the Correlator Trigger. Stage 2 canbe additionally included to incorporate the endcap calorimeter primitives and local muonand regional barrel calorimeter trigger via a scouting Local System (sLS). In terms ofhardware, Phase-2 L1DS will employ DAQ-800, the readout board of the Phase-2 CMScentral data acquisition (DAQ). It will host two Xilinx VU35P FPGAs, each with 24x25Gb/s of input bandwidth and 5 QSFP connections that provide 5x100 Gb/s of outputbandwidth [3].3. – The 40 MHz Scouting demonstrator for LHC Run-3The Run-3 demonstrator aims to demonstrate the ability to operate on distributeddata and includes processing units based on FPGA hardware and software [3]. Thetechnical design is displayed in fig. 2. The data recorded from the L1T boards aretransmitted at 10 Gb/s to a patch panel, from where they are distributed to the threeclasses of receiving boards. Table I outlines the four L1T sources and the characteristicsof each receiving board. The VCU128 cards are linked to the host PC via 100 Gb/s40 MHz SCOUTING FOR THE CMS LEVEL-1 TRIGGER 3Fig. 2. – Architecture of the 40 MHz Scouting demonstrator for Run-3 [4].Ethernet links, employing TCP/IP for transfer. The remaining boards utilize a directmemory access (DMA) engine through 10 Gb/s Ethernet links. Initially in firmware,then in software, a granular zero suppression is performed, reducing the data by a factorof about 10 [8]. The raw data are then written to RAM disks (DSBU) and furthercompressed on a second server before being stored in the CMS Lustre global file system.3.1. Preliminary test with the demonstrator and first results at LHC Run-3 . – Thedemonstrator gathered data from the muon and calorimeter detectors for the first timeduring the LHC test beam in October 2021. Regular data collection from the μGMTand DeMux boards began in July 2022, with the start of LHC Run-3. To observe theeffects of the beam halo, we can select μGMT muons reconstructed in the endcap regionsby the Endcap Muon Track Finder (EMTF) in non-colliding bunch crossings (BX). Thebeam halo muons are a consequence of the interaction of beam particles with the pipematerial or residual gas molecules in the vacuum chamber. Findings from the first testare displayed in fig. 3(a): larger beam halo in this fill can be seen for beam 2, whichtravels from the negative z side to the positive z side in CMS coordinates [5]. Figure 3(b)shows the occupancy in the azimuthal angle φ and pseudorapidity η for non-collidingbunch crossing in which the beam halo is visible. Beam halo muons are predominantlyproduced on the beam pipe plane, and a higher occupancy is observed in the detectorTable I. – Description of the L1DS demonstrator receiving FPGA-based boards and of the inputscollected from the Global Muon Trigger (μGMT), the Barrel Muon Track Finder (BMTF), theCalorimeter Trigger (DeMux) and the Global Trigger (GT).Source Objects per BX N. links Receiving boardsμGMT Up to 8 μGMT final muons and 8 Xilinx KCU1500 (KU115 FPGA)BMTF intermediate muons and Micron SB852 (VU9P FPGA)BMTF Input muon stubs 12x2 (24) Xilinx VCU128 (VU37P FPGA)DeMux Up to 12 e/γ, τ , jets. 7+1 spare Xilinx KCU1500 (KU115 FPGA)Energy sumsGT Decision algorithm bits 18 Xilinx VCU128 (FPGA VU37P)4 S. GIORGETTI on behalf of the CMS COLLABORATION(a)-2 -1 0 1 2EMTFGMT [physical units]-3-2-10123EMTFGMT[rad]CMSPreliminaryGMT 40MHz scouting LHC Fill 8484 (13.6 TeV) Beam halo100101102GMT muon occupancy(b)Fig. 3. – Observation of beam halo in L1DS data. (a) Occupancy in the negative (-z) andpositive (+z) muon endcap of opposite charge muons selected requiring a 50 ns delay from thenegative side to the positive, corresponding to ΔBX = 2 [5]. (b) Occupancy of endcap muonsin φ and η coordinates for 1 hour of 40MHz Scouting data taking in 2022 [6].region corresponding to the accelerator plane (φ  0 and |φ|  π).3.2. Boosting muons recalibration with ML algorithms . – Machine learning (ML)models have been developed in the L1DS chain to perform online recalibration of the L1Tmuons and improve the resolution for a physics analysis. The resolution is limited sincethe L1 reconstruction does not have access to the full-granularity data and informationfrom all sub-detectors, as in the offline case. The FPGA-based accelerator employedto run the inference is the Micron Deep Learning Accelerator (MDLA), a proprietaryinference engine that translates ML models into instructions for Micron Technology’sSB852 FPGA-based processing board [7]. The recalibration model takes as input φ, η,the transverse momentum pT, the charge c and the muons reconstruction quality Q; thetargets are the parameters of muons reconstructed offline matched to the L1 muons withΔR =√(Δφ)2 + (Δη)2 <0.1 . The neural network (NN) architecture consists of fourlayers, each with 128 hidden neurons. The training is performed using the Zero Bias 2022dataset, which was collected by CMS without any trigger selection bias, and is thereforeclosely comparable to the scouting data. The difference between the prediction from theNN inference and the results of the offline reconstruction is illustrated in fig. 4. The NNalgorithm proves to be a valid approach to boost the calibration of muon’s parameters.An improvement in resolution is evident for φ, but a smaller gain is expected for η sinceFig. 4. – The distribution of differences for φ, η and pT between the Micron DLA predictionor μGMT values, and the offline reconstructed matched muon. A significant improvement intrack parameter resolution is observed in the MDLA result (red) when compared to the μGMToutput (blue) [8].40 MHz SCOUTING FOR THE CMS LEVEL-1 TRIGGER 5particles do not bend with respect to this angle. Concerning transverse momentum, theobserved offset in the μGMT case is also corrected. This offset arises from the calibrationof L1T objects to ensure a specific efficiency at different pT thresholds. With an expectedthroughput in Run-3 of 2M muons per second, the performance in terms of latency andthroughput is satisfied with around 2.7M inferences per second [8].REFERENCES[1] CMS Collaboration, JINST, 3 (2008) S08004.[2] CMS Collaboration, Phys. Rev. Lett., 117 (2016) 031802.[3] CMS Collaboration, The Phase-2 Upgrade of the CMS Level-1 Trigger, Technical DesignReport CERN-LHCC-2020-004, CMS-TDR-021, https://cds.cern.ch/record/2714892.[4] CMS Collaboration, Development of the CMS detector for the CERN LHC Run 3,arXiv:2309.05466, submitted to JINST (2023).[5] Ardino R. et al., Nucl. Instum. Methods Phys. Res. A, 1047 (2023) 167805.[6] CMS Collaboration, 40 MHz Scouting with Deep Learning in CMS, https://cds.cern.ch/record/2843741 (2022).[7] Micron Technology, Micron DLA Software Development Kit - SDK, https://github.com/FWDNXT/SDK.[8] James T. O., The Level 1 Scouting system of the CMS experiment, CMS-CR-2023-024(2023).

40 MHz Scouting for the CMS Level-1 Trigger

http://eprints.bice.rm.cnr.it/23041/1/ncc12979.pdf

40 MHz Scouting for the CMS Level-1 Trigger

Abstract

Similar works

Full text

Available Versions

Scientific Open-access Literature Archive and Repository