Ardila-Perez, L. E.

Becker, J.

Brinkmann, K.-T.

Calvo, D.

Caselle, M.

Chilingaryan, S.

Cossio, F.

Dritschler, T.

Kopmann, A.

Lenta, F.

Manzhura, O.

Mazza, G.

Peter, M.

Sidorenko, V.

Staněk, P.

Stockmanns, T.

Tomášek, L.

Tröll, N.

Unger, K. L.

Zaunick, H.-G.

English

KITopen

Journal of Instrumentation     OPEN ACCESSThe data acquisition system for the PANDA Micro-Vertex DetectorTo cite this article: O. Manzhura et al 2024 JINST 19 C03036 View the article online for updates and enhancements.You may also likeEffect of Matching Algorithm and ProfileShape on Pulsar Pulse Time of ArrivalUncertaintiesJ. Wang, J. P. W. Verbiest, G. M.Shaifullah et al.-Wide-band Timing of GMRT-discoveredMillisecond PulsarsShyam S. Sharma, Jayanta Roy, BhaswatiBhattacharyya et al.-A 64 channels ASIC for the readout of thesilicon strip detectors of the PANDA micro-vertex detectorG. Mazza, D. Calvo, F. Cossio et al.-This content was downloaded from IP address 141.52.248.4 on 22/04/2024 at 07:222024 JINST 19 C03036Published by IOP Publishing for Sissa MedialabReceived: November 3, 2023Accepted: December 17, 2023Published: March 19, 2024Topical Workshop on Electronics for Particle PhysicsGeremeas, Sardinia, Italy1–6 October 2023The data acquisition system for the PANDA Micro-VertexDetectorO. Manzhura ,𝑎 M. Caselle ,𝑎,∗ L.E. Ardila-Perez ,𝑎 D. Calvo ,𝑏 S. Chilingaryan ,𝑎F. Cossio ,𝑏 T. Dritschler ,𝑎 A. Kopmann ,𝑎 F. Lenta,𝑏,𝑐 G. Mazza ,𝑏,𝑑 M. Peter,𝑒V. Sidorenko,𝑎 P. Staněk , 𝑓 T. Stockmanns ,𝑔 L. Tomášek , 𝑓 N. Tröll,𝑒 K. L. Unger ,𝑎H.-G. Zaunick ,𝑒 J. Becker 𝑎 and K.-T. Brinkmann 𝑒𝑎Karlsruhe Institute of Technology,Hermann-von-Helmholtz-Platz 1, 76344 Eggenstein-Leopoldshafen, Germany𝑏National Institute for Nuclear Physics TurinVia Pietro Giuria, 1 10125 Torino (TO), Italy𝑐Polytechnic University of Turin,Corso Duca degli Abruzzi, 24, 10129 Torino (TO), Italy𝑑University of Turin,Via Giuseppe Verdi, 8, 10124 Torino (TO), Italy𝑒Justus-Liebig-Universität Gießen,Erwin-Stein-Gebäude, Goethestraße 58, 35390 Gießen, Germany𝑓Department of Physics, Faculty of Nuclear Sciences and Physical Engineering,Czech Technical University in Prague,Břehová 7, Praha 1, Czech Republic𝑔Forschungszentrum Jülich GmbH,Wilhelm-Johnen-Straße, 52428 Jülich, GermanyE-mail: michele.caselle@kit.eduAbstract: The PANDA (antiProton ANnihilation at DArmstadt) experiment will study the stronginteraction in annihilation reactions between an antiproton beam and a stationary cluster jet target. ThePANDA detector will be composed of several sub-detectors designed for tracking, particle identificationand calorimetry. The Micro-Vertex Detector (MVD) is the innermost part of the tracking systemsurrounding the interaction region, which is designed for precise vertex and tracking detection. Itconsists of silicon pixel and double-sided microstrip detectors. For the readout of the microstrip sensorsan ASIC called ToASt (Torino Asic for Strip readout) is being developed in 0.11 μm CMOS technology∗Corresponding author.© 2024 The Author(s). Published by IOP Publishing Ltd on behalf ofSissa Medialab. Original content from this work may be used under theterms of the Creative Commons Attribution 4.0 licence. Any further distribution of thiswork must maintain attribution to the author(s) and the title of the work, journal citationand DOI.https://doi.org/10.1088/1748-0221/19/03/C030362024 JINST 19 C03036at INFN Turin. The ASIC takes advantage of both Time-over-Threshold and Time-of-Arrival methodsto accurately measure the event’s energy and timestamp. To sustain the acquisition of the microstripsensors a MDC (Module Data Concentrator) ASIC is under development at KIT. Up to eight ToAStfront-ends’ data streams are multiplexed, read out and processed by one MDC. The data of severalMDCs are collected and processed by the off-detector readout card MMB (MVD Multiplexer Board),also under development at KIT. The processed data is then transferred via 100 GbE optical links tothe computing nodes. The ToASt chips have been integrated with the FPGA implementation of theMDC to form the first fully functional detector module. Beam tests have been performed at the COSYfacility in Jülich. This paper focuses on the design of MDC ASIC and MMB board, the integrationwith the ToASt and presents preliminary beam test results.Keywords: Data acquisition concepts; Electronic detector readout concepts (solid-state); Modularelectronics; Front-end electronics for detector readout2024 JINST 19 C03036Contents1 Introduction 12 Data acquisition architecture of the PANDA-MVD 22.1 Development of the Module Data Concentrator 22.2 Development of the off-detector readout electronics 33 Prototype of the PANDA-MVD DAQ system and test beam results 44 Conclusion and outlook 51 IntroductionThe PANDA experiment, which stands for antiProton ANnihilation at DArmstadt, will be installedat the Facility for Antiproton and Ion Research (FAIR) at GSI in Darmstadt. This experiment aimsto explore fundamental questions of hadron physics through proton-antiproton annihilation. TheMicro-Vertex Detector (MVD), shown in figure 1, is the innermost detector of PANDA.Figure 1. Micro-Vertex Detector (left), barrel and disk detector modules (right).The main tasks are to provide high vertex resolution for primary interaction and secondary decayvertex, improve the momentum resolution and provide additional input for particle identification. TheMVD consists of four detection layers in the barrel region and eight disks in the forward spectrometerregion. To provide high-granularity and vertex information combined with a high multiplicityenvironment, the two innermost layers (barrel) and 6 disks (forward) comprise of more than 10 millionsilicon pixels, while the outer tracker consists of 2 layers (barrel) and 2 disks (forward) of morethan 200 k double-side microstrip channels. These modules, shown in figure 1, are complex hybridstructures that integrate sensor, front-end readout chips and a Module Data Concentrator (MDC)with a cooling pipe on a lightweight carbon fiber support. The DAQ system of PANDA is based ontrigger-less acquisition, where the sub-detectors are free-running and synchronized by the SODANETtime distribution system. Using zero-suppression, only physically relevant signals are transmittedto the computing node for high-level processing.– 1 –2024 JINST 19 C030362 Data acquisition architecture of the PANDA-MVDThe DAQ concept of PANDA requires that every subdetector is able to detect hits without an externaltrigger. Thus, the front-end electronics have to be able to distinguish physical events from noiseand send digital hit information along with a precise timestamp to the event builder. Due to thehigh luminosity, the detector must be able to cope with high interaction rates of up to 20 millionantiproton-proton annihilations per second.To meet these requirements, a new acquisition scheme has been developed. The architectureemploys state-of-the-art technologies to optimize performance while reducing the required hardwareresources. The data received by strip detector modules, processed by ToASt [1] front-ends, are groupedand formatted by the MDC ASIC. Each microstrip detector module will be equipped with two MDCs,one for each sensor polarity. As shown in figure 2, the MDC is connected to the off-detector electronics,the MVD Multiplexer Board (MMB), by optical low power GigaBit Transceiver (lpGBT) [2] link.Figure 2. Data acquisition architecture of the PANDA-MVD detector.The MDC receives commands, configurations, slow-control, and clock signals via the lpGBTin downstream direction and sends the detector data via lpGBT in upstream direction through thee-link channel [3]. The lpGBT links employ the radiation tolerant electro-optical transceivers VersatileLink PLUS (VTRX+) developed at CERN [4]. The MMB is connected to the Synchronization OfData Acquisition (SODANET) which provides information on the beam structure and global timing,necessary for accurate timestamping of the detector data.2.1 Development of the Module Data ConcentratorThe module data concentrator serves as a link between front-end chips and DAQ system. An overviewof the internal architecture of the MDC is shown in figure 3. The MDC acts as a local digital controlleron the detector module. It decodes and buffers the data frames from the front-end chips and performsa time-ordering. This is necessary as the data frames are not aligned according to the event time. TheMDC provides interfaces for slow-control and configuration, it also broadcasts clock and commandsto all front-end chips on the detector module. Additionally, depending on the detector occupancy, theToASt chips can be configured to operate with one or two high-speed serial data links.– 2 –2024 JINST 19 C03036Figure 3. MDC internal architecture and its logic blocks.To accommodate the requirement of different numbers of ToASts combined with a programmablenumber of serial links, the readout channels of the MDC, shown in figure 3, have been developedwith a dynamic negotiation of the number of serial links. If the serial data link of the ToASt isactive, the readout channel will de-serialize the data stream by a Serializer/Deserializer (SerDes)logic shown in figure 3, align the MSB/LSB of the parallel data according to a defined trainingpattern and LOCK the channel. If the channel is locked, it will automatically be included in thereadout, otherwise, it will remain in low-power mode to reduce the MDC’s power consumption. Alllocked readout channels are managed by the main readout Finite State Machine (FSM). The FSMacts as an intelligent multiplexing logic that balances the data occupancy in the FIFO channels. Thereadout data of the detector module are temporarily stored in a local FIFO before being transmittedto the DAQ system via the e-link interface.The proper control and configuration of the front-end chips are crucial for the optimal performanceof the detector. Single-event upsets (SEUs) on the front-end logic and memories, as well as bit errorsthat may occur during the transmission of the configuration sequences, pose a risk of malfunction. Tomitigate the errors, the MDC contains a dedicated configuration logic unit that is able to configureone or more ToASts in parallel, read back the configuration and validate it. If necessary, the logicautomatically re-programs the ToASt chips.2.2 Development of the off-detector readout electronicsThe MVD off-detector electronics consist of the MMB card fully compliant with the AdvancedMezzanine Card (AMC) standard hosted on a full-size Advanced Telecommunications ComputingArchitecture (ATCA) carrier [5]. This form factor makes the MMB also compatible with the DAQ ofother sub-detectors, which are planning to use the Micro Telecommunications Computing Architecture(μTCA) standard. On the MMB, the incoming data rate will be reduced and only physically relevantsignals will be transmitted to the burst-building network. The Zynq Ultrascale+ MultiprocessorSystem on Chips (MPSoCs) located on the MMB cards host multiple ARM processors and a powerfulprogrammable logic layer. The latter allows the implementation of algorithms based on AI for dataprocessing, such as feature extraction and track finding, as well as logic for the dynamic reconfigurationof detectors. The processor part gives the possibility of implementing an online data quality monitor.The MMB architecture is shown in figure 4 on the left, on the right the 3D render of the ATCAcarrier [5] is shown. The MMB will be equipped with six 12-lane FireFly modules for the interface– 3 –2024 JINST 19 C03036Figure 4. Architecture of the MMB card (left) and 3D render of the ATCA carrier [5] (right). The architectureof the carrier can be found in [6]. Reproduced with permission from [6].from/to the lpGBT, connected by GTH/GTY transceivers. Each link will be able to operate up to16.3 Gb/s with a total of up to 36 full-duplex optical links, resulting in a total data rate exceeding580 Gb/s. Up to two 100 GbE optical links will be utilized for the uplink connection to the burst buildingnetwork. It is expected that the data rate after the data processing will be reduced to less than 100GbE, thus presenting the opportunity to utilize the second uplink as a spare connection to the buildingnetwork. Moreover, the MMB provides several standard interfaces, such as PCIe (Gen2), SATA, andGbE (TCIP/IP), which makes it a versatile readout card for a wide range of scientific applications.The ATCA carrier [5] hosts four MMB cards. The carrier board is designed using the OpenIPMCDIMM [7], which uses the JEDEC MO-244 low profile form factor employed in several ATCAcards at CERN [8–10]. The OpenIPMC acts as board controller [11] according to the PICMGspecifications [12]. The carrier has several point-to-point connections between the Zone 2 connectorand the AMCs and a single multi-gigabit transceiver link to each AMC from the onboard AMD XilinxKRIA System on Module. The carrier also distributes the clock network to all AMC cards by meansof a jitter cleaner (SI5395) in the zero-delay configuration for precise timestamping of events. Finally,it provides Ethernet connectivity to all end-points using a 10-port Ethernet switch (VSC7420).3 Prototype of the PANDA-MVD DAQ system and test beam resultsTo test and validate the on-detector functionality of MDC and multiple ToASt/sensor modules, anFPGA version of the MDC has been developed. The prototype of the readout system, based ona HighFlex card [13], utilizes the powerful data transfer with direct FPGA-GPU communication,developed at KIT. The setup enables full control, testing and configuration of the microstrip detectors.For a first in-beam test of the complete detector chain a setup with two single-sided silicon stripsensors, each connected with 64 strip channels to a ToASt front-end, was operated at the COSYproton accelerator facility. The beam test setup is shown in figure 5 (left). The sensors were fullydepleted, one with the n-side strips and one with the p-side strips. The ToASt boards were attachedto a frame in such manner that the sensors are about 2 cm apart, but the strips are orthogonal to oneanother and the active areas overlap. The test sensors originate from the same wafers as the finalMVD sensors and therefore have comparable properties, but are smaller in size. The entire system– 4 –2024 JINST 19 C03036Figure 5. Beam test setup (left) and energy loss in ToASt02 at different angles (right).was mounted in such a way that the sensors could be moved in or out of the beam center and berotated to realise different angles relative to the beam plane.The experiments at COSY were carried out with a 2740 MeV proton beam at an intensity of 1010particles per spill. Since the experiment was a secondary user of the beam, it can be assumed thatimpact products also reached the sensor in addition to the protons. The particle rate should thereforebe significantly lower. Figure 5 (right) shows the energy loss of particles in one of the two ToASts,measured by ToT technique, for angles 0, 20, 60 and 240 degrees. The angle 0 degrees correspondsto the perpendicular impact of the beam on the active sensor surface.4 Conclusion and outlookThe new architecture of the MDC for the microstrip detector of PANDA has been developed andintegrated as fully functional prototype and tested at COSY. For the first time the sensor modulesand the complete readout chain has been tested. The results demonstrate that the concept matchesthe specifications. The prototype system will be used for in-depth characterization of the detectorperformance. Meanwhile, the implementation of the final electronics will be completed.AcknowledgmentsThis work is supported by the German Federal Ministry of Education and Research (BMBF) contractnumber 05P21VKFP1.References[1] G. Mazza et al., ToASt: A 64 Channels Readout ASIC for Silicon Strip Detectors in 0.11 μm CMOSTechnology, in the proceedings of the 2021 IEEE Nuclear Science Symposium (NSS) and Medical ImagingConference (MIC) and 28th International Symposium on Room-Temperature Semiconductor Detectors,Piscataway, NJ, U.S.A. (2021) [DOI:10.1109/NSS/MIC44867.2021.9875549].[2] lpGBT design team, lpGBT Documentation, https://cds.cern.ch/record/2809058/files/lpGBT_manual.pdf,(2022).– 5 –2024 JINST 19 C03036[3] CERN, lpGBT Documentation, https://cds.cern.ch/record/2809058/files/lpGBT_manual.pdf, (2022).[4] CERN, The Versatile Link PLUS Project, https://espace.cern.ch/project-Versatile-Link-Plus/SharedDocuments/VersatileLinkPLUSProjectV2.3.pdf, (2022).[5] L. Ardila-Perez and C. Schmerbeck, ATCA-Example-Project-Template,https://github.com/kit-ipe/ATCA-Example-Project-Template, (2023).[6] O. Manzhura et al., Data Acquisition System of the PANDA Micro-Vertex Detector, https://indico.cern.ch/event/1255624/contributions/5445316/attachments/2745130/4776402/TWEPP23_Poster_final.pdf, (2023).[7] L. Calligaris et al., A flexible and low-cost open-source IPMC mezzanine for ATCA boards based onOpenIPMC, 2022 JINST 17 C03007 [arXiv:2112.12888].[8] A. Rose et al., Serenity: An ATCA prototyping platform for CMS Phase-2, PoS TWEPP2018 (2019) 115.[9] A. Albert et al., The Apollo ATCA Platform, PoS TWEPP2019 (2020) 120 [arXiv:1911.06452].[10] CMS collaboration, The Phase-2 Upgrade of the CMS Data Acquisition, EPJ Web Conf. 251 (2021) 04023.[11] L. Calligaris et al., OpenIPMC: A Free and Open-Source Intelligent Platform Management ControllerSoftware, IEEE Trans. Nucl. Sci. 68 (2021) 2105 [arXiv:2011.01088].[12] PICMG, Advanced Telecommunications Computing Architecture Base Specification,https://www.picmg.org/product/advancedtca-base-specification, (2008).[13] M. Caselle et al., A high-speed DAQ framework for future high-level trigger and event building clusters,2017 JINST 12 C03015.– 6 –

The data acquisition system for the PANDA Micro-Vertex Detector

Repository KITopen

https://publikationen.bibliothek.kit.edu/1000170149/152715623

The data acquisition system for the PANDA Micro-Vertex Detector

Abstract

Similar works

Full text

Available Versions

KITopen

Repository KITopen