Ball, R

Chapman, J

Hu, L

Kuah, J

Mann, J

Schneider, M

Uzelac, J

CERN Document Server

Data Flow Simulations through the ATLAS Muon Front-End ElectronicsJ. Chapman, R. Ball, J. Kuah, J. Mann, M. Schneider, J. Uzelac, and L. HuUniversity of Michigan (email: umjwc@umich.edu)AbstractA VerilogHDL simulation of the data flow along thereadout chain of the ATLAS MDT front-end ispresented. The input rates for this simulation are takenfrom the chamber occupancies as provided by theATLAS physics Monte Carlo. The chamber hit-ratesinclude backgrounds as well as hits for collisions ofinterest. The program has been used to study varioustrigger tower groupings and to examine the bufferoccupancies at a range of luminosities.1.  INTRODUCTIONThe ATLAS muon precision chambers areinstrumented with Amplifier Shaper Discriminatorcircuits (ASD) and Time to Digital Converters (TDC)mounted directly on the chamber ends. The ASD unitsconvert the track ionization signals to digital form andthe TDC units digitize, store, and transmit time dataalong serial lines in LVDS form to the readout system.This study begins with this digital information andexamines the performance of various designs upthrough the final on-chamber module called the CSM.Future studies will examine the data flow up to thereadout buffers, called ROBs.Since the performance of the ATLAS muon TDC haspreviously been examined in simulation[1], the firststep in this simulation is to verify that the TDC isappropriately modeled for the hit rates of interest. Thisis done by matching the output buffer occupancies seenin the full TDC simulation to that modeled for thiswork. A simple model of the TDC is needed here sincethis simulation requires the emulation of 18 TDC unitsand full TDC simulation would be prohibitively slow.Eventually this simulation is expected to be extended toa full ROB group for which 108 TDC models wouldhave to be simultaneously run.With a suitable TDC model for emulation of thetrigger and data rates complete and tested, a design forthe next module along the readout chain, the CSM wasundertaken. This unit is required to accept serial datafrom 18 TDC chips, buffer them to avoid data loss,multiplex them into a single output path that is alsobuffered awaiting transmission to the next unit locatedin a Tower Summary Crate (TSC) and called the MuonReadOut Driver (MROD). To date this simulation hasbeen completed up through the CSM module.A preliminary version of the CSM is also to befabricated for chamber testing. This version of theCSM, called the CSM-0, is being designed in VMEformat and will not be chamber mounted as in the finaldesign. It will also not rely on the MROD units forevent assembly. The simulations described in this noteare for the CSM-0 which does contain event assemblylogic. The VME card implementation will also bediscussed briefly.2.  DATA RATES AND TDC MODELING2.1  Predicted Rates in the MDT ChambersData from the MDT tubes flows along a data path asindicated schematically in Figure 1. The individual tubesense wires attach to ASD inputs within a Faraday cagecovering the tube ends. The mezzanine card that holdsthree 8 channel ASD chips also contains a 24 channelTDC. The single tube shown in Figure 1 is thus 1 of 24input to the TDC. Data from all 24 channels is directedto the output upon receipt of a trigger signal to theTDC.Figure 1 also shows the TDC serial data entering thenext module, the CSM. This connection is one of 18such serial links from 18 distinct TDC chips. The CSMmust process each of these 18 sources into a single lineto the TSC based MROD. The data path to the MRODis expected to be a fibre link running at 640Mb/s orgreater. The MROD is required to handle six CSMoutputs and must therefore deal with the data from 108TDC chips or 2592 tube channels.The MROD is designed to accept data from a fulltrigger group, which in the ATLAS MDT requires it toaccept up to 6 chamber units or 6 CSM outputs. Apreliminary grouping of chambers into towers has beenused for this study. Although this grouping is not finalit is representative of the choices that are likely. Inorder to examine the data rates from the MDT up to theROB a preliminary choice has been made for thechamber groups feeding each MROD. The ratesexpressed in the Table 1 are for this preliminarygrouping. The table has been truncated to a section ofthe barrel for simplification.The physics Monte Carlo designated TP43 was usedto calculate the hit rates given in Table 1 for eachchamber of the MDT. This Monte Carlo contains hitsfrom events of interest and hits from backgroundprocesses. All backgrounds are included except halomuons, which are expected to be negligible comparedto those included. All hits  estimated by the physicsMonte Carlo must be handled by the ASD and storedwithin the TDC. All edges, however, are nottransmitted by the TDC to the CSM. Only those foundto be within the drift time window are processed andsent to the output FIFO within the TDC upon receipt ofan external trigger signal. These hits are serialized andsent along the data path to the CSM. Table 1 shows thenumber of tubes for each chamber (#Chn), the averagetube rates (KHz/Chn), the composite rate of all channelhits accepted by the TDC within the drift interval(MHz), the number of  mega-bits sent from the TDCeach second (Mb/s/TDC), the number of mega-bits sentfrom the CSM each second (Mb/s/CSM), the number ofCSM units attached to each MROD (CSM/TSC), andthe number of giga-bits sent each second (Gb/s/ROB)from each MROD. Clearly, handling these rates will bea challenge. The rates from Table 1 represent the rangeof values the CSM and MROD designs must accept.Figure 1 A block diagram of the units throughwhich data flow from the MDT tubes up to theROB.Table 1: Average Data Rates#Chn KHzChnMHz Mb/sTDCMb/sCSMCSMTSCGb/sROBgroup 1BIL 1 240 36 0.9 2.3 22.7 1 22.7BIL 2 288 41 1.0 2.5 29.9 1 29.9BML 1 336 107 2.6 6.0 84.5 1 84.5BML 2 288 107 2.6 6.0 72.5 1 72.5BOL 1 432 75 1.8 4.3 77.6 2 38.8BOL 2 432 75 1.8 4.3 77.6 2 38.8group 2BIL 3 288 405 9.7 22.0 263.6 2 131.8BML 3 288 107 2.6 6.0 72.5 1 72.5BML 4 288 107 2.6 6.0 72.5 1 72.5BOL 3 432 75 1.8 4.3 77.6 2 38.8BOL 4 336 75 1.8 4.3 60.4 2 30.2group 3BIL 4 288 41 1.0 2.5 29.9 1 29.9BIL 5 288 41 1.0 2.5 29.9 1 29.9BIL 6 288 41 1.0 2.5 29.9 1 29.9BML 5 288 107 2.6 6.0 72.5 1 72.5BML 6 288 107 2.6 6.0 72.5 1 72.5Figure 2a The TDC output FIFO occupancy fromthe full TDC simulation.Figure 2b The output FIFO occupancy for thesimplified TDC model for various rates from 1x to9x.2.2  TDC ModelingThe TDC design was formulated in VerilogHDL anda full simulation of its performance exists. Since thecharacter of the simulation is most critical at high rates,a comparison of the full TDC simulation at the highestrate with the simplified version used in this data-flowsimulation has been made. Also since the serializationunit of the TDC processes data from the output FIFOonly, this particular unit has been simulated only. Thus,for each trigger, the number of hits for each TDC isgenerated and injected into the output FIFO. Theprocess begins with a trigger, defined to occur arandomly selected time (exponential distribution) afterthe previous with the appropriate average to producethe desired rate. For each trigger a number of hits isselected randomly from a Poisson distribution with theappropriate mean for the average hit rate.Figures 2a and 2b show the TDC output bufferoccupancy for the full TDC simulation at 9 times theTP43 value, 2a, and for the simplified simulation forvalues from 1x to 9x. The occupancies match well forthe 9x situation. One difference is clearly observed.The TDC has 32 locations in its output FIFO where theTDC Output FIFO - Full Sim - 9x0200040006000800010000120001400016000180000 5 10 15 20 25 30Words UsedFrequencyTDC FIFO Occupancy0100002000030000400000 20 40 60Words UsedFrequency9x6x4x3x2x1xTDCASD CSM TSCAmplifierTime DigitalChamberTower SummaryMDTFaraday Cage Chamber End USA-15ShaperDiscriminiator Convertorwith MRODModulesServiceModulesimplified simulation has 64. Since the TDC has a otherinternal buffers for data, the final position of its outputFIFO is seen to be occupied for cases when the data inthe simplified simulation extends beyond 32. Thisdifference is not important since when the FIFO ishighly occupied the serial unit operates continuouslyunloading the output FIFO. The simplified versionmust have 64 positions in order to avoid loosing datasince it has no other place to hold the hits.3. THE DATA FLOW SIMULATION3.1  The ComponentsThat part of the simulation concerned with modelingthe TDC has already been described. It is shown inbubble of Figure 3 labelled “Emulate 18 TDCs”. Forthe results shown the simulation was performed at 5Hits/TDC and a trigger rate of 75KHz. Other rates havealso been studied.Figure 3 The components of the VerilogHDLsimulation including those that provide the inputspecification, module definition, and performancemonitoring.A second bubble labelled Storage and VME I/O hasalso been represented. This part of the simulation isused for initialization of the TDC and CSM but is notdescribed since it does not function during data flowand is not timing critical.A third bubble labelled TTCem represents thesimulation code for emulating the trigger, timing, andcontrol in accordance with the LHC design. This codeis needed for development of the control signals to theTDC and CSM modules but is not specific to the MDTsystem and is not described further. The actualVerilogHDL code for the TTCem is synthesized so thatthe CSM-0 module performs the appropriate trigger,timing, and control.The primary unit studied in this report is the CSMmodule. It contains the core of the data flow functions.It deserializes the data from the 18 TDC chips andFIFO buffers them awaiting acceptance by a scanningmultiplexer. Data from the multiplexer is accepted if itrepresents input for the current event being sought. TheCSM-0 must therefore have an event FIFO that isloaded from the TTCem event stream and unloaded inturn as events are sought from the multiplexer. TheCSM includes an output FIFO that accumulates data forthe event. The final CSM will send data from its outputFIFO to the MROD within the TSC. The CSM-0,however, sends it output FIFO data to a deep FIFO onthe VME card. It also sends a word count for eachevent along with the event ID to a second FIFO. For theCSM-0 the readout sequence includes a VME read ofthe word count followed by a block transfer of thecomplete event from the data FIFO.A final bubble in Figure 3 represents theperformance monitoring code of the simulation. Thiscode forms histograms of the FIFO occupancies, wordcounts/event, and processing time/event.4.  THE RESULTSA representative simulation is shown in Figure 4athrough 4d. The TDC output FIFO is shown in Figure4a for the 5 Hits/TDC at a 75kHz trigger rate.Figure 4a The TDC output FIFO occupancy at 5Hits/TDC and 75kHz trigger rate.Figure 4b shows the CSM input FIFO occupancy.TDC Output FIFO-100001000200030000 5 10 15 20 25 30Words UsedNumberCSM Input FIFO Occupancy-50005001000150020000 20 40 60Words UsedFrequencyEmulate 18 TDCs(with 5 Hits/TDC at atrigger rate of 75KHz)CSM Module Verilog(with deserialization, buffers,and multiplexers)Serial data from 18 unitsStorage & VME I/O(with TDC initialization,JTAG, and Readout)MonitoringPerformanceTTCem Module Verilog(with trigger, timing and,control)Figure 4b displays the occupancy of the input FIFObuffer of the CSM module. This is the buffer that holdsTDC data awaiting acceptance by the multiplexer.Although the multiplexer scans the incoming datarapidly, this buffer holds appreciable data while theCSM-0 builds an event. The event assembly logic holdsoff processing data for the next event awaiting datafrom the last TDC for a given event. During this timethe other TDC chips that have continued to send newdata.Figure 4c exhibits the word count per event. Forevents with 5 Hits/TDC plus headers and trailers, oneexpects 128 words on the average. The slightly smallerpeak in the distribution remains to be investigated. Itmay be due to round-down of the Poisson generationsince the number generation is integer based.Figure 4c Displays the words/event calculated bythe CSM-0 as it processes an event.The final plot of Figure 4d shows the processingtime (latency) of the CSM-0. The longest time comesfrom events caught behind a burst of data fromprevious events. The shortest time represents theminimum transmission time of events with few hits andwithout contention from previous events. The largestlatency is about 50ms.SUMMARYModeling of hardware with VerilogHDL offers theadvantage of performance determination for criticaldesigns. It also provides the source for the developmentof actual components. If the synthesis of the HDL codeinto either FPGA or ASIC devices can be shown tomeet the clocking specifications, the simulatedperformance can be delivered by the actual hardware.We expect to commit the CSM-0 code to a XilinxFPGA and construct the module within weeks.Figure 4d displays the distribution of processingtime for events assembled by the CSM-0 at the75kHz trigger rate and with 5 Hits/TDC.5.  REFERENCES[1] Requirements and Specifications of the TDC forATLAS Precision Muon Tracker, Yasuo Arai andJorgen Christensen, ATLAS Internal Note, MUON-NO-179, 14 May 1997Words/Event-20020406080 90 100 110 120 130Number of wordsFrequencyProcessing time-500501000 20 40 60Time in 800ns unitsFrequency

Data flow simulations through the ATLAS Muon front-end electronics

Abstract

Similar works

Full text

Available Versions

CERN Document Server