Cameron, W

Foudas, C

Hall, G

Iles, G

Marinelli, N

CERN Document Server

TheAPVE emulator to prevent front-end buffer overflows within the CMSSilicon Strip TrackerG. Iles, W. Cameron, C. Foudas, G. Hall, N. MarinelliBlackett Laboratory, Imperial College London, Prince Consort Road, London, SW7 2BWgm.iles@ic.ac.ukAbstractA digital circuit board, employing Field ProgrammableGate Array (FPGA) technology, has been built to emulatethe logic of the pipeline memory of the APV25 readoutcircuit for the CMS Silicon Strip Tracker. The primaryfunction of the APVE design is to prevent buffer overflowsin the APV25. It will also provide information to the FrontEnd Drivers (FEDs) to ensure synchronisation throughoutthe Silicon Strip Tracker. The purpose and functionality ofthe APVE is presented along with a prediction of theperformance from simulation results.I. INTRODUCTIONThe CMS Silicon Strip Tracker APV25 readout chip [1,2] is designed to record analogue data at a rate of 40MHz.Data are stored in analogue pipelines on the readout chip.Upon reception of a Level 1 Accept (L1A) signal from theTrigger Control System (TCS) [3], data are transferred~100m via optical links [4] to Front End Driver (FED)cards [5] located in the CMS electronics room. The FEDsdigitise the analogue data and employ fast FPGAs to applypedestal and noise corrections and to reduce the raw datasample by cluster finding. The clustered data are thentransmitted to the CMS DAQ via S-links[3, 6].The maximum average frequency of L1As will be100kHz (10s period) while events can be read out fromthe APV25 at a rate of one event per 7s. To allow forPoisson fluctuations of the First Level Trigger (FLT) rate,buffers have been introduced both at the APV25 and theFED level. In the APV25 this has been achieved by simplyextending the analogue pipeline already necessary forbuffering data until receipt of a L1A. The CMS Triggerdesign [7, 8] requires that all readout buffers are monitoredand that their status classification (Busy, Ready, Warning-Overflow, Error, Out-Of-Sync) is transmitted back to theTCS. The TCS may then inhibit triggers from the FLT ifthe status is Busy or take other action depending on theinformation it receives.Monitoring the APV25 buffers, which are located onthe CMS detector, poses a particular challenge becausePoisson fluctuations of the FLT rate can produce L1Aswithin time intervals which are shorter than the travel timeof the monitoring signals from the CMS detector to theCMS electronics room. Fast monitoring signals comingfrom the CMS detector cannot therefore providesufficiently early warning that APV25 buffers are about tooverflow. The consequence is that the APV25 bufferswould overflow and data would be lost. The APV25 wouldneed to be reset.Figure 1: The APVE boardTo avoid buffer overflows, an APV25 buffer emulatorboard (APVE) which emulates exactly the status of theAPV25 buffers is under development (fig. 1). The APVEwill be installed very close to the TCS crate. Hence, it willbe able to report the status of the APV25 buffers to theTCS (Trigger Control System) quickly enough. The APVEuses fast FPGA technology and is capable of determiningthe APV25 status in  2 LHC clock cycles, thus ensuringmaximum buffer efficiency and preventing APV25 buffersfrom overflowing.The APVE will also be used to transmit the address ofthe memory cell in the APV25 that was used to buffer theL1A event. This address is included in the header of theAPV25 data frames sent to the FEDs. The “golden”pipeline address received from the APVE will be comparedwith the actual ones coming from the tracker, thusproviding an important verification that synchronisation hasbeen maintained.II.BUFFER OVERFLOWS IN THE APV25When the APVE detects that a buffer overflow is aboutto occur it will send the status signal Busy to the TCS. TheTCS will respond by vetoing further L1As.L1As may be issued at maximum rate of 1 every 3bunch crossings. Consequently, if we want to makemaximum use of the APV25 buffers by only vetoing L1Aswhen all the buffers are full we need the control loop fromTCS L1A inhibit gate to APVE and back again to take lessthan 3 bunch crossings. If this were not the case a furtherL1A may be allowed through by the TCS before the APVEhas had time to report that the APV25 buffers are full. Theadditional L1A would cause the full buffer to overflow andall APV25s would need to be reset. Control loops largerthan 3 bunch crossings must assert Busy before the buffersare full to provide space for additional L1As that may passthrough the TCS before the L1A inhibit is applied. Thenumber of buffers required for additional L1As after Busyhas been asserted (i.e. determined by the magnitude of thecontrol loop) is set by a VME write.III. CONTROL STRUCTURETo form a small TCS-APVE control loop the followingcontrol structure has been adopted (fig. 2). The APVEreceives the LHC clock and following control signals; L1A;Level 1 Reset (L1Reset); Bunch Crossing 0 (BC0); EventCounter Reset (ECR); Orbit Counter Reset (OCR) fromboth a Central TCS (CTCS) & Local TCS (LTCS). TheLocal TCS allows the Tracker to operate when the mainTrigger system is down for maintenance. CTCS or LTCScan be selected under VME control.The APVE sends back the Tracker status signal whichis either Busy, Warning-Overflow, Out-Of-Sync, Ready orError to the active TCS. Error is asserted to the unusedTCS. The Tracker status is derived by combining theAPVE status signal with the equivalent ones coming fromthe FEDs.The APVE clock and control signal input from eachTCS, the APVE status output to each TCS and mergedFED status input are transmitted as LVDS logic onstandard Ethernet type cable and connectors (i.e. 4 bit,twisted pair cables of approx 100 ohm impedance with RJ-45 connectors). The control signals are encoded using3bits, with the remaining bit used for the LHC clock. TheAPVE status is encoded with all 4 bits.The “golden” APV25 pipeline address is transmitted tothe FEDs via the TTCci B channel [9]. APV25s in theTracker receive L1As and control information via the FECand CCU ring control system. They send data frames,which include the pipeline address of each event, to theFEDs where the pipeline address is checked.The Tracker can be operated in 4 separate partitions andthus 4 APVE modules are required.APVE TTCci/ex/txAPVCTCS LTCSFECCCU RingFEDFMMOther FEDsFigure 2: The CMS Tracker control system. Control signals(solid line) are generated by the Central or Local TCS. Data issent from the APV25s to the FEDs (dotted line). The APVE andFED status are reported back to the Central and Local TCS (dot-dash line). The “golden” pipeline address (dashed line) istransmitted to the FEDs via the TTCci B channelIV. IMPLEMENTATIONThe APVE is a standard 6U VME board with A24/D16addressing. All the logic, with the exception of that used toprovide a back up clock is implemented in a single XilinxVirtex-II FPGA (XC2V1000) [10].To monitor the buffer status of the APV25 two optionshave been implemented. The first and simplest methoduses a real APV25 chip and a counter to keep track of thenumber of events waiting to be read out of the APV25.The counter is incremented every time a L1A arrives anddecremented every time an event is read out from theAPV25.The second method uses an FPGA emulation of theAPV25 in real time. This provides complete knowledgeabout the status of the internal APV25 buffer logic and thusthe maximum buffer efficiency is achieved. This methodplaces stringent demands on timing within the FPGA yetearly indications are that it will work.In addition to the buffer status logic the FPGA containsa VME bus to Wishbone bus [11] bridge that allows extraFPGA components to be easily added. The componentscurrently attached to the Wishbone bus are; the APVEcontrol and status registers; an I2C interface forcommunication with the real APV25; a 64bit wide by 4kdeep status memory. The latter records any change instatus of the APV25 emulator, FED input, or Central andLocal TCS feedback signals. The 64bits are composed ofthe status signals (4x4bits), bunch crossing number(12bits), orbit number (32bits) and reserved area (4bits).V. TRACKER EFFICIENCYThe Tracker induced dead time versus the length of thecontrol loop has been studied. The APV25 can be operatedin two modes. In peak mode 31 L1A buffers are availablewhereas in deconvolution mode only 10 L1A buffers, eachcontaining three pipeline samples, are available.Deconvolution is therefore the most demanding mode ofoperation.0.0124680.124681TrackerdeadtimeduetoAPV25(%)10 9 8 7Number of APV buffers used129630Control loop size (LHC bunch crossings)Figure 3: Tracker deadtime versus the magnitude of the controlloop and the maximum buffer size allowed by such a control loop.The APV25 is operating in deconvolution mode with a maximumbuffer size of 10 and a raw L1A rate of 100kHz. Buffer sizecalculated and thus deadtime calculated using both a real APV25(dashed line) and a simulation of the APV25 (solid line). Thebetter performance achieved with simulation is due to a greaterknowledge of the state of internal APV25 buffer logic.The Tracker deadtime has been plotted (fig. 3) versusthe number of buffers used for a 100kHz raw L1A rate withthe APV25 in deconvolution mode. L1As which wouldhave produced a L1A separation of less than 3 bunchcrossings were vetoed, but any other possible trigger ruleswere ignored. The deadtime was calculated from the ratioof L1As that were vetoed because the APVE was assertingBusy and the raw L1A rate. The buffers were monitoredusing a real APV25 (dashed line) and a C code simulationof the APV25 internal buffer logic (solid line) derived fromprevious work [12].The C code simulation performs better than the realAPV25 because we have more precise knowledge of thebuffer states within the APV25. The C code has beentranslated into the hardware description language VHDL,which will be downloaded into the FPGA.The current control structure (fig. 2) should produce acontrol loop of just a few clock cycles. The number ofbuffers used in deconvolution mode should therefore be 8,9 or all 10. This corresponds to a deadtime of <0.25% if areal APV25 is used or <0.13% if a hardware emulation ofthe APV25 is used.VI. PIPELINE ADDRESS TRANSMISSIONThe APVE “golden” pipeline address is transmitted tothe FEDs by the Trigger, Timing and Control (TTC)system [13]. The TTC distributes to the experiment, via asingle optical fibre, a clock, and 2 data channels. ChannelA is used for L1A transmission whereas channel B is usedfor control commands. The TCS initiates B channelcommands and the TTCci formats them. Channels A & Bare then sent to a TTCex for encoding and transmission onoptical fibre.The “golden” pipeline address will be sentasynchronously on the B channel so that it does notinterfere with synchronous commands being initiated bythe TCS. However, the pipeline address must not beblocked by transmission of synchronous B channelcommands for a long time period if the pipeline address isto reach the FED before the FED is ready to check it. Thisrequires guaranteed asynchronous bandwidth on the Bchannel of  42clks (pipeline address transmission length)[14] every 280clks (APV25 data frame length). Thisrequires the TCS, that initiates the B commands, andTTCci, that implements the B commands, to follow certainrules.The TCS (Central & Local) must separate synchronousB channel commands by  88clk cycles (fig. 4). Thisensures that a pipeline address may be transmitted betweensuccessive synchronous B channel commands. The lattershould take no longer than 84clk cycles to complete if aninhibit period of 42clk cycles is chosen on the TTCci. Thepipeline address will take longer than the 4clk cycle gap tobe sent, but once started it is allowed to complete duringthe inhibit period that exists prior to a synchronouscommand being sent (16 or 42clk cycles). The purpose ofthe inhibit period is to free the B channel for a synchronouscommand by allowing any currently executing command tofinish, but by blocking any new command from starting.Tracker TTCci must maintain this command separation.All B channel commands must be initiated by the TCS andhave the same inhibit duration of 42clk cycles, and havethe same inhibit delay. TTCci “doubles” are not allowed.VII. CONCLUSIONSA VME board has been designed to prevent bufferoverflows in the CMS Tracker. The design of the boardand its integration into the CMS control structure ensurethat the Tracker deadtime is kept to a minimum.The deadtime is determined by the magnitude of thecontrol loop formed between the APVE and the TCS andthe type of buffer monitoring chosen. The latter can beachieved with either a real APV25 chip or a hardwareemulation of the chip in an FPGA. If we assume a TCS-APVE control loop of less than 9 clock cycles these yield adeadtime of less than 0.25% and less than 0.13%respectively.VIII. ACKNOWLEDGEMENTSWe would like to thank the following; SarahGreenwood for her layout of the design; John Coughlan forhis comments on pipeline address transmission on the Bchannel; Rob Halsall for his help with FPGA designsoftware; Joao Varela and Anton Taurok for discussionsabout integrating the APVE into the CMS control structure.IX. REFERENCES[1] M. Raymond et. al., “The CMS Tracker APV250.25µm CMOS readout chip”, Sixth Workshop onElectronics for LHC Experiments,CERN/LHCC/2000-041.[2] The Tracker Project Technical Design Report,CERN/LHCC/98-6.[3] A. Racz et. al., “Trigger Throttling System forCMS DAQ”, Sixth Workshop on Electronics forLHC Experiments, CERN/LHCC/2000-041.[4] F. Vasey et. al., “Project status of the CMS opticallinks”, Sixth Workshop on Electronics for LHCExperiments, CERN/LHCC/2000-041.[5] J. Coughlan et. al., “The Front-End Driver Cardfor the CMS Silicon Strip Tracker Readout”,These proceedings.[6] A. Racz et. al., “CMS Front-End/DAQInterfacing”, Sixth Workshop on Electronics forLHC Experiments, CERN/LHCC/2000-041.[7] The Trigger and Data Acquisition (TriDAS)Project Technical Design Report, Volume 1, TheLevel-1 Trigger, CERN/LHCC/2000-038.[8] J. Varela, “Timing and Synchronization in theLHC Experiments”, Sixth Workshop onElectronics for LHC Experiments,CERN/LHCC/2000-041[9] J. Varela, “CMS L1 Trigger Control System”,Draft CMS Note 2002[10] Xilinx, “Virtex-II Platform FPGA Handbook”,www.xilinx.com.[11] WISHBONE System-On-Chip (SoC)Interconnection Architecture for Portable IPCores, Revision: B.1, Silicore Corporation, 6310Butterworth Lane, Corcoran, MN 55340[12] N. Marinelli, “APV Logic Simulations”, CMSNote 1999/028[13] B. Taylor, “Timing Distribution at the LHC”,These proceedings.[14] J. Christiansen, A. Marchioro, P. Moreira and T.Toifl, “TTCrx Reference Manual, February 2001,Version 3.2”, RD12 Working Document, CERN.Inhibit_X (42)Sync_X (16/42) Pipe Address_B (42)BGo_Y (1)BGo_X (1)Inhibit_Y (42)Sync_YB Chan Data88clksTCS initiates command “X” with BGo_X signal.TTCci inhibits new commands from starting for 42clk cycles thus guaranteeing theB channel to be free when the synchronous command “X” is sent.Sync Command “X” is transmitted.New B channel command “Y” initiated by TCS after 88clks, but pipeline address “B” is startedbefore inhibit “Y” starts (shaded area) and once started is allowed to complete.Figure 4: Diagram showing the “golden” pipeline address being transmitted on the B channel. The top line shows the TCS initiatingtwo different synchronous commands. The middle line shows the inhibit generated for each synchronous command. The bottom lineshows B channel usage. The time taken in clock cycles for each event to complete is shown in brackets after the event description.

The APVE emulator to prevent front-end buffer overflows within the CMS silicon strip tracker

Abstract

Similar works

Full text

Available Versions

CERN Document Server