Achenbach, R

Anagnostou, G

Barnett, B M

Bauss, B

Bohm, C

Brawn, I P

Bright-Thomas, P G

Edwards, J

Eisenhandler, Eric F

Engström, M

Garvey, J

Gee, C N P

Gibson, W R

Gillman, A R

Hanke, P

Hatley, R

Hellman, S

Husmann, D

Jakobs, K

Jayananda, K

Keller, M

Kluge, E E

Krause, J

Landon, M P J

Mahboubi, K

Megele, R

Meier, K

Perera, V J O

Pfeiffer, U

Schatz, V

Schmitt, K

Schumacher, C

Schäfer, U

Shah, A A

Shah, T P

Silverstein, S

Staley, R J

Stelzer, B

Stelzer, O

Stock, P

Stokes, W N

Talbot, S D

Thomas, J

Ulses, E

Watkins, P M

Watson, A

CERN Document Server

Study of LVDS Serial Links for the ATLAS Level-1 Calorimeter TriggerG. Anagnostou, P. Bright-Thomas, J. Garvey, R. Staley, W. Stokes,S. Talbot, P. Watkins, A. WatsonUniversity of Birmingham, Birmingham, UKR. Achenbach, P. Hanke, D. Husmann, M. Keller, E.-E. Kluge, J. Krause, K. Mahboubi, R. Megele,K. Meier, U. Pfeiffer, V. Schatz, K. Schmitt, C. Schumacher, B. Stelzer, O. Stelzer, P. Stock, E. UlsesUniversity of Heidelberg, Heidelberg, GermanyB. Bauss, K. Jakobs, U. Schaefer, J. Thomas (e-mail: Juergen.Thomas@Uni-Mainz.de)University of Mainz, Mainz, GermanyE. Eisenhandler, W.R. Gibson, M.P.J. LandonQueen Mary and Westfield College, University of London, London, UKB.M. Barnett, I.P. Brawn, J. Edwards, C.N.P. Gee, A.R. Gillman, R. Hatley,K. Jayananda, V.J.O. Perera, A.A. Shah, T.P. ShahRutherford Appleton Laboratory, Chilton, Didcot, UKC. Bohm, M. Engstrom, S. Hellman, S.B. SilversteinUniversity of Stockholm, Stockholm, SwedenAbstractThis paper presents an evaluation of the proposed LVDSserial data transmission scheme for the ATLAS level-1calorimeter trigger. Approximately 7000 high-bandwidth linksare required to carry data into the level-1 algorithmic processorsfrom the Preprocessor crates. National Semiconductor’sBus LVDS serialiser/deserialiser chipsets offer low powerconsumption at low cost and synchronous data transmissionwith minimal latency. Test systems have been built tomeasure real-time bit-error rates using pseudo-random binarysequences. Results show that acceptable error rates betterthan 10 13 per link can be achieved through compact cableconnector assemblies over distances up to 20 m.I. INTRODUCTIONThe ATLAS level-1 calorimeter trigger [1] requiresapproximately 7000 high-bandwidth serial links to transferdata from the Preprocessor into the algorithmic processorsystems. Each processor module must receive data in excessof 4 GByte/s over these links, with minimal latency and a bit-error rate (BER) better than 10 10 for each link.Serial data transmission has been chosen because the largedata volume (about 800 bits/event per module) makes paralleltransmission unfeasible. The serial links must be able to acceptand transmit data arriving at 40 MHz, which is the LHC bunch-crossing (b.c.) rate. The trigger data fit most efficiently in 20-bit words, so a serial link chipset with either 10 or 20-bit framewidth is preferred to avoid wasted bandwidth.II. REQUIREMENTS ON SERIAL LINK CHIPSETSAND CABLESA. Requirements on serial link chipsetsIt was originally proposed to use Agilent (HP) HDMP-1022/1024 G-link chipsets [2], which have performed well intests, but their very high power consumption gave cause forconcern over crate power and cooling issues. LVDS links offermuch lower power consumption, and the National Bus LVDSserialiser/deserialiser DS92LV1021/LV1212 and LV1023/1224chipsets [3] are easily interfaced to the trigger system whiletransmitting data synchronously with minimal latency. Anoverview of advantages and disadvantages of the two chipsetsis shown in Table 1. The Agilent HDMP-1032/1034 chipsetis ruled out because of its 16-bit frame size, though it hasimproved power consumption. The transmitting chip willbe situated inside a Multi-Chip Module, therefore it must beavailable in die form. The package size must also be compact,since the receiving modules are populated with up to 96 linkreceivers.B. Requirements on cable assembliesThe processor modules will share data via a high-speedbackplane, and the LVDS links will be connected throughthis backplane rather then the front-panel in order to alloweasy installation and replacement of modules. Compact cableassemblies are needed because of the high channel count permodule: up to 96 LVDS channels per module are required andeach 9U processor module requires up to 830 backplane pins. ACompact PCI-type connector series has been chosen to matchthose requirements, therefore cable assemblies need to havematching connectors. The tests focused on the following cabletypes, which are all halogen-free:Table 1Comparison of serial link chipsets. Advantages are shown in bold.Agilent (HP) National Sem.HDMP-1022/1024 DS92LV1201/1212Frame Size 20/21 bits 10 bitsat 40 MHz (800 Mbit/s) (400 Mbits/s)Device Power 2.5 W 0.15 WConsumptionSupply Voltage 5 V 3.3 VTotal Power 100 W 12 Wper ModuleCost/Tower  $170 Tx+Rx  $30 Tx+Rx(May 2000) (x2 for 20 bits)Specified 7.5–75 MHz 16–40 MHzFrame RateConnection DC, AC, Optical DC diff.Methods Single or diff. Kerpen Megaline 726 flex AWG 26 ethernet patch cable[4] Detwyler Uninet flex 4P 600 MHz ethernet cable [5] AMP 1370754-1 AWG28 cable assembly with 2 mm Z-pack connectors. [6]Both the Kerpen and Detwyler cables contain fourindividually-shielded twisted pairs, are relatively stiff, and arefitted with bulky connectors. The AMP assembly containsfour individually-shielded parallel pairs, is more flexible, and isfitted with compact 2 mm connectors matching the connectorschosen for the system backplane.The final installation within ATLAS requires inter-cratelinks over distances of 10–15 m, and a low bit-error rate iscrucial for these links in order to minimise false triggers. Inorder to have a significant safety margin, the tests were runusing cable lengths of up to 20 m.For minimum latency, only error detection, not correction,is possible. To minimise the error rates, the cable assembliesbeing considered require some form of equalisation for theattenuation at high frequencies, as the raw data rate on eachlink is 480 MBaud. Both active and passive pre-compensationtechniques at the transmitter have been investigated.III. LVDS LINK TEST SETUPSThree separate test systems were produced at Birmingham,Heidelberg and Mainz. These involved up to eight channels inparallel, and measured bit-error rates for differing combinationsof precompensation, cable length and cable type. Test systemswere designed to transmit and check pseudo-random andrepetitive data patterns in real time in order to achieve thestatistics required for measurements of very low bit-error rates.In order to do full speed tests with high statistics, the datapatterns transmitted via the serial link have to be checked forbit errors on-board, which is done using programmable logicdevices (FPGAs) on the test modules. Error counters areimplemented for long-time link stability tests: one per linkchannel counts every mismatch of the expected bit pattern andthe data word actually received, and a second one counts alllink breaks, which can be detected by a rise of the LOCK pinon the LVDS deserialiser.A. LVDS Link Tests with TTC clock at BirminghamThe LVDS Link evaluation setup at Birmingham (see Figure1) used Data Source and Sink (DSS) VME modules [7] fittedwith LVDS Serialiser and Deserialiser Common MezzanineCards (CMC). The DSS+LVDS combination was clocked froma TTC test system, using the TTCrx ASIC [8]. Up to eightLVDS link channels have been tested running simultaneously.Xilinx FPGAs on the DSS modules were used to generatepseudo-random bit patterns and detect bit errors within thedata words after transmission over the LVDS link. A passiveprecompensation circuit shown in Figure 2 is used to equalizefor the cable impedance at long lengths greater than 10 m andhigh frequencies.Figure 1: LVDS test setup using DSS motherboard, clocked fromTTC system.Figure 2: Precompensation method using LR (100mH, 51).Initial results showed the original LVDS deserialiserDS92LV1210, when run at its maximum frequency of 40 MHz,had little tolerance to power-supply noise and clock jitter.Although most links could be run with error rates less than10 13, some showed errors at levels of about 10 12 even withsignificant power-supply filtering. These parts could only beused provided that: Transmitters were fed from a high-stability clock source, Receiver supplies were heavily filtered, and Cable HF loss was over-compensated.A faster LVDS chip set has recently been released, and theLVDS link tests were repeated using the faster deserialiser partDS92LV1224. This device has an improved timing margin onincoming data compared with the original part, and showedmuch better tolerance to power supply noise. A comparisonof those two chipsets is given in Table 2.Figure 3: Stream of pseudo-random patterns after 15 m Detwylerethernet cable at receiving end (LV1212). Top: no precompensation,bottom: with precompensation using LR.Table 2Comparison of serial-link chipsets when operated at 40 MHz.Parameter Original devices New devices(LV1210/LV1212) (LV1212A/LV1224)Timing Margin 100 ps 450 psReceiver 100 mV 50 mVThreshold (max)Power 145 mW 191 mWConsumptionLatency 1.75 b.c. 1.75 b.c.No errors were found after several overnight tests on variouscable assemblies, including 15 m of high-density AMP cableand 20 m of Datwyler ethernet cable. A total of 31013 bitswere sent without error over each link. These faster partsno longer need a high-stability 40 MHz clock source feedingthe transmitter. The cable equalisation circuit is less critical,with circuit values now covering a much larger range of cablelengths.B. LVDS Link Tests at HeidelbergThe LVDS link test setup at Heidelberg shown in Figure 4used two VME motherboards of the Modular VME Test System(described in [9]) as carrier for two CMC daughter-boards. TwoCMC cards were designed: a sender daughter-board carryingtwo LVDS serialisers (DS92LV1021) and a receiver daughter-board with two LVDS deserialisers (DS92LV1212). Cables byFigure 4: LVDS link test setup at Heidelberg.Kerpen and AMP already described above were used for serialdata transmission at lengths of both 15 m and 20 m.Bit-error rate measurements are performed using a XilinxFPGA and a dual-port memory sited on the motherboard, andwhich are programmable from a host computer via VME.Various patterns can be tested, either pseudo-random bitpatterns, or repetitive patterns which might be critical for linkstability.Several sets of measurements were performed in order totest data integrity across the transmission assembly. Passiveprecompensation for the cable on the transmitting side of thedata-link has been incorporated, using an inductance (10 nH)in series with a resistor (100 ) between the differentialpair carrying the serial data stream. In Figure 5, a pseudo-random stream is shown at the transmitting and receivingend. A significant loss of average signal amplitude has beenobserved due to the additional resistor across the differentialpair. However, the inductance led to a sharpening of the leadingedges for each transition, compensating for the integratingeffect of the cable itself.The beneficial effect of the precompensation can bedetermined best by examination of the data stream at thereceiving end after the cable, as shown in the bottom partof Figure 5. There is a loss of average amplitude withprecompensation. But the cleaner and wider open eye-patternfor every data-bit is also clearly recognizable. The net result issafer operation of the data-link in terms of bit-error rates. Theinput sensitivity of the LVDS receiver is 100 mV in amplitude,which is only about one-half of the vertical opening of the biteye-pattern. Reliability of data transmission is not only securedby sufficient amplitude, but also by sufficient width on the time-scale.C. LVDS Link Tests at MainzThe LVDS link test setup at Mainz, shown in Figure 6,included a modified version of the Jet/Energy-sum ModuleFigure 5: LVDS link signals: effect of cable length. Top: atransmitting end (LV1021), bottom: at receiving end (LV1212) after20 m cable (AMP assembly with 2 mm connectors); also shown: 40MHz crystal oscillator.Figure 6: LVDS link test setup at Mainz.technology demonstrator (JEM ‘–2’) equipped with twoLV1021 serialiser parts as LVDS link sources. A programmablelogic device (Xilinx FPGA) on the board is used to provideclock strobes and generate the bit pattern. The JEM ‘–2’includes 55 cm of differential onboard tracks (Z=100 ). TheLVDS Data Sink Board receives the data using two LV1212deserialisers. A Xilinx Virtex FPGA does bit-error checks andmonitors the stability of the link lock. The LVDS data sinkboard has 11 cm of on-board tracks (micro-striplines).An active precompensation circuit has been added at thetransmitting end, which is shown in Figure 7. An RC circuitand an ECL driver (EL 89) are used with reference voltageadapted to LVDS signal levels. Cables by Kerpen and AMPMC10H116Vbb Vee270Vcc=+2VOn-board regulators100LVDS in1001K22pF100Vee=Vcc-5V100Precompensation circuitMC10EL89MC10EL16Figure 7: Active precompensation with RC using ECL driver.already described were used for serial data transmission, atlengths of both 15 m and 20 m. The Data Source and the DataSink modules were installed in two different VME crates, bothconstantly accessed from host computers via VME in orderto test the system for stability against noise produced by suchactivity.In order to generate a signal load similar to the final systemon the module on both the serial and parallel side of the links,a circuit was built to supply all eight LVDS deserialisers,with one receiving valid data, and the other seven used as‘noise generators’. It was also found that the Xilinx VirtexFPGA series needs heavily-filtered power supplies (see [10]),otherwise the noise generated by those devices can cause biterrors on the LVDS link.A bit error rate of 10 13 on one channel using 20 m ofAMP cable was reached, which was limited only by the timeof the test run. It was observed that long on-board tracklinescause attenuation of the LVDS signals similar to long cables.Without the precompensation circuit the maximum cable lengthwithout bit errors decreased significantly when using the on-board tracks for transmission. This effect, however, was notseen again when precompensation was used. It was concludedthat the length of the tracklines from the cable connector tothe link deserialiser should be minimized on future modulesin order to optimize the signal quality at the receiving end,therefore improving link stability.IV. TEST RESULTSThe National Bus LVDS serialiser/deserialiser chipsets(DS92LV1021/LV1212 and LV1023/LV1224) are easilyinterfaced to the trigger system while transmitting datasynchronously with minimal latency. BERs better than 10 13per link have been achieved with cable lengths from 10 m to20 m even with simple and straightforward LR equalisation.Experience with the first-generation chipset(LV1021/LV1212) showed that use of these parts wasnot straightforward, operating as they are at the limit of theirspecified data rate. The causes of power supply noise must bekept to a minimum, and board layout is critical. In particular,it is important to ensure that the transmitter clock has a lowlevel of jitter. However, the problems encountered have beenunderstood and solutions found.However, the later version of the deserializer(LV1224/LV1212A) device was found to be much lesssensitive to clock jitter and needs less power supplydecoupling. Therefore, those devices were chosen for theprototype modules. Also, this latest chipset has higheroperating frequencies (40–66 MHz), which allows it to runat the lower end of its rate specification in the level-1 triggersystem.Regarding the choice of cables, it was found that the AMPassembly matches the requirements of the system best, bothelectrically and mechanically. It is flexible and is fitted withcompact 2 mm connectors, which can be easily fitted to thebackplane of the processor systemsV. CONCLUSIONIn conclusion, the LVDS links form a viable scheme fortransfer of large volumes of data, having the advantages oflow latency, low power and low cost. They also offer high-density connectivity, which is essential for compact cableplant. Prototype processors are now being designed that willincorporate a large number of such links.VI. REFERENCES[1] ATLAS Level-1 Trigger Group, ATLAS First-LevelTrigger Technical Design Report, ATLAS TDR-12,CERN/LHCC/98-14, 24 June 1998,[2] Agilent Technologies HDMP-1022/1024 datasheethttp://www.agilent.com[3] National Semiconductor DS92LV1021/ LV1212/ LV1224datasheethttp://www.national.com/appinfo/lvds[4] Kerpen Website: http://www.kerpen.com[5] Detwyler Website: http://www.datwyler.co.uk[6] AMP website: http://www.amp.com[7] DSS Data Source and Sink Module - Specificationshttp://hepwww.rl.ac.uk/Atlas-L1/Modules/Modules.html[8] RD12 - Timing, Trigger and Control (TTC)Systems for LHC Detectors at CERN:http://ttc.web.cern.ch/TTC/intro.html[9] Volker Schatz, Test of a Readout and Compression ASICfor the ATLAS Level-1 Calorimeter Trigger HD-KIP 00-13, Heidelberg, June 2000[10] Xilinx Application Note 158 - Powering Virtex Deviceshttp://www.xilinx.com/xapp/xapp158.pdf

Study of LVDS serial links for the ATLAS level-1 calorimeter trigger

Study of LVDS serial links for the ATLAS level-1 calorimeter trigger

Abstract

Similar works

Full text

Available Versions

CERN Document Server