Beker, H

Carena, W

Divià, R

Fuchs, M

Schindler, M

Van de Vyvre, P

Vascotto, Alessandro

CERN Document Server

ARCHITECTURAL IMPACT OF CUSTOMIZEDCOMPRESSION METHODS ON THESWITCH-BASED ALICE DAQ SYSTEMH. Beker, W. Carena, R. Divia, P. Vande Vyvre, A. VascottoCERN, Geneva, SwitzerlandM. SchindlerTechnical University of Vienna, AustriaM. FuchsUniversity of Frankfurt, GermanyAbstractIn this paper we describe how lossless customized compression can signif-icantly reduce the implementation and operational costs of the ALICE DAQsystem.1. The ALICE DAQ environmentALICE will study phase transition in collisions of heavy ions at ultra-relativistic energies at the projected Large Hadron Collider at CERN. It willproduce data volumes of up to 2.5 Gbyte/s1with typical trigger rates of 80Hz and average event sizes of 30 Mbyte.The parallel architecture shown in Figure 1 distributes this load over anumber of parallel read-out systems, termed Local Data Concentrators. Eachis exposed only to the trac of a part of the detector. A number of parallel eventbuilders process and store the data of all sub-detectors on permanent storagefor later o-line analysis. However, each event builder sees only a fraction of allevents. While this architecture is completely scalable to almost any data rate,for cost reasons we will reduce the throughput as much as possible in order tolimit the number of parallel systems and the amount of data to be stored.The presented methods are not for general use but aimed in particular atexperimental data. As they are adapted to our data they are far superior togeneric algorithms (Human, arithmetic coding, LZW) both in compressionratio and in speed so as a to keep up with the required throughput. They arenow available as commercial hardware with (de)compression speeds of up to 40Mbyte/s. Our approach uses these algorithms after transforming the data intoa more \compressible" data space. Here we concentrate on the compressibility-1RORC103Hz Pb-Pbrun104Hz Pb-PbrunL0 TriggerL1 TriggerL2 TriggerSwitchFCL40 Hz central+40 Min. Bias+1000 Hz dimuon:  Pb-Pb run1000 Hz: p-p runFEE: Front-End ElectronicTDL: Trigger Distribution LinkRORC: Read-Out Receiver CardDDL: Detector Data LinkFEDC: Front-End Digital CrateEBL: Event Building LinkLDC: Local Data ConcentratorGDC: Global Data CollectorTriggerDECISIONSEBLTDLLDCFEDCDDLInnerTrackingSystemFEDCTimeProjectionChamberFEDCParticleIdentificationFEDCPhotonSpectrometerFEDCMuonFEDCTrigger Detectors:- Micro Channel Plate- Zero-DegreeFEERORC RORC RORC RORC RORCFEE FEE FEE FEE FEELDC LDC LDC LDC LDCEDMGDCGDC GDC GDCPDS PDS PDSSwitchPDSFig. 1. Schematics of DAQ.enhancing transformations specic to physics data.2. General remarks on compressionA distinction can be made between lossy and lossless compression methods.Lossy compression is usable where it is permissible to lose quality in trans-mission and storage while retaining the general content of the data such as inpicture, video and audio transmission or storage (e.g. JPEG, MPEG). Here weonly discuss lossless methods which permit a bit-by-bit reconstruction of theoriginal data.Lossless data compression uses the fact that certain data symbols or symbolstrings in the input data are more frequent than others. Variable-length codesare assigned to the input data: i.e. shorter and longer codes are assigned tomore or less frequent symbols respectively. An intuitive example is the Morsecode which assigns \short" to the frequent letter \e" but \short short shortlong long" to the infrequent symbol \:".The Shannon theorem denes the information content of a message in thefollowing way:Given a message made up N symbols { out of a set of n dierent symbols{ the information content of the message measured in bits is the following:I = NPni=1 pilog2(pi)where piis the occurrence probability of symbol i.The denition of symbol depends on the application; it might be an ASCIIcode, 16- or 32-bit words, words in a text etc.Two entropy encoders { Human and Arithmetic Coding { compress dataclose to their Shannon information content. They are described in2and areavailable both as hardware and software.These encoders only consider single symbols and not their order. This articlecan be compressed much better when we introduce meta symbols for sequencessuch as \the", \compression" and \symbol'.The Lempel/Ziv/Welch algorithm autonomously searches for frequent sub-strings and codes them arithmetically. It achieves compression ratios whichare typically higher than 60%. On modern high-end microprocessors the im-plementation of the popular program gzip achieves a throughput of about 1Mbyte/s in compression, which is not sucient for our application.3. Compression of physics quantitiesLet us assume that a charge or any other quantity is measures using an8-bit digitizer. It is distributed approximately exponentially with a mean valueequal to one tenth of the dynamic range i.e. 26. Each value between [0..255] isconsidered a symbol. Applying the above formula to this distribution a meaninformation content of 6.11 bits per measured value is obtained. This is almost25% less than the 8 bits required for saving the data as a sequence of 8-bitbytes. If the dynamic range is increased by a factor of four using a ten-bitADC, it turns out that the mean information content is virtually the sameand hence the possible compression gain even higher (39%). The reason forthis is that an exponential distribution delivers a value which is more than tentimes its mean only every e10= 22026 samples. Even quite long codes for suchmeasurements have no appreciable inuence on the compression rates. Giventhat in a realistic architecture 10 bits have to be expanded to 16, the gain is62%.3.1. Data-specic compressionThe general-purpose algorithms proceed in the following three steps:1. They create the statistics for symbols and symbol sequences.2. They create an optimal (or at least good) symbol to code table.3. They encode the data according to this table.Steps 1 and 2 are the most time consuming. In experimental data the statisticalproperties of the data are known a priori. This allows step 3 to be performedimmediately with precalculated symbol{code tables.General algorithms have another basic disadvantage in our application.They make no assumption regarding the structure of their input data. Theyperform processor-intensive searches for features such as symbol sequenceswhich might not be present at all. They do not know what to consider asa symbol (1-bit, byte, word, long word etc) and usually assume bytes. In ourcase data will often be 10- or 12-bit quantities.It is often possible to transform the data bijectively into a space where theyare less evenly distributed and hence compress much better.3.2. Compression of zero-suppressed data3.2.1. Eect of zero suppression on data compressionThe ALICE detector will contain about 109sensitive elements. If they wereto be recorded for each event, this would result in an enormous data volume.Therefore the analogue front end electronics and connected read-out busesrecord only data above a certain threshold. Instead of retrieving all data valuesin a stream, alternating sequences of addresses and measurements are presentedto the read-out bus systems. This method is not lossless, but detector design-ers know the minimum signal which can be distinguished from the inherentdetector noise. Only a minor fraction of all connected channels in a given eventsurpass this threshold. This results in an enormous reduction factor without in-troducing dead time, as the suppression algorithm can be executed in hardwareand in parallel over all detector channels.However, the resulting output shows much less, if any, statistical correlation,particularly in the address part. If present, the correlation is hard to detectby standard algorithms as data and addresses are packed into compound datawords. The limit in general does not coincide with a byte boundary. Therefore,generic compression algorithms perform poorly on zero-suppressed data.In calibration events in which many or all data channels are to be recorded,the amount of data can actually increase as the addresses are added to thedata. Compression will not reduce the highly-redundant address part in theseevents.3.2.2. Enhancement of the channel-address compressibilityThe measurement values are separated from the added address. Measure-ment values follow their own statistical distributions; this can be exploited byassigning them variable length codes.The ALICE pixel detector, for example, produces only addresses and nomeasurement values. The front-end electronics outputs a list of pixel numbers.16-bit addresses are required for the 65.536 channels of each detector unit. Thebinary data value is implicit and zero-suppression lossless; if an address doesnot appear in the stream, it means that the associated data value is zero.There are no major inhomogeneities in the occupancy, i.e. the distributionof addresses will be quite uniform between 0 and 65.535. Therefore standardcompression methods are of no use.The addresses are replaced with the address dierence of a hit pixel withrespect to the previous pixel. This can be accomplished by a very minor changein the multiplexing hardware.The original addresses can easily be reconstructed from the new sequenceand the address dierences will be distributed exponentially with a mean of50 in the case of a 2% occupancy. The Shannon information content of thisThreshold 1Cluster Threshold 2ChargeThresholdCluster 1Cluster 2Threshold Cluster 2 before  + 2 afterHysteresisTime Time TimeFig. 2. Cluster denition.distribution is 9.3 bits per hit instead of 16, resulting in a 42% data compression.A residual correlation in the address dierences due to charge sharing betweenneighbouring pixels will actually allow a 50% compression.3.3. Enhancement of the compressibility of measurement valuesThe format and amount of data coming from the time projection chamber(TPC) in ALICE are discussed in4. At present, we assume that the data fromthe TPC will be zero-suppressed. Figure 2 shows how clustered zero suppressioncan be performed on the TPC raw data. As the time charge buckets are highlyclustered it is not necessary to apply an address to every word; the followingdata structure should suce:charge 1, charge 2, .... ,charge N,address of rst bucket,# of charge buckets.next cluster.We expect about 12.000 tracks in the TPC. Each charged particle traverseson average 75 circular concentric pad rows. Each of these crossings will involve4{5 neighbouring pads and about 6{7 consecutive time charge buckets. Twobytes are needed for the encoding of the cluster start and the cluster length.At this point we assume a dynamic range of 10 bits and 8-bit logarithmicencoding. The total amount of data produced after zero suppression will beabout 35 Mbyte/s before compression. This is the dominant contribution toour sustained bandwidth of up to 2.5 Gbyte/s.3.3.1. Treatment of analog measurementsThe following considerations will be applicable to a wide range of detectorsdata: Digitization accuracy has to be traded o against data volume. In mostpractical cases the accuracy is chosen so as to be equal to or double theroot mean square of the electronic noise. The introduced digitization noiseonly adds 4 or 16% respectively to the intrinsic uctuations. On the other hand { as already illustrated in the initial example { dy-namic range does not inuence the information content signicantly, aslong as the measured quantity is distributed approximately exponentially.Coding measurement values logarithmically presents no gain after entropyencoding.ChargeBucket numberRaw Cluster Raw Model functionfor cluster length 6Raw Cluster -Normalized Model functionNormalized Model functionfor cluster length 6Bucket number Bucket number Bucket numberFig. 3. Lossless waveform modelling. Furthermore, compression benets from on-line equalization of the gainsand pedestals of the amplication electronics and digitizers. This narrowsthe collective distribution of the measurement values and leads to bettercompressibility. Finally the digitizer sensitivity can be limited to the plausible spatialand temporal frequency response of the detector by employing analogueor digital lters to the measurements. Besides reducing noise and henceuseless information content, this procedure will be particularly benecialfor the waveform modelling method of the TPC and the silicon drift dataillustrated in the following section.3.3.2. Lossless waveform modellingThe cluster addresses and lengths will be encoded using their own speciccode trees. The incremental address-storing approach shown for the pixel de-tector will further reduce the amount of data from addresses.Using the assumptions of the initial example which match our TPC dataonly 25% data reduction on charge measurements can be achieved through en-tropy encoding. For higher compression rates a detector-specic compressibility-enhancing transformation is perforemed which also applicable for the silicondrift chamber data.As sketched in Fig. 3, instead of saving only cluster address, length andthe time buckets, the mean bucket contents will be added to the cluster data.The time charge buckets are replaced by their residuals with respect to a scaledmodel function.Scaling involves integer divisions, but rounding errors are not relevant, as theresiduals also account for the rounding errors. The residuals have a narrowerdistribution than the raw measurements and yield better compression morethan compensating for the added cluster average.The model function is obtained a priori by measuring the mean pulse shapeof uncompressed data which is known to the encoder and decoder and hencenot part of the data stream.Exact values for the compression ratio will be obtained after evaluation withreal data. In the ideal case, the waveform is modelled perfectly and the residualsare only due to the inherent detector noise. If the digitization accuracy chosenis equal to the noise, the number of bits required is 2.15 per residual or 14 forthe whole pulse containing 6{7 samples. After compression, address, clusterlength, and cluster mean require 16 bits. This makes about 4 bytes comparedwith the initial 8{9. When choosing half this accuracy, 3 bytes are for encodinga cluster.Because of correlation of the noise in the time domain, residuals will formtypical sequences and be encoded more eciently by advanced compressionmethods.This method can losslessly reduce the TPC data by 50%.3.4. Data-compression implementationData will be compressed as early as possible and no later than in the front-end digital crates in order to relax the bandwidth requirements for all followingsubsystems, including the automatic o-line tape-handling facility.As the various transformation steps in the proposed data-modelling schemesare very experiment- and detector-specic, they will be handled in software orin reprogrammable hardware. The nal step of encoding can be handled bygeneral-purpose chips. They perform a combination of LZW and arithmeticcoding in real time.4. ConclusionsCustomized compression algorithms could allow us to reduce the bandwidthrequirements of the ALICE DAQ system by a factor of about two. This has apositive impact on the cost of all subsystems which in most cases scales linearlywith the achieved data reduction. Assuming a tape cost in the order of $1 perGbyte, this will help to reduce the operational DAQ cost stemmingmainly fromthe magnetic tapes by several million dollars a year.5. References1. The ALICE Collaboration, ALICE Technical Proposal for A LargeIon Collider Experiment at the CERN LHC, CERN/LHCC 95-71,LHCC/P32. M. Nelson, The data compression book Prentice Hall, 1991.3. IBM, IBM ALDC1-40S Data Sheet4. A. Rybicki, Analytical Previsions of r Resolution of the ALICE TPC,available from the author (e-mail: arybicki@chall.ifj.edu.pl)

Architectural impact of customised compression methods on the switch based ALICE DAQ system

Abstract

Similar works

Full text

Available Versions

CERN Document Server