Fostier, Jan

Hernaez, M.

Ostermann, J.

Paridaens, Tom

Voges, J.

English

Ghent University Academic Bibliography

GABAC: an arithmetic coding solution for genomic dataTom Paridaens, Jan Voges, Mikel Hernaez, Jan Fostier, and Jo¨rn OstermannIn an effort to provide a response to the ever-expanding gener-ation of genomic data, MPEG (Moving Picture Experts Group),under the auspices of ISO (International Organization for Stan-dardization), is designing a new solution for the representation,compression and management of genomic sequencing data: theMPEG-G standard. This standard specifies an abstract repre-sentation of sequencing data and offers support for, among oth-ers, multi-dimensional random access, extensive meta-data infor-mation, data and privacy protection, data storage and stream-ing. Part 2 of the MPEG-G standard focuses on specifyingthe coding of the sequencing data. Note that the standard isa set of specifications (i.e., a book) assembled after an opencompetition for technologies, and thus, it does not provide, perse, actual encoding implementations. This paper discusses thefirst baseline implementation of an MPEG-G compliance entropyencoder/decoder: GABAC. GABAC combines proven codingtechnologies, such as context-adaptive binary arithmetic coding(CABAC) [1], binarization schemes, and transformations intoone straight-forward solution for the compression of the sequenc-ing data. GABAC will be part of the open-source full MPEG-Gencoder/decoder suite Genie, currently under development bythe Mitogen initiative1.Methods: The MPEG-G standard represents genomic infor-mation by splitting the data into a set of descriptor streams.Each descriptor stream has been designed to contain one spe-cific type of data (e.g., mapping positions, quality scores or mis-match information). Thanks to this approach, all data containedin a descriptor stream is homogeneous. Note that other genomicdata compressors also employ a similar approach of generatingstreams containing statistically similar data [2, 3]. This propertyallows for effective data compression, as redundancies are moreprobable and value distributions are more predictable.Given an input stream, the compression process specified inthe MPEG-G standard consists of a five-step pipeline: inputparsing, (optional) 3-step transformations, symbol binarization,context selection, and CABAC.In the input parsing step, the descriptor stream is parsed intoa stream of symbols. These symbols are then processed by the 3-step transformation step, which converts the symbol stream intoone or more transformed sub-streams. Available transformationsare as follows. i) run-length encoding, where repetitions of sym-bols are translated into a symbol sub-stream and a length sub-stream); ii) match coding, an LZ77-style transformation whereblocks of symbols are replaced by a pointer and length value,indicating either the position and the length of a perfect matchof this block in the previously encoded symbols, or indicating araw symbol if no match is found; iii) equality coding, where asymbol is replaced by a flag indicating equality of the symboland its predecessor and a correction symbol, if required; iv) alook-up table transformation; and v) differential coding.In the first part of the transformation step, the symbols areprocessed using either run-length encoding, match coding, orequality coding. In the second part of the transformation step,1https://github.com/mitogenwhich is applied to each transformed sub-stream separately, alook-up table transformation can be performed. Finally, in thethird part of the transformation step, a differential coding canbe applied.Each transformed sub-stream is then processed separately dur-ing the rest of the process. For each sub-stream a binarizationalgorithm, used for conversion of each symbol into a bit string,is chosen together with a context selection algorithm. In the laststep, each bit of the binarization (called a bin) is combined witha context (selected using the context selection algorithm) andboth are processed using CABAC.Results: To analyze the performance of the GABAC encoder,a test set of 206 descriptor stream files (cropped to a maximumsize of 200 MiB each to emulate random access capabilities) hasbeen selected. These files contain data (such as mapping posi-tions, pairing information, or unmapped reads) generated fromitems 02, 03, 05, 07, 08, 09, 10, and 20 of the MPEG-G Ge-nomic Information Database2. The total uncompressed size is12.97 GiB.The performance of GABAC was compared to the codecs thatare used in CRAM, i.e. gzip, bzip2, xz, rANS order-0, and rANSorder-1. The rANS codecs were extracted from CRAM and arepublicly available3. The tests were performed in parallel on aset of five servers, each equipped with 2 Intel Xeon E5-2650 v3CPUs and 128 GiB of RAM, running Ubuntu 14.04.Table 1 shows the total compressed size and the total encodingand decoding times for each of the tested codecs for all consideredstreams. Additionally, the table shows the compressed size forthe complete test set when, for each file, the codec with thehighest compression ratio across the CRAM encoders is selected(CRAM), and across the CRAM encoders and GABAC (CRAM+ GABAC). GABAC offers the highest compression ratio acrossall coding solutions, while being 5.5 times faster than the secondbest compressor. Additionally, adding GABAC to the CRAMset of encoders offers an additional compression gain of 79 MiBand a speed-up with a factor of 2.4 in compression time.Compressed Encoding DecodingSize (MiB) Time Timegzip 3,524 3h 25m 18s 06m 02sbzip2 3,088 33m 55s 20m 00sxz 2,944 4h 47m 38s 09m 25srANS-0 4,143 06m 01s 07m 08srANS-1 3,400 06m 54s 08m 20sGABAC 2,877 45m 25s 20m 18sCRAM 2,879 2h 25m 58s 09m 32sCRAM + GABAC 2,800 1h 01m 17s 20m 08sTable 1: Total compressed size and decoding & encoding times.Figure 1 shows the compression ranking for each codec andeach descriptor stream file. The x-axis shows the MPEG-G Ge-2https://mpeg.chiariglione.org/standards/MPEG-G/genomic-information-representation/MPEG-G-genomic-information-database3https://github.com/voges/ransFigure 1: Rank of compression performance. Dots were jittered for clarity.Figure 2: Rank of compression speed performance. Dots were jittered for clarity.nomic Information Database Test Set ID from which the descrip-tor stream files were generated. The y-axis denotes the actualranks. Each dot depicts the ranking of a codec achieved on onespecific descriptor stream file. The boxes visualize the followingstatistics: the black lines denote the medians, the lower and up-per hinges correspond to the first and third quartiles, and thewhiskers extends to the smallest/largest values no further than1.5 times the inter-quartile range from the hinges. The dottedread lines denote the mean compression rank for each tool, aver-aged over all Test Set IDs. Overall, GABAC yields the smallestcompressed size among all codecs for 127 out of 206 test items,which is over 60%. As expected gzip never achieves the bestcompression while bzip2, xz and rANS-1 show good compressionresults for certain test items. Furthermore, note that GABACis never ranked in either of the last two positions.Figure 2 shows the compression speed ranks for each codec andeach descriptor stream file. The axis and box plot semantics arethe same as those used in Figure 1. GABAC ranks 3rd in termsof mean compression speed and it is only outperformed by thetwo rANS variants.In conclusion: This work presents the first implementationof a codec which is compliant with the entropy decoding processas specified in the MPEG-G specifications. We have shown thatthe proposed implementation already outperforms well estab-lished entropy codecs improving in several cases both the com-pression ratio and the compression speed. We hope that it willserve as a baseline of future implementations as the performanceof new implementations is expected to improve over time.Acknowledgments: The authors thank IDLab (Ghent Univer-sity - imec) for the use of their computing cluster.[1] D. Marpe, H. Schwarz and T. Wiegand, “Context-based adaptive bi-nary arithmetic coding in the H.264/AVC video compression stan-dard”, IEEE Transactions on Circuits and Systems for Video Tech-nology, vol. 13, no. 7, pp. 620–636, 2003.[2] M. Hsi-Yang Fritz, R. Leinonen, G. Cochrane and E. Birney, “Efficientstorage of high throughput DNA sequencing data using reference-basedcompression”, Genome Research, vol. 21, no. 5, pp. 734–740, 2011.[3] J.K. Bonfield, “The Scramble conversion tool”, Bioinformatics, vol. 30,no. 19, p. 2818, 2014.

GABAC : an arithmetic coding solution for genomic data

https://biblio.ugent.be/publication/8624151/file/8624153.pdf

GABAC : an arithmetic coding solution for genomic data

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography