Acte onlineReference paper: Benoit et al. (2016) Multiple comparative metagenomics using multiset k-mer counting. PeerJComputer Science. https://doi.org/10.7717/peerj-cs.94Simka: large scale [i]de novo[/i] comparative metagenomics. JOBIM 2017 - Journées Ouvertes Biologie Informatique Mathématique

BENOIT, Gaëtan

Drezen, Erwan

Lavenier, Dominique

Lemaitre, Claire

Mariadassou, Mahendra

Peterlongo, Pierre

Schbath, Sophie

Acte online Reference paper: Benoit et al. (2016) Multiple comparative metagenomics using multiset k-mer counting. PeerJ Computer Science. https://doi.org/10.7717/peerj-cs.94 Acte online Reference paper: Benoit et al. (2016) Multiple comparative metagenomics using multiset k-mer counting. PeerJ Computer Science. https://doi.org/10.7717/peerj-cs.94Acte onlineReference paper: Benoit et al. (2016) Multiple comparative metagenomics using multiset k-mer counting. PeerJComputer Science. https://doi.org/10.7717/peerj-cs.94Simka: large scale [i]de novo[/i] comparative metagenomics. JOBIM 2017 - Journées Ouvertes Biologie Informatique Mathématique

Benoit, Gaëtan

INRIA a CCSD electronic archive server

Version preprintComment citer ce document :Benoit, G. (Auteur de correspondance), Peterlongo, P., Mariadassou, M., Drezen, E., Schbath,S., Lavenier, D., Lemaitre, C. (2017). Simka: large scale de novo comparative metagenomics. In:JOBIM 2017 LILLE (p. 3-5).  Presented at JOBIM 2017 - Journées Ouvertes Biologie InformatiqueMathématiques, Lille, FRA (2017-07-03 - 2017-07-06).Simka: large scale de novo comparative metagenomicsGaëtan BENOIT1, Pierre PETERLONGO1, Mahendra MARIADASSOU2, Erwan DREZEN1,3, SophieSCHBATH2, Dominique LAVENIER1 and Claire LEMAITRE11 INRIA/IRISA, Genscale team, UMR6074 IRISA CNRS/INRIA/Université de Rennes 1, Campus deBeaulieu, 35042, Rennes, France2 MaIAGE, INRA, Université Paris-Saclay, 78350, Jouy-en-Josas, France3 CHU Pontchaillou, 35000, Rennes, FranceCorresponding author: gaetan.benoit@inria.frReference paper: Benoit et al. (2016) Multiple comparative metagenomics using multiset k-mer counting. PeerJComputer Science. https://doi.org/10.7717/peerj-cs.94Large metagenomic projects, such as Human Microbiome Project [1] or Tara Ocean Project [2], are be-coming increasingly important for understanding life processes at the individual scale or at more global scales.Therefore, serious efforts are put in funding projects, collecting DNA, sequencing, and analyzing the resultingsequences. Comparative metagenomics is one of the most ubiquitous and informative of those analyzes. Thepurpose is mainly to estimate proximity (or distance) between two or more environmental sites at the genomiclevel. The comparisons are often based on identified species content. However, this approach is limited to se-quences correctly assigned to known species documented in public biobanks, and this may correspond to smallfractions of the datasets, in particular for environmental samples such as sea water. This limitation motivatedthe development of de novo comparison tools, such as Compareads [3] or Mash [4], based only on the nonassembled read set comparisons. Compareads was for instance successfully used for the first analyses of theTara Ocean data [5].These reference-free methods share the use of k-mers as the fundamental unit used for comparing samples.Actually, k-mers are a natural unit for comparing communities: (1) sufficiently long k-mers are usually specificof a genome [6], (2) k-mer frequency is linearly related to genome’s abundance [7], (3) k-mer aggregates or-ganisms with very similar k-mer composition (e.g. related strains from the same bacterial species) without needfor a classification of those organisms [8]. However, even if Compareads approach was designed to scale-upto large metagenomic read sets, its use on data generated by large scale projects is turning into a bottleneckin terms of time requirements. By contrast, Mash outperforms by far all other methods in terms of computa-tional resource usage. However, this frugality comes at the expense of result quality and precision: the outputdistances and Jaccard indexes do not take into account relative abundance information and are not computedexactly due to k-mer sub-sampling. This is what motivated this work in which we propose a new de novo com-parative metagenomic method, called Simka. Simka compares N metagenomic datasets based on their exactk-mers counts. It computes a large collection of distances classically used in ecology to compare communities,by replacing species counts by k-mer counts, for a large range of k-mer sizes, including large ones (up to 30).Simka is, to our knowledge, the first method able to rapidly compute a full range of distances enabling the com-parison of any number of datasets. Simka outperforms state-of-the-art read comparison methods in terms ofcomputational needs and result quality. For instance, Simka ran on 690 samples from the Human MicrobiomeProject (HMP) (totalling 32 billion reads) in less than 10 hours and using no more than 70 GB RAM.Simka works as follows. Firstly, the k-mer spectrum of each dataset is computed. The k-mer spectrum ofa dataset is the set of all its distinct k-mers associated with their abundance in the dataset. Secondly, k-merspectra are compared in a pairwise manner to compute their distance. This comparison process basically aimsat identifying which k-mers are shared by both spectra and which ones are not. It can be computationally veryexpensive because each k-mer spectrum can contains millions to billions of distinct k-mers when k is large(> 15). Moreover, the number of comparisons grows quadratically with the number of input datasets. Totackle this issue, we have designed a new k-mer counting strategy of numerous datasets, called Multiset k-merCounting (MKC). MKC takes N datasets as input and provides an abundance vector for each distinct k-mer.The abundance vector of a k-mer consists of its N counts in the N datasets. The abundance vector generationby the MKC task is divided into two phases: (1) Sorting Count, (2) Merging Count. During the first step, thek-mers of each dataset are counted independently. This is performed by sorting the k-mers in lexicographicalorder. Distinct k-mers can thus be identified and their number of occurrences computed. This task can be veryefficiently performed by popular disk-based k-mer counting tool such as DSK [9] or KMC2 [10]. The resultingk-mer spectra are written on the disk. During the second step, a Merge-Sort algorithm can be efficientlyArticle3Version preprintComment citer ce document :Benoit, G. (Auteur de correspondance), Peterlongo, P., Mariadassou, M., Drezen, E., Schbath,S., Lavenier, D., Lemaitre, C. (2017). Simka: large scale de novo comparative metagenomics. In:JOBIM 2017 LILLE (p. 3-5).  Presented at JOBIM 2017 - Journées Ouvertes Biologie InformatiqueMathématiques, Lille, FRA (2017-07-03 - 2017-07-06).applied on the sorted k-mer spectra to directly generate abundance vectors. Given those abundance vectors, thedistances between each pair of datasets can be computed simultaneously. Interestingly, most of the ecologicaldistances are additive over the distinct k-mers, meaning that they can be iteratively updated one abundancevector at a time. Once an abundance vector has been processed, there is thus no need to keep it on record,allowing Simka to have a very low memory footprint.One advantage of the overall Simka workflow is its high parallelism potential. During the sorting countphase of the MKC, a first parallelism level is given by the independent counts of each dataset. N processescan thus be run in parallel, each one dealing with a specific dataset. A second level is given by the fine grainedparallelism implemented in software such as DSK or KMC2 that intensively exploit today multicore processorcapabilities. As the number of distinct k-mers is generally huge, those tools separate the k-mers in P smallerdisjoint sets that can be counted independently and thus result in P k-mer spectrum chunks per dataset. Duringits second step, the MKC exploit this partitionning to merge up to P k-mer spectrums chucks in parallel. Eachof these merge processes generates abundance vectors from which independent contribution to the distancesare computed. Since the distance computed by Simka are additive over the distinct k-mers, each contributionis simply accumulated and the final distance is computed. Simka implementation is based on the GATB library[11], a C++ library optimized to handle very large sets of k-mers. Simka is usable on standard computers andhas also been entirely parallelized for grid infrastructures made of hundred of nodes, and where each nodeimplements 8 or 16-core systems.The quality of the distances computed by Simka were evaluated answering two questions. First, are theysimilar to distances between read sets computed using other de novo approaches? Second, do they recover theknown biological structure of HMP samples? For the first evaluation, we show that Simka result are perfectlywell correlated with Compareads results. We go further in this evaluation by showing that Simka results arehighly correlated with costly but extremely accurate de novo comparison techniques relying on all-versus-allsequence alignment strategy. For the second evaluation, Simka distances were compared to taxonomic distancesthat are a traditional way of comparing metagenomic samples. Taxonomic distances are based on sequenceassignation to taxons by mapping to reference databases. To compare Simka to such traditional reference-based methods, we used the HMP dataset. One advantage of this dataset is that it has been extensively studied,in particular the microbial communities are relatively well represented in reference databases [1,12]. We showthat substituting k-mer counts by species counts gives admittedly different distances but that those distances arebiologically relevant as they capture the same underlying biological structure and lead to the same conclusionsas those based on taxonomic composition. In particular, Simka was able to retrieve two major biological results.The first one is the segregation of the HMP datasets by body sites. The second one reveals that the organisationof the gut samples is mainly driven by the relative abundances of three bacterial genera, known as enterotypes,and characterized by the relative abundances of a few genera: Bacteroides, Prevotella and genera from theRuminococcaceae family. In contrast of Simka, Mash performed badly when considering HMP datasets perbody site since this tool can only take into account presence/absence information and not relative abundances.As a matter of fact, differences in relative abundances are subtler signals that are often at the heart of interestingbiological insights in comparative genomics studies [13,14,15,16,17].We introduced Simka, a new method for computing a collection of ecological distances, between manylarge metagenomic datasets, based on their k-mer composition. This was made possible thanks to the Multisetk-mer Counting algorithm (MKC), a new strategy that counts k-mers of numerous datasets with state-of-the-arttime, memory and disk performances.AcknowledgementsThis work was supported by the French ANR-14-CE23-0001 Hydrogen Project.References[1] Human Microbiome Project Consortium. Structure, function and diversity of the healthy human microbiome. Na-ture, 486(7402):207–214, 2012.[2] Eric Karsenti, Silvia G. Acinas, Peer Bork, Chris Bowler, Colomban de Vargas, Jeroen Raes, Matthew Sullivan,Detlev Arendt, Francesca Benzoni, Jean Michel Claverie, Mick Follows, Gaby Gorsky, Pascal Hingamp, DanieleIudicone, Olivier Jaillon, Stefanie Kandels-Lewis, Uros Krzic, Fabrice Not, Hiroyuki Ogata, Stéphane Pesant, Em-Article Simka: large scale de novo comparative metagenomics4Version preprintComment citer ce document :Benoit, G. (Auteur de correspondance), Peterlongo, P., Mariadassou, M., Drezen, E., Schbath,S., Lavenier, D., Lemaitre, C. (2017). Simka: large scale de novo comparative metagenomics. In:JOBIM 2017 LILLE (p. 3-5).  Presented at JOBIM 2017 - Journées Ouvertes Biologie InformatiqueMathématiques, Lille, FRA (2017-07-03 - 2017-07-06).manuel Georges Reynaud, Christian Sardet, Michael E. Sieracki, Sabrina Speich, Didier Velayoudon, Jean Weis-senbach, and Patrick Wincker. A holistic approach to marine Eco-systems biology. PLoS Biology, 9, 2011.[3] Nicolas Maillet, Claire Lemaitre, Rayan Chikhi, Dominique Lavenier, and Pierre Peterlongo. Compareads: compar-ing huge metagenomic experiments. BMC bioinformatics, 13(Suppl 19):S10, 2012.[4] Brian D. Ondov, Todd J. Treangen, Páll Melsted, Adam B. Mallonee, Nicholas H. Bergman, Sergey Koren, andAdam M. Phillippy. Mash: fast genome and metagenome distance estimation using minhash. Genome Biol,17(1):132, 2016.[5] E. Villar, G. K. Farrant, M. Follows, L. Garczarek, S. Speich, S. Audic, L. Bittner, B. Blanke, J. R. Brum, C. Brunet,R. Casotti, A. Chase, J. R. Dolan, F. Ortenzio, J.-P. Gattuso, N. Grima, L. Guidi, C. N. Hill, O. Jahn, J.-L. Jamet,H. Le Goff, C. Lepoivre, S. Malviya, E. Pelletier, J.-B. Romagnan, S. Roux, S. Santini, E. Scalco, S. M. Schwenck,A. Tanaka, P. Testor, T. Vannier, F. Vincent, A. Zingone, C. Dimier, M. Picheral, S. Searson, S. Kandels-Lewis, S. G.Acinas, P. Bork, E. Boss, C. de Vargas, G. Gorsky, H. Ogata, S. Pesant, M. B. Sullivan, S. Sunagawa, P. Wincker,E. Karsenti, C. Bowler, F. Not, P. Hingamp, and D. Iudicone. Environmental characteristics of agulhas rings affectinterocean plankton transport. Science, 348(6237):1261447–1261447, may 2015.[6] Y. Fofanov, Y. Luo, C. Katili, J. Wang, Y. Belosludtsev, T. Powdrill, C. Belapurkar, V. Fofanov, T.-B. Li, S. Chu-makov, and B. M. Pettitt. How independent are the appearances of n-mers in different genomes? Bioinformatics,20(15):2421–2428, apr 2004.[7] Yu-Wei Wu and Yuzhen Ye. A novel abundance-based algorithm for binning metagenomic sequences using l-tuples.Journal of Computational Biology, 18(3):523–534, 2011.[8] Hanno Teeling, Jost Waldmann, Thierry Lombardot, Margarete Bauer, and Frank O Glöckner. Tetra: a web-serviceand a stand-alone program for the analysis and comparison of tetranucleotide usage patterns in dna sequences. BMCbioinformatics, 5(1):163, 2004.[9] Guillaume Rizk, Dominique Lavenier, and Rayan Chikhi. DSK: k-mer counting with very low memory usage.Bioinformatics, page btt020, 2013.[10] Sebastian Deorowicz, Marek Kokot, Szymon Grabowski, and Agnieszka Debudaj-Grabysz. KMC 2: Fast andresource-frugal k-mer counting. Bioinformatics, 31(10):1569–1576, 2015.[11] Erwan Drezen, Guillaume Rizk, Rayan Chikhi, Charles Deltel, Claire Lemaitre, Pierre Peterlongo, and DominiqueLavenier. GATB: Genome assembly & analysis tool box. Bioinformatics, 30(20):2959–2961, 2014.[12] Human Microbiome Project Consortium. A framework for human microbiome research. Nature, 486(7402):215–221, Jun 2012.[13] Sébastien Boutin, Simon Y. Graeber, Michael Weitnauer, Jessica Panitz, Mirjam Stahl, Diana Clausznitzer, LarsKaderali, Gisli Einarsson, Michael M. Tunney, J. Stuart Elborn, Marcus A. Mall, and Alexander H. Dalpke. Com-parison of microbiomes from different niches of upper and lower airways in children and adolescents with cysticfibrosis. PLoS ONE, 10(1):1–19, 01 2015.[14] A. Shade, S. E. Jones, J. G. Caporaso, J. Handelsman, R. Knight, N. Fierer, and J. A. Gilbert. Conditionally raretaxa disproportionately contribute to temporal changes in microbial diversity. mBio, 5(4):e01371–14–e01371–14,jul 2014.[15] S. Genitsaris, S. Monchy, E. Viscogliosi, T. Sime-Ngando, S. Ferreira, and U. Christaki. Seasonal variations ofmarine protist community structure based on taxon-specific traits using the eastern english channel as a modelcoastal system. FEMS Microbiology Ecology, 91(5):fiv034–fiv034, mar 2015.[16] Suzanne Coveley, Mostafa S Elshahed, and Noha H Youssef. Response of the rare biosphere to environmentalstressors in a highly diverse ecosystem (zodletone spring, ok, usa). PeerJ, 3:e1182, 2015.[17] V. Gomez-Alvarez, S. Pfaller, J. G. Pressman, D. G. Wahman, and R. P. Revetta. Resilience of microbial communi-ties in a simulated drinking water distribution system subjected to disturbances: role of conditionally rare taxa andpotential implications for antibiotic-resistant bacteria. Environ. Sci.: Water Res. Technol., 2(4):645–657, 2016.Article Simka: large scale de novo comparative metagenomics5

Simka: large scale de novo comparative metagenomics

HAL-Rennes 1

HAL Descartes

Hal-Diderot

https://hal.archives-ouvertes.fr/hal-01595071/file/jobim17-simka_%7BB1CA3899-8F9F-415B-937C-9CB82A7FECC6%7D.pdf

Simka: large scale de novo comparative metagenomics

Abstract

Similar works

Full text

Available Versions

INRIA a CCSD electronic archive server

HAL-Rennes 1

HAL Descartes

Hal-Diderot