Chen, Dexiong

Jacob, Laurent

Mairal, Julien

Hal - Université Grenoble Alpes

Biological Sequence Modeling with Convolutional Kernel Networks

INRIA a CCSD electronic archive server

HAL Id: hal-02388776https://hal.archives-ouvertes.fr/hal-02388776Submitted on 3 Dec 2019HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Biological Sequence Modeling with ConvolutionalKernel NetworksDexiong Chen, Laurent Jacob, Julien MairalTo cite this version:Dexiong Chen, Laurent Jacob, Julien Mairal. Biological Sequence Modeling with Convolutional Ker-nel Networks. RECOMB 2019 - 23rd Annual International Conference Research in ComputationalMolecular Biology, May 2019, Washington DC, United States. pp.292-293, ￿10.1007/978-3-030-17083-7￿. ￿hal-02388776￿Biological Sequence Modeling withConvolutional Kernel NetworksDexiong Chen1, Laurent Jacob2, and Julien Mairal11 Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, 38000 Grenoble, France{fisrtname.lastname}@inria.fr2 Univ. Lyon, Université Lyon 1, CNRS, Laboratoire de Biométrie et BiologieÉvolutive UMR 5558, Lyon, Francelaurent.jacob@univ-lyon1.frUnderstanding the relationship between biological sequences and the asso-ciated phenotypes is a fundamental problem in molecular biology. Accordingly,machine learning techniques have been developed to exploit the growing num-ber of phenotypic sequences in automatic annotation tools. Typical applicationsinclude classifying protein domains into superfamilies [6, 9], predicting whethera DNA or RNA sequence binds to a protein [1], its splicing outcome [3], orits chromatin accessibility [4], predicting the resistance of a bacterial strain to adrug [2], or denoising a ChIP-seq signal [5]. Choosing how to represent biologicalsequences is a critical part of methods that predict phenotypes from genotypes.Kernel-based methods [6, 9, 8] have often been used for this task. They havebeen proven efficient to represent biological sequences in various tasks but onlyconstruct fixed representations and lack scalability to large amount of data. Bycontrast, convolutional neural networks (CNN) [1] have recently shown scalableand able to optimize data representations for specific tasks. However, they typ-ically lack interpretability and require large amounts of annotated data, whichmotivates us to introduce more data-efficient approaches.In this work we introduce CKN-seq, a strategy combining kernel methodsand deep neural networks for sequence modeling, by adapting the convolutionalkernel network (CKN) model originally developed for image data [7]. CKN-seqrelies on a convolutional kernel, a continuous relaxation of the mismatch kernel[6], and the Nyström approximation. The relaxation makes it possible to learnthe kernel from data, and we provide an unsupervised and a supervised algorithmto do so — the latter leading to a special case of CNNs.On a transcription factor binding prediction task and a protein remote homol-ogy detection task, both approaches show better performance than DeepBind,another existing CNN [1], especially when the amount of training data is small.On the other hand, the supervised algorithm produces task-specific and small-dimensional sequence representations while the unsupervised version dominatesall other methods on small-scale problems but leads to higher dimensional rep-resentations. Consequently, we introduce a hybrid approach which enjoys thebenefits of both supervised and unsupervised variants, namely the ability oflearning low-dimensional models with good prediction performance in all datasize regimes. Finally, the kernel point of view of our method provides us simpleways to visualize and interpret our models, and obtain sequence logos. On some2 D. Chen et al.simulated data, the logos given by CKN-seq are more informative and matchbetter with the ground truth in terms of any probabilistic distance measures.We provide a free implementation of CKN-seq for learning from biological se-quences, which can easily be adapted to other sequence prediction tasks and isavailable at https://gitlab.inria.fr/dchen/CKN-seq.The fact that CKNs retain the ability of CNNs to learn feature spaces fromlarge training sets of data while enjoying a reproducing kernel Hilbert spacestructure has other uncharted applications which we would like to explore infuture work. First, it will allow us to leverage the existing literature on kernelsfor biological sequences to define the bottom kernel instead of the mismatchkernel, possibly capturing other aspects than sequence motifs. More generally,it provides a straightforward way to build models for non-vector objects suchas graphs, taking as input molecules or protein structures. Finally, it paves theway for making deep networks amenable to statistical analysis, in particularto hypothesis testing. This important step would be complementary to the in-terpretability aspect, and necessary to make deep networks a powerful tool formolecular biology beyond prediction.A full version of the paper is available at https://doi.org/10.1101/217257.References1. Alipanahi, B., Delong, A., Weirauch, M.T., Frey, B.J.: Predicting the sequence speci-ficities of DNA-and RNA-binding proteins by deep learning. Nature biotechnology33(8), 831–838 (2015)2. Drouin, A., Giguère, S., Déraspe, M., Marchand, M., Tyers, M., Loo, V.G., Bour-gault, A.M., Laviolette, F., Corbeil, J.: Predictive computational phenotypingand biomarker discovery using reference-free genome comparisons. BMC Genomics17(1), 754 (2016)3. Jha, A., Gazzara, M.R., Barash, Y.: Integrative deep mod-els for alternative splicing. Bioinformatics 33(14), 274–282 (2017).https://doi.org/10.1093/bioinformatics/btx2684. Kelley, D.R., Snoek, J., Rinn, J.L.: Basset: learning the regulatory code of the ac-cessible genome with deep convolutional neural networks. Genome Research 26(7),990–999 (2016)5. Koh, P.W., Pierson, E., Kundaje, A.: Denoising genome-wide histone chip-seq withconvolutional neural networks. Bioinformatics 33(14), i225–i233 (2017)6. Leslie, C.S., Eskin, E., Cohen, A., Weston, J., Noble, W.S.: Mismatch string kernelsfor discriminative protein classification. Bioinformatics 20(4), 467–476 (2004)7. Mairal, J.: End-to-end kernel learning with supervised convolutional kernel net-works. In: Advances in Neural Information Processing Systems (NIPS). pp. 1399–1407 (2016)8. Rangwala, H., Karypis, G.: Profile-based direct kernels for remote homology detec-tion and fold recognition. Bioinformatics 21(23), 4239–4247 (2005)9. Saigo, H., Vert, J.P., Ueda, N., Akutsu, T.: Protein homology detection using stringalignment kernels. Bioinformatics 20(11), 1682–1689 (2004)

https://hal.archives-ouvertes.fr/hal-02388776/file/recomb.pdf

Biological Sequence Modeling with Convolutional Kernel Networks

Abstract

Similar works

Full text

Available Versions

Hal - Université Grenoble Alpes

INRIA a CCSD electronic archive server