Co-Attentive Cross-Modal Deep Learning for Medical Evidence Synthesis and Decision Making

Modern medicine requires generalised approaches to the synthesis and integration of multimodal data, often at different biological scales, that can be applied to a variety of evidence structures, such as complex disease analyses and epidemiological models. However, current methods are either slow and expensive, or ineffective due to the inability to model the complex relationships between data modes which differ in scale and format. We address these issues by proposing a cross-modal deep learning architecture and co-attention mechanism to accurately model the relationships between the different data modes, while further reducing patient diagnosis time. Differentiating Parkinson's Disease (PD) patients from healthy patients forms the basis of the evaluation. The model outperforms the previous state-of-the-art unimodal analysis by 2.35%, while also being 53% more parameter efficient than the industry standard cross-modal model. Furthermore, the evaluation of the attention coefficients allows for qualitative insights to be obtained. Through the coupling with bioinformatics, a novel link between the interferon-gamma-mediated pathway, DNA methylation and PD was identified. We believe that our approach is general and could optimise the process of medical evidence synthesis and decision making in an actionable way

The resurgence of structure in deep neural networks

Machine learning with deep neural networks ("deep learning") allows for learning complex features directly from raw input data, completely eliminating hand-crafted, "hard-coded" feature extraction from the learning pipeline. This has lead to state-of-the-art performance being achieved across several---previously disconnected---problem domains, including computer vision, natural language processing, reinforcement learning and generative modelling. These success stories nearly universally go hand-in-hand with availability of immense quantities of labelled training examples ("big data") exhibiting simple grid-like structure (e.g. text or images), exploitable through convolutional or recurrent layers. This is due to the extremely large number of degrees-of-freedom in neural networks, leaving their generalisation ability vulnerable to effects such as overfitting. However, there remain many domains where extensive data gathering is not always appropriate, affordable, or even feasible. Furthermore, data is generally organised in more complicated kinds of structure---which most existing approaches would simply discard. Examples of such tasks are abundant in the biomedical space; with e.g. small numbers of subjects available for any given clinical study, or relationships between proteins specified via interaction networks. I hypothesise that, if deep learning is to reach its full potential in such environments, we need to reconsider "hard-coded" approaches---integrating assumptions about inherent structure in the input data directly into our architectures and learning algorithms, through structural inductive biases. In this dissertation, I directly validate this hypothesis by developing three structure-infused neural network architectures (operating on sparse multimodal and graph-structured data), and a structure-informed learning algorithm for graph neural networks, demonstrating significant outperformance of conventional baseline models and algorithms.The work depicted in this dissertation was in part supported by funding from the European Union's Horizon 2020 research and innovation programme PROPAG-AGEING under grant agreement No 634821

Progress and challenges for the machine learning-based design of fit-for-purpose monoclonal antibodies

Although the therapeutic efficacy and commercial success of monoclonal antibodies (mAbs) are tremendous, the design and discovery of new candidates remain a time and cost-intensive endeavor. In this regard, progress in the generation of data describing antigen binding and developability, computational methodology, and artificial intelligence may pave the way for a new era of in silico on-demand immunotherapeutics design and discovery. Here, we argue that the main necessary machine learning (ML) components for an in silico mAb sequence generator are: understanding of the rules of mAb-antigen binding, capacity to modularly combine mAb design parameters, and algorithms for unconstrained parameter-driven in silico mAb sequence synthesis. We review the current progress toward the realization of these necessary components and discuss the challenges that must be overcome to allow the on-demand ML-based discovery and design of fit-for-purpose mAb therapeutic candidates

지도 학습 기반 바이오패닝 클론 증폭 패턴 분석을 통한 항원 결합 반응성 예측

학위논문(박사) -- 서울대학교대학원 : 의과대학 의과학과, 2021.8. 정준호.Background: Monoclonal antibodies (mAbs) are produced by B cells and specifically binds to target antigens. Technical advances in molecular and cellular cloning made it possible to purify recombinant mAbs in a large scale, enhancing the multiple research area and potential for their clinical application. Since the importance of therapeutic mAbs is increasing, mAbs have become the predominant drug classes for various diseases over the past decades. During that time, immense technological advances have made the discovery and development of mAb therapeutics more efficient. Owing to advances in high-throughput methodology in genomic sequencing, phenotype screening, and computational data analysis, it is conceivable to generate the panel of antibodies with annotated characteristics without experiments. Thesis objective: This thesis aims to develop the next-generation antibody discovery methods utilizing high-throughput antibody repertoire sequencing and bioinformatics analysis. I developed novel methods for construction of in vitro display antibody library, and machine learning based antibody discovery. In chapter 3, I described a new method for generating immunoglobulin (Ig) gene repertoire, which minimizes the amplification bias originated from a large number of primers targeting diverse Ig germline genes. Universal primer-based amplification method was employed in generating Ig gene repertoire then validated by high-throughput antibody repertoire sequencing, in the aspect of clonal diversity and immune repertoire reproducibility. A result of this research work is published in ‘Journal of Immunological Methods (2021). doi: 10.1016/j.jim.2021. 113089’. In chapter 4, I described a novel machine learning based antibody discovery method. In conventional colony screening approach, it is impossible to identify antigen specific binders having low clonal abundance, or hindered by non-specific phage particles having antigen reactivity on p8 coat protein. To overcome the limitations, I applied the supervised learning algorithm on high-throughput sequencing data annotated with binding property and clonal frequency through bio-panning. NGS analysis was performed to generate large number of antibody sequences annotated with its’ clonal frequency at each selection round of the bio-panning. By using random forest (RF) algorithm, antigen reactive binders were predicted and validated with in vitro screening experiment. A result of this research work is published in ‘Experimental & Molecular Medicine (2017). doi:0.1038/emm.2017.22’ and ‘Biomolecule (2020). doi:10.3390/biom10030421’. Conclusion: By combining conventional antibody discovery techniques and high-throughput antibody repertoire sequencing, it was able to make advances in multiple attributes of the previous methodology. Multi-cycle amplification with Ig germline gene specific primers showed the high level of repertoire distortion, but could be improved by employing universal primer-based amplification method. RF model generates the large number of antigen reactive antibody sequences having various clonal enrichment pattern. This result offers the new insight in interpreting clonal enrichment process, frequency of antigen specific binder does not increase gradually but depends on the multiple selection rounds. Supervised learning-based method also provides the more diverse antigen specific clonotypes than conventional antibody discovery methods.연구의 배경: 단일 클론 항체 (monoclonal antibody, mAb) 는 B 세포에서 생산되어 표적 항원에 특이적으로 결합하는 폴리펩타이드 복합체 이다. 분자 및 세포 클로닝 기술의 발전으로 재조합 단일 클론 항체를 대용량으로 생산하는것이 가능해졌으며, 이를 바탕으로 다양한 연구 및 임상 분야에서의 활용이 확대되고 있다. 또한 치료용 항체를 효율적으로 발굴하고 개발하는 기술에 대한 비약적인 발전이 이루어졌다. 유전자 서열 분석, 표현형 스크리닝, 컴퓨팅 기반 분석법 분야에서 이루어진 고집적 방법론 (high-throughput methodology) 의 발전과 이의 응용을 통해, 비실험적 방법을 통해 항원 반응성 항체 패널을 생산하는것이 가능해졌다. 연구의 목표: 본 박사 학위 논문은 고집적 항체 레퍼토어 시퀀싱 (high-throughput antibody repertoire sequencing) 과 생물정보학 (bioinformatics) 기법을 활용하여 신규한 (novel) 차세대 항체 발굴법 (next-generation antibody discovery method) 을 개발하는것을 목표로 하고 있다. 본 연구를 통해 in vitro display 항체 라이브러리를 제작하기 위한 신규 프로토콜 및 기계 학습을 기반으로한 항체 발굴법을 개발 하였다. Chapter 3: 항체 레퍼토어를 증폭하는 과정에서, 다수의 생식세포 면역 글로불린 유전자 (germline immunoglobulin gene) 특이적 프라이머 사용에 의해 발생하는 증폭 편차 (amplification bias) 를 최소화 하는 방법론에 대해 기술하였다. 유니버셜 (universal) 프라이머를 사용한 다중 사이클 증폭 (multi-cycle amplification) 법이 사용되었으며, 고집적 항체 레퍼토어 시퀀싱을 통해, 클론 다양성 (clonal diversity) 및 면역 레퍼토어 재구성도 (immune repertoire reproducibility) 를 생물정보학적 기법으로 측정하여 신규 방법론에 대한 검증을 수행하였다. 본 연구의 연구결과는 다음의 학술지에 출판 되었다: Journal of Immunological Methods (2021). doi: 10.1016/j.jim.2021. 113089. Chapter 4: 기계 학습 기반의 항체 발굴법 개발에 대해 기술하였다. 전통적 콜로니 스크리닝 (colony screening) 방법에서는, 클론 빈도 (clonal abundance) 가 낮은 클론을 발굴 하거나 선택압 (selective pressure) 이 부여되는 과정에서, p8 표면 단백질의 비 특이적 항원 특이성을 제거할 수 없다. 이러한 제한점을 극복하기 위해서 항원 결합능 및 바이오패닝 에서의 클론 빈도가 측정 되어있는 고집적 항체 서열 데이터를 대상으로 지도 학습 알고리즘을 적용하였다. 랜덤 포레스트 (random forest, RF) 알고리즘을 적용하여 항원 특이적 항체 클론을 예측하였으며, 시험관 내 스크리닝을 통해 항원 특이성을 검증하였다. 본 연구의 연구 결과는 다음의 학술지에 출판되었다: 1) Experimental & Molecular Medicine (2017). doi:0.1038/emm.2017.22., 2) Biomolecule (2020). doi:10.3390/biom10030421. 결론: 전통적 항체 발굴 기술과 고집적 항체 레퍼토어 시퀀싱 기술을 융합함으로써, 기존 방법론의 다양한 한계점을 개선할 수 있었다. 면역 글로불린 생식세포 유전자 특이적 프라이머를 사용한 다중 사이클 증폭은 클론 빈도 및 다양성에 왜곡을 유도 하였으나, 유니버셜 프라이머를 사용한 증폭법을 통해 높은 효율로 레퍼토어 왜곡을 개선시킬 수 있음을 관찰할 수 있었다. RF 모델은 다양한 클론 증폭 패턴 (enrichment pattern) 을 가지는 항원 반응성 항체 서열을 생성하였다. 이를 통해 항원에 특이적으로 결합하는 클론이 단계적으로 증폭되는 것이 아니라 초기 및 후기의 다수의 선별 단계 (selection round) 에 의존함을 확인할 수 있었으며, 바이오패닝 에서의 클론 증폭에 대한 새로운 해석을 제시하였다. 또한 지도 학습을 기반으로 발굴 된 클론들에서, 전통적 콜로니 스크리닝 방법과 대비하여 더 높은 서열 다양성을 관찰할 수 있었다.1. Introduction 8 1.1. Antibody and immunoglobulin repertoire 8 1.2. Antibody therapeutics 16 1.3. Methodology: antibody discovery and engineering 21 2. Thesis objective 28 3. Establishment of minimally biased phage display library construction method for antibody discovery 29 3.1. Abstract 29 3.2. Introduction 30 3.3. Results 32 3.4. Discussion 44 3.5. Methods 47 4. In silico identification of target specific antibodies by high-throughput antibody repertoire sequencing and machine learning 58 4.1. Abstract 58 4.2. Introduction 60 4.3. Results 64 4.4. Discussion 111 4.5. Methods 116 5. Future perspectives 129 6. References 135 7. Abstract in Korean 150박

A Multi-Resolution Graph Convolution Network for Contiguous Epitope Prediction

Computational methods for predicting binding interfaces between antigens and antibodies (epitopes and paratopes) are faster and cheaper than traditional experimental structure determination methods. A sufficiently reliable computational predictor that could scale to large sets of available antibody sequence data could thus inform and expedite many biomedical pursuits, such as better understanding immune responses to vaccination and natural infection and developing better drugs and vaccines. However, current state-of-the-art predictors produce discontiguous predictions, e.g., predicting the epitope in many different spots on an antigen, even though in reality they typically comprise a single localized region. We seek to produce contiguous predicted epitopes, accounting for long-range spatial relationships between residues. We therefore build a novel Graph Convolution Network (GCN) that performs graph convolutions at multiple resolutions so as to represent and constrain long-range spatial dependencies. In evaluation on a standard epitope prediction benchmark, we see a significant boost with the multi-resolution approach compared to a previous state-of-the-art GCN predictor, with half of the test cases increasing in AUC-PR by an average of 0.15 and the other half decreasing by only 0.05. We further introduce a clustering algorithm that takes advantage of the contiguity yielded by our model, grouping the raw predictions into a small set of discrete potential epitopes. We show that within the top 3 clusters, 73% of test cases contain a cluster covering most of the actual epitope, demonstrating the utility of contiguous predictions for guiding experimental methods by yielding a small set of reasonable hypotheses for further investigation

Computational Analysis of T Cell Receptor Repertoire and Structure

The human adaptive immune system has evolved to provide a sophisticated response to a vast body of pathogenic microbes and toxic substances. The primary mediators of this response are T and B lymphocytes. Antigenic peptides presented at the surface of infected cells by major histocompatibility complex (MHC) molecules are recognised by T cell receptors (TCRs) with exceptional specificity. This specificity arises from the enormous diversity in TCR sequence and structure generated through an imprecise process of somatic gene recombination that takes place during T cell development. Quantification of the TCR repertoire through the analysis of data produced by high-throughput RNA sequencing allows for a characterisation of the immune response to disease over time and between patients, and the development of methods for diagnosis and therapeutic design. The latest version of the software package Decombinator extracts and quantifies the TCR repertoire with improved accuracy and compatibility with complementary experimental protocols and external computational tools. The software has been extended for analysis of fragmented short-read data from single cells, comparing favourably with two alternative tools. The development of cell-based therapeutics and vaccines is incomplete without an understanding of molecular level interactions. The breadth of TCR diversity and cross-reactivity presents a barrier for comprehensive structural resolution of the repertoire by traditional means. Computational modelling of TCR structures and TCR-pMHC complexes provides an efficient alternative. Four generalpurpose protein-protein docking platforms were compared in their ability to accurately model TCR-pMHC complexes. Each platform was evaluated against an expanded benchmark of docking test cases and in the context of varying additional information about the binding interface. Continual innovation in structural modelling techniques sets the stage for novel automated tools for TCR design. A prototype platform has been developed, integrating structural modelling and an optimisation routine, to engineer desirable features into TCR and TCR-pMHC complex models