Deep Learning -- A first Meta-Survey of selected Reviews across Scientific Disciplines, their Commonalities, Challenges and Research Impact

Deep learning belongs to the field of artificial intelligence, where machines perform tasks that typically require some kind of human intelligence. Similar to the basic structure of a brain, a deep learning algorithm consists of an artificial neural network, which resembles the biological brain structure. Mimicking the learning process of humans with their senses, deep learning networks are fed with (sensory) data, like texts, images, videos or sounds. These networks outperform the state-of-the-art methods in different tasks and, because of this, the whole field saw an exponential growth during the last years. This growth resulted in way over 10,000 publications per year in the last years. For example, the search engine PubMed alone, which covers only a sub-set of all publications in the medical field, provides already over 11,000 results in Q3 2020 for the search term 'deep learning', and around 90% of these results are from the last three years. Consequently, a complete overview over the field of deep learning is already impossible to obtain and, in the near future, it will potentially become difficult to obtain an overview over a subfield. However, there are several review articles about deep learning, which are focused on specific scientific fields or applications, for example deep learning advances in computer vision or in specific tasks like object detection. With these surveys as a foundation, the aim of this contribution is to provide a first high-level, categorized meta-survey of selected reviews on deep learning across different scientific disciplines. The categories (computer vision, language processing, medical informatics and additional works) have been chosen according to the underlying data sources (image, language, medical, mixed). In addition, we review the common architectures, methods, pros, cons, evaluations, challenges and future directions for every sub-category.Comment: 83 pages, 22 figures, 9 tables, 100 reference

Using ODIN for a PharmGKB revalidation experiment

The need for efficient text-mining tools that support curation of the biomedical literature is ever increasing. In this article, we describe an experiment aimed at verifying whether a text-mining tool capable of extracting meaningful relationships among domain entities can be successfully integrated into the curation workflow of a major biological database. We evaluate in particular (i) the usability of the system's interface, as perceived by users, and (ii) the correlation of the ranking of interactions, as provided by the text-mining system, with the choices of the curators

Using Neural Networks for Relation Extraction from Biomedical Literature

Using different sources of information to support automated extracting of relations between biomedical concepts contributes to the development of our understanding of biological systems. The primary comprehensive source of these relations is biomedical literature. Several relation extraction approaches have been proposed to identify relations between concepts in biomedical literature, namely, using neural networks algorithms. The use of multichannel architectures composed of multiple data representations, as in deep neural networks, is leading to state-of-the-art results. The right combination of data representations can eventually lead us to even higher evaluation scores in relation extraction tasks. Thus, biomedical ontologies play a fundamental role by providing semantic and ancestry information about an entity. The incorporation of biomedical ontologies has already been proved to enhance previous state-of-the-art results.Comment: Artificial Neural Networks book (Springer) - Chapter 1

Privacy in the Genomic Era

Genome sequencing technology has advanced at a rapid pace and it is now possible to generate highly-detailed genotypes inexpensively. The collection and analysis of such data has the potential to support various applications, including personalized medical services. While the benefits of the genomics revolution are trumpeted by the biomedical community, the increased availability of such data has major implications for personal privacy; notably because the genome has certain essential features, which include (but are not limited to) (i) an association with traits and certain diseases, (ii) identification capability (e.g., forensics), and (iii) revelation of family relationships. Moreover, direct-to-consumer DNA testing increases the likelihood that genome data will be made available in less regulated environments, such as the Internet and for-profit companies. The problem of genome data privacy thus resides at the crossroads of computer science, medicine, and public policy. While the computer scientists have addressed data privacy for various data types, there has been less attention dedicated to genomic data. Thus, the goal of this paper is to provide a systematization of knowledge for the computer science community. In doing so, we address some of the (sometimes erroneous) beliefs of this field and we report on a survey we conducted about genome data privacy with biomedical specialists. Then, after characterizing the genome privacy problem, we review the state-of-the-art regarding privacy attacks on genomic data and strategies for mitigating such attacks, as well as contextualizing these attacks from the perspective of medicine and public policy. This paper concludes with an enumeration of the challenges for genome data privacy and presents a framework to systematize the analysis of threats and the design of countermeasures as the field moves forward

생물학적 사전 지식을 활용한 고차원의 다중 오믹스 관계를 찾는 컴퓨터 공학적 접근 방법

학위논문(박사) -- 서울대학교대학원 : 공과대학 컴퓨터공학부, 2021.8. 김선.세포가 어떻게 기능하고 외부 자극에 반응하는지 이해하는 것은 생물학, 의학에서 가장 중요한 관심사 중 하나이다. 기술의 발전으로 과학자들은 단일 생물학적 실험으로 세포의 변화요인들을 쉽게 측정할 수 있게 되었다. 주목할만한 예시로 게놈 시퀀싱, 유전자 발현량 측정, 유전자 발현을 조절하는 후성 유전체 측정 같은 다중 오믹스 데이터가 있다. 세포의 상태를 더 자세히 이해하기 위해서 다중 오믹스 조절자와 유전자 사이의 조절 관계를 알아내는 것이 중요하다. 하지만 다중 오믹스 조절 관계는 매우 복잡하고 모든 세포 상태 특이적인 관계를 실험적으로 검증하는 것은 불가능하다. 따라서, 서로 다른 유형의 고차원 오믹스 데이터로부터 관계를 예측하기 위한 효율적인 컴퓨터 공학적 접근방법이 요구된다. 이러한 고차원 데이터를 처리하는 한 가지 방법은 다양한 데이터베이스에서 선별된 유전자의 기능과 오믹스 간의 관계와 같은 외부 생물학적 지식을 통합하여 활용하는 것이다. 본 박사학위 논문은 생물학적 사전 지식을 활용하여 다중 오믹스 데이터로부터 유전자의 발현을 조절하는 관계를 예측하기 위한 세 가지 컴퓨터 공학적인 접근법을 제안하였다. 첫 번째는 마이크로 알엔에이와 유전자의 일대다 관계를 예측하기 위한 기법이다. 마이크로 알엔에이 표적 예측 문제는 가능한 표적 유전자의 개수가 너무 많으며 거짓 양성과 거짓음성의 비율을 조절해야 하는 문제가 있다. 이러한 문제를 해결하기 위해 마이크로 알엔에이-유전자와 데이터의 맥락 사이의 연관성을 문헌 지식을 활용하여 결정하고 마이크로 알엔에이-유전자 관계를 예측하기 위한 ContextMMIA를 개발하였다. ContextMMIA는 통계적 유의성과 문헌 관련성을 기반으로 마이크로 알엔에이-유전자 관계의 점수를 계산하여 관계의 우선순위를 결정한다. 예후가 다른 유방암 데이터에 대한 실험에서 ContextMMIA는 예후가 나쁜 유방암에서 활성화된 마이크로 알엔에이-유전자 관계를 예측하였고 기존 실험적으로 검증된 관계가 높은 우선순위로 예측되었으며 해당 유전자들이 유방암 관련 경로에 관여하는 것으로 알려졌다. 두 번째는 약물 반응을 일으키는 유전자의 다대일 조절 관계를 예측하기 위한 기법이다. 약물 반응 예측을 위해서 약물 반응 매개 유전자를 결정해야 하며 이를 위해 20,000개 유전자의 다중 오믹스 데이터를 통합 분석하는 방법이 필요하다. 이 문제를 해결하기 위해 저차원 임베딩 방법, 약물-유전자 연관성에 대한 문헌 지식 및 유전자-유전자 상호 작용 지식을 활용하여 약물 반응을 예측하기 위한 DRIM을 개발하였다. DRIM은 오토인코더, 텐서 분해, 약물-유전자 연관성을 이용하여 다중 오믹스 데이터에서 다대일 관계를 결정한다. 결정된 매개 유전자의 조절 관계를 유전자-유전자 상호 작용 지식과 약물 반응 시계열 유전자 발현 데이터의 상호 상관관계를 이용하여 결정한다. 유방암 세포주 데이터에 대한 실험에서 DRIM은 라파티닙이 표적으로 하는 PI3K-Akt 패스웨이에 관여하는 유전자들의 약물 반응 조절 관계를 예측하였고 라파티닙 반응성과 관련된 매개 유전자를 예측하였다. 그리고 예측된 조절 관계가 세포주 특이적인 패턴을 보이는 것을 확인하였다. 세 번째는 세포의 상태를 설명하는 조절자와 유전자의 다대다 조절 관계를 예측하기 위한 기법이다. 다대다 관계 예측을 위해 관찰된 유전자 발현 값과 유전자 조절 네트워크로부터 추정된 유전자 발현 값 사이의 차이를 측정하는 목적 함수를 만들었다. 목적 함수를 최소화하기 위하여 조절인자와 유전자의 수에 따라 기하급수적으로 증가하는 검색 공간을 탐색해야 한다. 이 문제를 해결하기 위해 조절자-유전자 상호 작용 지식을 활용하여 두 가지 연산을 반복하여 조절 관계를 찾는 최적화 기법을 개발하였다. 첫 번째 단계는 네트워크에 간선을 추가하기 위해 강화 학습 기반 휴리스틱을 통해 조절자를 선택하는 다대일 유전자 중심 관계를 탐색하는 단계이다. 두 번째 단계는 네트워크에서 간선을 제거하기 위해 유전자를 확률적으로 선택하는 일대다 조절자 중심 관계를 탐색하는 단계이다. 유방암 세포주 데이터에 대한 실험에서 제안된 방법은 이전의 최적화 방법보다 더 정확한 유전자 발현량 추정을 하였고 조절자 및 유전자 발현 데이터로 유방암 아형 특이적 네트워크를 구성하였다. 또한, 유방암 아형 관련 실험 검증된 조절 관계를 예측하였다. 요약하면, 본 박사학위 논문은 다중 오믹스 조절자와 유전자의 사이의 일대다, 다대일, 다대다 관계를 예측하기 위하여 생물학적 지식을 활용한 컴퓨터 공학적 접근법을 제안하였다. 제안된 방법은 증가하고 있는 분자 생물학 데이터를 분석하여 유전자 조절 상호 작용을 이해함으로써 세포 기능에 대한 심층적인 이해를 도와줄 수 있을 것으로 기대된다.Understanding how cells function or respond to external stimuli is one of the most important questions in biology and medicine. Thanks to the advances in instrumental technologies, scientists can routinely measure events within cells in single biological experiments. Notable examples are multi-omics data: sequencing of genomes, quantifications of gene expression, and identification of epigenetic events that regulate expression of genes. In order to better understand cellular mechanisms, it is essential to identify regulatory relationships between multi-omics regulators and genes. However, regulatory relationships are very complex and it is infeasible to validate all condition-specific relationships experimentally. Thus, there is an urgent need for an efficient computational method to extract relationships from different types of high-dimensional omics data. One way to address these high-dimensional data is to incorporate external biological knowledge such as relationships between omics and functions of genes curated in various databases. In my doctoral study, I developed three computational approaches to identify the regulatory relationships from multi-omics data utilizing biological prior knowledge. The first study proposes a method to predict one-to-m relationships between miRNA and genes. The computational challenge of miRNA target prediction is that there are many miRNA target candidates, and the ratio of false positives to false negatives needs to be adjusted. This challenge is addressed by utilizing literature knowledge for determining the association between miRNA-gene and a given context. In this study, I developed ContextMMIA to predict miRNA-gene relationships from miRNA and gene expression data. ContextMMIA computes scores of miRNA-gene relationships based on statistical significance and literature relevance and prioritizes the relationships based on the scores. In experiments on breast cancer data with different prognosis, ContextMMIA predicted differentially activated miRNA-gene relationships in invasive breast cancer. The experimentally verified miRNA-gene relationships were predicted with high priority and those genes are known to be involved in breast cancer-related pathways. The second study proposes a method to predict n-to-one relationships between regulators and gene on drug response. The computational challenge of drug response prediction is how to integrate multi-omics data of 20,000 genes for determining drug response mediator genes. This challenge is addressed by utilizing low-dimensional embedding methods, literature knowledge of drug-gene associations, and gene-gene interaction knowledge. For this problem, I developed DRIM to predict drug response relationships from the multi-omics data and drug-induced time-series gene expression data. DRIM uses autoencoder, tensor decomposition, and drug-gene association to determine n-to-one relationships from multi-omics data. Then, regulatory relationships of mediator genes are determined by gene-gene interaction knowledge and cross-correlation of drug-induced time-series gene expression data. In experiments on breast cancer cell line data, DRIM extracted mediator genes relevant to drug response and regulatory relationships of genes involved in the PI3K-Akt pathway targeted by lapatinib. In addition, DRIM revealed distinguished patterns of relationships in breast cancer cell lines with different lapatinib resistance. The third study proposes a method to predict n-to-m relationships between regulators and genes. In order to predict n-to-m relationships, this study formulated an objective function that measures the deviation between observed gene expression values and estimated gene expression values derived from gene regulatory networks. The computational challenge of minimizing the objective function is to navigate the search space of relationships exponentially increasing according to the number of regulators and genes. This challenge is addressed by the iterative local optimization with regulator-gene interaction knowledge. In this study, I developed a two-step iterative RL-based method to predict n-to-m relationships from regulator and gene expression data. The first step is to explore the n-to-one gene-oriented step that selects regulators by reinforcement learning based heuristic to add edges to the network. The second step is to explore the one-to-m regulator-oriented step that stochastically selects genes to remove edges from the network. In experiments on breast cancer cell line data, the proposed method constructed breast cancer subtype-specific networks from the regulator and gene expression profiles with a more accurate gene expression estimation than previous combinatorial optimization methods. Moreover, regulatory relationships involved in the networks were associated with breast cancer subtypes. In summary, in this thesis, I proposed computational methods for predicting one-to-m, n-to-one, and n-to-m relationships between multi-omics regulators and genes utilizing external domain knowledge. The proposed methods are expected to deepen our knowledge of cellular mechanisms by understanding gene regulatory interactions by analyzing the ever-increasing molecular biology data such as The Cancer Genome Atlas, Cancer Cell Line Encyclopedia.Chapter 1 Introduction 1 1.1 Biological background 1 1.1.1 Multi-omics analysis 1 1.1.2 Multi-omics relationships indicating cell state 2 1.1.3 Biological prior knowledge 4 1.2 Research problems for the multi-omics relationship 6 1.3 Computational challenges and approaches in the exploring multiomics relationship 6 1.4 Outline of the thesis 12 Chapter 2 Literature-based condition-specific miRNA-mRNA target prediction 13 2.1 Computational Problem & Evaluation criterion 14 2.2 Related works 15 2.3 Motivation 17 2.4 Methods 20 2.4.1 Identifying genes and miRNAs based on the user-provided context 22 2.4.2 Omics Score 23 2.4.3 Context Score 24 2.4.4 Confidence Score 26 2.5 Results 26 2.5.1 Pathway analysis 27 2.5.2 Reproducibility of validated targets in humans 31 2.5.3 Sensitivity tests when different keywords are used 33 2.6 Summary 34 Chapter 3 DRIM: A web-based system for investigating drug response at the molecular level by condition-specific multi-omics data integration 36 3.1 Computational Problem & Evaluation criterion 37 3.2 Related works 38 3.3 Motivation 42 3.4 Methods 44 3.4.1 Step 1: Input 45 3.4.2 Step 2: Identifying perturbed sub-pathway with time-series 45 3.4.3 Step 3: Embedding multi-omics for selecting potential mediator genes 47 3.4.4 Step 4: Construct TF-regulatory time-bounded network and identify regulatory path 52 3.4.5 Step 5: Analysis result on the web 52 3.5 Case study: Comparative analysis of breast cancer cell lines that have different sensitivity with lapatinib 54 3.5.1 Multi-omics analysis result before drug treatment 56 3.5.2 Time-series gene expression analysis after drug treatment 57 3.6 Summary 61 Chapter 4 Combinatorial modeling and optimization using iterative RL search for inferring sample-specific regulatory network 63 4.1 Computational Problem & Evaluation criterion 64 4.2 Related works 64 4.3 Motivation 66 4.4 Methods 68 4.4.1 Formulating an objective function 68 4.4.2 Overview of an iterative search method 70 4.4.3 G-step for exploring n-to-one gene-oriented relationship 73 4.4.4 R-step for exploring one-to-m regulator-oriented relationship 79 4.5 Results 80 4.5.1 Cancer cell line data 80 4.5.2 Hyperparameters 81 4.5.3 Quantitative evaluation 82 4.5.4 Qualitative evaluation 83 4.6 Summary 86 Chapter 5 Conclusions 88 국문초록 111박

Warfarin Dose Estimation on High-dimensional and Incomplete Data

Warfarin is a widely used oral anticoagulant worldwide. However, due to the complex relationship between individual factors, it is challenging to estimate the optimal warfarin dose to give full play to its ideal efficacy. Currently, there are plenty of studies using machine learning or deep learning techniques to help with the optimal warfarin dose selection. But few of them can resolve missing values and high-dimensional data naturally, that are two main concerns when analyzing clinical real world data. In this work, we propose to regard each patient’s record as a set of observed individual factors, and represent them in an embedding space, that enables our method can learn from the incomplete date directly and avoid the negative impact from the high-dimensional feature set. Then, a novel neural network is proposed to combine the set of embedded vectors non-linearly, that are capable of capturing their correlations and locating the informative ones for prediction. After comparing with the baseline models on the open source data from International Warfarin Pharmacogenetics Consortium, the experimental results demonstrate that our proposed method outperform others by a significant margin. After further analyzing the model performance in different dosing subgroups, we can conclude that the proposed method has the high application value in clinical, especially for the patients in high-dose and medium-dose subgroups