Deep Semantic Matching with Foreground Detection and Cycle-Consistency

Establishing dense semantic correspondences between object instances remains a challenging problem due to background clutter, significant scale and pose differences, and large intra-class variations. In this paper, we address weakly supervised semantic matching based on a deep network where only image pairs without manual keypoint correspondence annotations are provided. To facilitate network training with this weaker form of supervision, we 1) explicitly estimate the foreground regions to suppress the effect of background clutter and 2) develop cycle-consistent losses to enforce the predicted transformations across multiple images to be geometrically plausible and consistent. We train the proposed model using the PF-PASCAL dataset and evaluate the performance on the PF-PASCAL, PF-WILLOW, and TSS datasets. Extensive experimental results show that the proposed approach performs favorably against the state-of-the-art methods.Comment: ACCV 2018. PAMI 2020 extension: arXiv:1906.0585

Benchmarking and Developing Novel Methods for G Protein-coupled Receptor Ligand Discovery

G protein-coupled receptors (GPCR) are integral membrane proteins mediating responses from extracellular effectors that regulate a diverse set of physiological functions. Consequently, GPCR are the targets of ~34% of current FDA-approved drugs.3 Although it is clear that GPCR are therapeutically significant, discovery of novel drugs for these receptors is often impeded by a lack of known ligands and/or experimentally determined structures for potential drug targets. However, computational techniques have provided paths to overcome these obstacles. As such, this work discusses the development and application of novel computational methods and workflows for GPCR ligand discovery. Chapter 1 provides an overview of current obstacles faced in GPCR ligand discovery and defines ligand- and structure-based computational methods of overcoming these obstacles. Furthermore, chapter 1 outlines methods of hit list generation and refinement and provides a GPCR ligand discovery workflow incorporating computational techniques. In chapter 2, a workflow for modeling GPCR structure incorporating template selection via local sequence similarity and refinement of the structurally variable extracellular loop 2 (ECL2) region is benchmarked. Overall, findings in chapter 2 support the use of local template homology modeling in combination with de novo ECL2 modeling in the presence of a ligand from the template crystal structure to generate GPCR models intended to study ligand binding interactions. Chapter 3 details a method of generating structure-based pharmacophore models via the random selection of functional group fragments placed with Multiple Copy Simultaneous Search (MCSS) that is benchmarked in the context of 8 GPCR targets. When pharmacophore model performance was assessed with enrichment factor (EF) and goodness-of-hit (GH) scoring metrics, pharmacophore models possessing the theoretical maximum EF value were produced in both resolved structures (8 of 8 cases) and homology models (7 of 8 cases). Lastly, chapter 4 details a method of structure-based pharmacophore model generation using MCSS that is applicable to targets with no known ligands. Additionally, a method of pharmacophore model selection via machine learning is discussed. Overall, the work in chapter 4 led to the development of pharmacophore models exhibiting high EF values that were able to be accurately selected with machine learning classifiers

Hidden semi-Markov models to segment reading phases from eye movements

Our objective is to analyze scanpaths acquired through participants achieving a reading task aiming at answering a binary question: Is the text related or not to some given target topic? We propose a data-driven method based on hidden semi-Markov chains to segment scanpaths into phases deduced from the model states, which are shown to represent different cognitive strategies: normal reading, fast reading, information search, and slow confirmation. These phases were confirmed using different external covariates, among which semantic information extracted from texts. Analyses highlighted some strong preference of specific participants for specific strategies and more globally, large individual variability in eye-movement characteristics, as accounted for by random effects. As a perspective, the possibility of improving reading models by accounting for possible heterogeneity sources during reading is discussed

Advanced Methods for Entity Linking in the Life Sciences

The amount of knowledge increases rapidly due to the increasing number of available data sources. However, the autonomy of data sources and the resulting heterogeneity prevent comprehensive data analysis and applications. Data integration aims to overcome heterogeneity by unifying different data sources and enriching unstructured data. The enrichment of data consists of different subtasks, amongst other the annotation process. The annotation process links document phrases to terms of a standardized vocabulary. Annotated documents enable effective retrieval methods, comparability of different documents, and comprehensive data analysis, such as finding adversarial drug effects based on patient data. A vocabulary allows the comparability using standardized terms. An ontology can also represent a vocabulary, whereas concepts, relationships, and logical constraints additionally define an ontology. The annotation process is applicable in different domains. Nevertheless, there is a difference between generic and specialized domains according to the annotation process. This thesis emphasizes the differences between the domains and addresses the identified challenges. The majority of annotation approaches focuses on the evaluation of general domains, such as Wikipedia. This thesis evaluates the developed annotation approaches with case report forms that are medical documents for examining clinical trials. The natural language provides different challenges, such as similar meanings using different phrases. The proposed annotation method, AnnoMap, considers the fuzziness of natural language. A further challenge is the reuse of verified annotations. Existing annotations represent knowledge that can be reused for further annotation processes. AnnoMap consists of a reuse strategy that utilizes verified annotations to link new documents to appropriate concepts. Due to the broad spectrum of areas in the biomedical domain, different tools exist. The tools perform differently regarding a particular domain. This thesis proposes a combination approach to unify results from different tools. The method utilizes existing tool results to build a classification model that can classify new annotations as correct or incorrect. The results show that the reuse and the machine learning-based combination improve the annotation quality compared to existing approaches focussing on the biomedical domain. A further part of data integration is entity resolution to build unified knowledge bases from different data sources. A data source consists of a set of records characterized by attributes. The goal of entity resolution is to identify records representing the same real-world entity. Many methods focus on linking data sources consisting of records being characterized by attributes. Nevertheless, only a few methods can handle graph-structured knowledge bases or consider temporal aspects. The temporal aspects are essential to identify the same entities over different time intervals since these aspects underlie certain conditions. Moreover, records can be related to other records so that a small graph structure exists for each record. These small graphs can be linked to each other if they represent the same. This thesis proposes an entity resolution approach for census data consisting of person records for different time intervals. The approach also considers the graph structure of persons given by family relationships. For achieving qualitative results, current methods apply machine-learning techniques to classify record pairs as the same entity. The classification task used a model that is generated by training data. In this case, the training data is a set of record pairs that are labeled as a duplicate or not. Nevertheless, the generation of training data is a time-consuming task so that active learning techniques are relevant for reducing the number of training examples. The entity resolution method for temporal graph-structured data shows an improvement compared to previous collective entity resolution approaches. The developed active learning approach achieves comparable results to supervised learning methods and outperforms other limited budget active learning methods. Besides the entity resolution approach, the thesis introduces the concept of evolution operators for communities. These operators can express the dynamics of communities and individuals. For instance, we can formulate that two communities merged or split over time. Moreover, the operators allow observing the history of individuals. Overall, the presented annotation approaches generate qualitative annotations for medical forms. The annotations enable comprehensive analysis across different data sources as well as accurate queries. The proposed entity resolution approaches improve existing ones so that they contribute to the generation of qualitative knowledge graphs and data analysis tasks