Semantic Representation and Inference for NLP

Semantic representation and inference is essential for Natural Language Processing (NLP). The state of the art for semantic representation and inference is deep learning, and particularly Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and transformer Self-Attention models. This thesis investigates the use of deep learning for novel semantic representation and inference, and makes contributions in the following three areas: creating training data, improving semantic representations and extending inference learning. In terms of creating training data, we contribute the largest publicly available dataset of real-life factual claims for the purpose of automatic claim verification (MultiFC), and we present a novel inference model composed of multi-scale CNNs with different kernel sizes that learn from external sources to infer fact checking labels. In terms of improving semantic representations, we contribute a novel model that captures non-compositional semantic indicators. By definition, the meaning of a non-compositional phrase cannot be inferred from the individual meanings of its composing words (e.g., hot dog). Motivated by this, we operationalize the compositionality of a phrase contextually by enriching the phrase representation with external word embeddings and knowledge graphs. Finally, in terms of inference learning, we propose a series of novel deep learning architectures that improve inference by using syntactic dependencies, by ensembling role guided attention heads, incorporating gating layers, and concatenating multiple heads in novel and effective ways. This thesis consists of seven publications (five published and two under review).Comment: PhD thesis, the University of Copenhage

Enhanced Convolutional Neural Network for Non-Small Cell Lung Cancer Classification

Lung cancer is a common type of cancer that causes death if not detectedearly enough. Doctors use computed tomography (CT) images to diagnoselung cancer. The accuracy of the diagnosis relies highly on the doctor\u27sexpertise. Recently, clinical decision support systems based on deep learningvaluable recommendations to doctors in their diagnoses. In this paper, wepresent several deep learning models to detect non-small cell lung cancer inCT images and differentiate its main subtypes namely adenocarcinoma,large cell carcinoma, and squamous cell carcinoma. We adopted standardconvolutional neural networks (CNN), visual geometry group-16 (VGG16),and VGG19. Besides, we introduce a variant of the CNN that is augmentedwith convolutional block attention modules (CBAM). CBAM aims to extractinformative features by combining cross-channel and spatial information.We also propose variants of VGG16 and VGG19 that utilize a supportvector machine (SVM) at the classification layer instead of SoftMax. Wevalidated all models in this study through extensive experiments on a CTlung cancer dataset. Experimental results show that supplementing CNNwith CBAM leads to consistent improvements over vanilla CNN. Resultsalso show that the VGG variants that use the SVM classifier outperform theoriginal VGGs by a significant margin

A Survey on Knowledge Graphs: Representation, Acquisition and Applications

Human knowledge provides a formal understanding of the world. Knowledge graphs that represent structural relations between entities have become an increasingly popular research direction towards cognition and human-level intelligence. In this survey, we provide a comprehensive review of knowledge graph covering overall research topics about 1) knowledge graph representation learning, 2) knowledge acquisition and completion, 3) temporal knowledge graph, and 4) knowledge-aware applications, and summarize recent breakthroughs and perspective directions to facilitate future research. We propose a full-view categorization and new taxonomies on these topics. Knowledge graph embedding is organized from four aspects of representation space, scoring function, encoding models, and auxiliary information. For knowledge acquisition, especially knowledge graph completion, embedding methods, path inference, and logical rule reasoning, are reviewed. We further explore several emerging topics, including meta relational learning, commonsense reasoning, and temporal knowledge graphs. To facilitate future research on knowledge graphs, we also provide a curated collection of datasets and open-source libraries on different tasks. In the end, we have a thorough outlook on several promising research directions