Neural Graph Transfer Learning in Natural Language Processing Tasks

Natural language is essential in our daily lives as we rely on languages to communicate and exchange information. A fundamental goal for natural language processing (NLP) is to let the machine understand natural language to help or replace human experts to mine knowledge and complete tasks. Many NLP tasks deal with sequential data. For example, a sentence is considered as a sequence of works. Very recently, deep learning-based language models (i.e.,BERT \citep{devlin2018bert}) achieved significant improvement in many existing tasks, including text classification and natural language inference. However, not all tasks can be formulated using sequence models. Specifically, graph-structured data is also fundamental in NLP, including entity linking, entity classification, relation extraction, abstractive meaning representation, and knowledge graphs \citep{santoro2017simple,hamilton2017representation,kipf2016semi}. In this scenario, BERT-based pretrained models may not be suitable. Graph Convolutional Neural Network (GCN) \citep{kipf2016semi} is a deep neural network model designed for graphs. It has shown great potential in text classification, link prediction, question answering and so on. This dissertation presents novel graph models for NLP tasks, including text classification, prerequisite chain learning, and coreference resolution. We focus on different perspectives of graph convolutional network modeling: for text classification, a novel graph construction method is proposed which allows interpretability for the prediction; for prerequisite chain learning, we propose multiple aggregation functions that utilize neighbors for better information exchange; for coreference resolution, we study how graph pretraining can help when labeled data is limited. Moreover, an important branch is to apply pretrained language models for the mentioned tasks. So, this dissertation also focuses on the transfer learning method that generalizes pretrained models to other domains, including medical, cross-lingual, and web data. Finally, we propose a new task called unsupervised cross-domain prerequisite chain learning, and study novel graph-based methods to transfer knowledge over graphs

Phase-field modeling and simulation of fracture in brittle materials with strongly anisotropic surface energy

Crack propagation in brittle materials with anisotropic surface energy is important in applications involving single crystals, extruded polymers, or geological and organic materials. Furthermore, when this anisotropy is strong, the phenomenology of crack propagation becomes very rich, with forbidden crack propagation directions or complex sawtooth crack patterns. This problem interrogates fundamental issues in fracture mechanics, including the principles behind the selection of crack direction. Here, we propose a variational phase-field model for strongly anisotropic fracture, which resorts to the extended Cahn-Hilliard framework proposed in the context of crystal growth. Previous phase-field models for anisotropic fracture were formulated in a framework only allowing for weak anisotropy. We implement numerically our higher-order phase-field model with smooth local maximum entropy approximants in a direct Galerkin method. The numerical results exhibit all the features of strongly anisotropic fracture and reproduce strikingly well recent experimental observations.Peer ReviewedPostprint (author’s final draft

NNKGC: Improving Knowledge Graph Completion with Node Neighborhoods

Knowledge graph completion (KGC) aims to discover missing relations of query entities. Current text-based models utilize the entity name and description to infer the tail entity given the head entity and a certain relation. Existing approaches also consider the neighborhood of the head entity. However, these methods tend to model the neighborhood using a flat structure and are only restricted to 1-hop neighbors. In this work, we propose a node neighborhood-enhanced framework for knowledge graph completion. It models the head entity neighborhood from multiple hops using graph neural networks to enrich the head node information. Moreover, we introduce an additional edge link prediction task to improve KGC. Evaluation on two public datasets shows that this framework is simple yet effective. The case study also shows that the model is able to predict explainable predictions.Comment: DL4KG Workshop, ISWC 202

The Boolean power sequence of graphs

The adjacency matrix A of a graph G is a 0-1 matrix. The Boolean power sequence of A is convergent or periodic of period p = 2: The index y­ of A is the least integer m such that Am = Am+1 if A converges and the least integer m such that Am = Am+2 if A is periodic. In this paper we determine the index ­y of A if the graph G is bipartite. In the case of non-bipartite connected graphs, we give new lower and upper bounds for y , which are sharp.peer-reviewe

Method and apparatus for conducting structural health monitoring in a cryogenic, high vibration environment

Sensors affixed to various such structures, where the sensors can withstand, remain affixed, and operate while undergoing both cryogenic temperatures and high vibrations. In particular, piezoelectric single crystal transducers are utilized, and these sensors are coupled to the structure via a low temperature, heat cured epoxy. This allows the transducers to monitor the structure while the engine is operating, even despite the harsh operating conditions. Aspects of the invention thus allow for real time monitoring and analysis of structures that operate in conditions that previously did not permit such analysis. A further aspect of the invention relates to use of piezoelectric single crystal transducers. In particular, use of such transducers allows the same elements to be used as both sensors and actuators