Transductive Learning with String Kernels for Cross-Domain Text Classification

For many text classification tasks, there is a major problem posed by the lack of labeled data in a target domain. Although classifiers for a target domain can be trained on labeled text data from a related source domain, the accuracy of such classifiers is usually lower in the cross-domain setting. Recently, string kernels have obtained state-of-the-art results in various text classification tasks such as native language identification or automatic essay scoring. Moreover, classifiers based on string kernels have been found to be robust to the distribution gap between different domains. In this paper, we formally describe an algorithm composed of two simple yet effective transductive learning approaches to further improve the results of string kernels in cross-domain settings. By adapting string kernels to the test set without using the ground-truth test labels, we report significantly better accuracy rates in cross-domain English polarity classification.Comment: Accepted at ICONIP 2018. arXiv admin note: substantial text overlap with arXiv:1808.0840

Graph Neural Networks for Natural Language Processing

By constructing graph-structured data from the input data, Graph Neural Network (GNN) enhances the performance of numerous Natural Language Processing (NLP) tasks. In this thesis, we mainly focus on two aspects of NLP: text classification and knowledge graph completion. TextGCN shows excellent performance in text classification by leveraging the graph structure of the entire corpus without using any external resources, especially under a limited labelled data setting. Two questions are explored: (1) Under the transductive semi-supervised setting, how to utilize the documents better and learn the complex relationship between nodes. (2) How to transform TextGCN into an inductive model and also reduce the time and space complexity? In detail, firstly, a comprehensive analysis was conducted on TextGCN and its variants. Secondly, we propose ME-GCN, a novel method for text classification that utilizes multi-dimensional edge features in a graph neural network (GNN) for the first time. It uses the corpus-trained word and document-based edge features for semi-supervised classification and has been shown to be effective through experiments on benchmark datasets under the limited labelled data setting. Thirdly, InducT-GCN, an inductive framework for GCN-based text classification that does not require additional resources is introduced. The framework introduces a novel approach to make transductive GCN-based text classification models inductive, improving performance and reducing time and space complexity. Most existing work for Temporal Knowledge Graph Completion (TKGC) overlooks the significance of explicit temporal information and fails to skip irrelevant snapshots based on the entity-related relation in the query. To address this, we introduced Re-Temp (Relation-Aware Temporal Representation Learning), a model that leverages explicit temporal embedding and a skip information flow after each timestamp to eliminate unnecessary information for prediction

Explicit Learning Curves for Transduction and Application to Clustering and Compression Algorithms

Inductive learning is based on inferring a general rule from a finite data set and using it to label new data. In transduction one attempts to solve the problem of using a labeled training set to label a set of unlabeled points, which are given to the learner prior to learning. Although transduction seems at the outset to be an easier task than induction, there have not been many provably useful algorithms for transduction. Moreover, the precise relation between induction and transduction has not yet been determined. The main theoretical developments related to transduction were presented by Vapnik more than twenty years ago. One of Vapnik's basic results is a rather tight error bound for transductive classification based on an exact computation of the hypergeometric tail. While tight, this bound is given implicitly via a computational routine. Our first contribution is a somewhat looser but explicit characterization of a slightly extended PAC-Bayesian version of Vapnik's transductive bound. This characterization is obtained using concentration inequalities for the tail of sums of random variables obtained by sampling without replacement. We then derive error bounds for compression schemes such as (transductive) support vector machines and for transduction algorithms based on clustering. The main observation used for deriving these new error bounds and algorithms is that the unlabeled test points, which in the transductive setting are known in advance, can be used in order to construct useful data dependent prior distributions over the hypothesis space

Distribution matching for transduction

Many transductive inference algorithms assume that distributions over training and test estimates should be related, e.g. by providing a large margin of separation on both sets. We use this idea to design a transduction algorithm which can be used without modification for classification, regression, and structured estimation. At its heart we exploit the fact that for a good learner the distributions over the outputs on training and test sets should match. This is a classical two-sample problem which can be solved efficiently in its most general form by using distance measures in Hilbert Space. It turns out that a number of existing heuristics can be viewed as special cases of our approach.