Discriminator-based adversarial networks for knowledge graph completion

Knowledge graphs (KGs) inherently lack reasoning ability which limits their effectiveness for tasks such as question-answering and query expansion. KG embedding (KGE) is a predominant approach where proximity between relations and entities in the embedding space is used for reasoning over KGs. Most existing KGE approaches use structural information of triplets and disregard contextual information which could be crucial to learning long-term relations between entities. Moreover, KGE approaches mostly use discriminative models which require both positive and negative samples to learn a decision boundary. KGs, by contrast, contain only positive samples, necessitating that negative samples are generated by replacing the head/tail of predicates with randomly-chosen entities. They are thus usually irrational and easily discriminable from positive samples, which can prevent the learning of sufficiently robust classifiers. To address the shortcomings, we propose to learn contextualized KGE using pretrained adversarial networks. We assume multi-hop relational paths(mh-RPs) as textual sequences for competitively learning discriminator-based KGE against the negative mh-RP generator. We use a pre-trained ELECTRA model and feed it with relational paths. We employ a generator to corrupt randomly-chosen entities with plausible alternatives and a discriminator to predict whether an entity is corrupted or not. We perform experiments on multiple benchmark knowledge graphs and the results show that our proposed KG-ELECTRA model outperforms BERT in link prediction

Contrastive representation learning: a framework and review

Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper, we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead