7 research outputs found
Interpreting Embedding Models of Knowledge Bases: A Pedagogical Approach
Knowledge bases are employed in a variety of applications from natural
language processing to semantic web search; alas, in practice their usefulness
is hurt by their incompleteness. Embedding models attain state-of-the-art
accuracy in knowledge base completion, but their predictions are notoriously
hard to interpret. In this paper, we adapt "pedagogical approaches" (from the
literature on neural networks) so as to interpret embedding models by
extracting weighted Horn rules from them. We show how pedagogical approaches
have to be adapted to take upon the large-scale relational aspects of knowledge
bases and show experimentally their strengths and weaknesses.Comment: presented at 2018 ICML Workshop on Human Interpretability in Machine
Learning (WHI 2018), Stockholm, Swede
A Fully Attention-Based Information Retriever
Recurrent neural networks are now the state-of-the-art in natural language
processing because they can build rich contextual representations and process
texts of arbitrary length. However, recent developments on attention mechanisms
have equipped feedforward networks with similar capabilities, hence enabling
faster computations due to the increase in the number of operations that can be
parallelized. We explore this new type of architecture in the domain of
question-answering and propose a novel approach that we call Fully Attention
Based Information Retriever (FABIR). We show that FABIR achieves competitive
results in the Stanford Question Answering Dataset (SQuAD) while having fewer
parameters and being faster at both learning and inference than rival methods.Comment: Accepted for presentation at the International Joint Conference on
Neural Networks (IJCNN) 201
On Pruning for Score-Based Bayesian Network Structure Learning
Many algorithms for score-based Bayesian net-work structure learning (BNSL), in particularexact ones, take as input a collection of po-tentially optimal parent sets for each variablein the data. Constructing such collectionsnaively is computationally intensive since thenumber of parent sets grows exponentiallywith the number of variables. Thus, pruningtechniques are not only desirable but essen-tial. While good pruning rules exist for theBayesian Information Criterion (BIC), currentresults for the Bayesian Dirichlet equivalentuniform (BDeu) score reduce the search spacevery modestly, hampering the use of the (oftenpreferred) BDeu. We derive new non-trivialtheoretical upper bounds for the BDeu scorethat considerably improve on the state-of-the-art. Since the new bounds are mathematicallyproven to be tighter than previous ones andat little extra computational cost, they are apromising addition to BNSL methods
An experimental study of prior dependence in Bayesian network structure learning
The Bayesian Dirichlet equivalent uniform (BDeu) function is a popular score to evaluate the goodness of a Bayesian network structure given complete categorical data. Despite its interesting properties, such as likelihood equivalence, it does require a prior expressed via a user-defined parameter known as Equivalent Sample Size (ESS), which significantly affects the final structure. We study conditions to obtain prior independence in BDeu-based structure learning. We show in experiments that the amount of data needed to render the learning robust to different ESS values is prohibitively large, even in big data times