102,447 research outputs found
Comparing Methods to Extract the Knowledge from Neural Networks
Neural networks (NN) have been shown to be accurate classifiers in many domains. Unfortunately, the lack of NN’s explanatory capability of knowledge learned has somewhat limited their application. A stream of research has therefore developed focusing on knowledge extraction from within neural networks. The literature, unfortunately, lacks consensus on how best to extract knowledge from help neural networks. Additionally, there is a lack of empirical studies that compare existing algorithms on relevant performance measures. Therefore, this study attempts to help fill this gap by comparing two different approaches to extracting IF-THEN rules from feedforward NN. The results show a significant difference in the performance of the two algorithms depending on the structure of the dataset utilized
Interpreting Embedding Models of Knowledge Bases: A Pedagogical Approach
Knowledge bases are employed in a variety of applications from natural
language processing to semantic web search; alas, in practice their usefulness
is hurt by their incompleteness. Embedding models attain state-of-the-art
accuracy in knowledge base completion, but their predictions are notoriously
hard to interpret. In this paper, we adapt "pedagogical approaches" (from the
literature on neural networks) so as to interpret embedding models by
extracting weighted Horn rules from them. We show how pedagogical approaches
have to be adapted to take upon the large-scale relational aspects of knowledge
bases and show experimentally their strengths and weaknesses.Comment: presented at 2018 ICML Workshop on Human Interpretability in Machine
Learning (WHI 2018), Stockholm, Swede
Learning Horn Envelopes via Queries from Large Language Models
We investigate an approach for extracting knowledge from trained neural
networks based on Angluin's exact learning model with membership and
equivalence queries to an oracle. In this approach, the oracle is a trained
neural network. We consider Angluin's classical algorithm for learning Horn
theories and study the necessary changes to make it applicable to learn from
neural networks. In particular, we have to consider that trained neural
networks may not behave as Horn oracles, meaning that their underlying target
theory may not be Horn. We propose a new algorithm that aims at extracting the
"tightest Horn approximation" of the target theory and that is guaranteed to
terminate in exponential time (in the worst case) and in polynomial time if the
target has polynomially many non-Horn examples. To showcase the applicability
of the approach, we perform experiments on pre-trained language models and
extract rules that expose occupation-based gender biases.Comment: 35 pages, 2 figures; manuscript accepted for publication in the
International Journal of Approximate Reasoning (IJAR
Machine Learning Meets the Semantic Web
Remarkable progress in research has shown the efficiency of Knowledge Graphs (KGs) in extracting valuable external knowledge in various domains. A Knowledge Graph (KG) can illustrate high-order relations that connect two objects with one or multiple related attributes. The emerging Graph Neural Networks (GNN) can extract both object characteristics and relations from KGs. This paper presents how Machine Learning (ML) meets the Semantic Web and how KGs are related to Neural Networks and Deep Learning. The paper also highlights important aspects of this area of research, discussing open issues such as the bias hidden in KGs at different levels of graph representation
Clinical Assistant Diagnosis for Electronic Medical Record Based on Convolutional Neural Network
Automatically extracting useful information from electronic medical records
along with conducting disease diagnoses is a promising task for both clinical
decision support(CDS) and neural language processing(NLP). Most of the existing
systems are based on artificially constructed knowledge bases, and then
auxiliary diagnosis is done by rule matching. In this study, we present a
clinical intelligent decision approach based on Convolutional Neural
Networks(CNN), which can automatically extract high-level semantic information
of electronic medical records and then perform automatic diagnosis without
artificial construction of rules or knowledge bases. We use collected 18,590
copies of the real-world clinical electronic medical records to train and test
the proposed model. Experimental results show that the proposed model can
achieve 98.67\% accuracy and 96.02\% recall, which strongly supports that using
convolutional neural network to automatically learn high-level semantic
features of electronic medical records and then conduct assist diagnosis is
feasible and effective.Comment: 9 pages, 4 figures, Accepted by Scientific Report
Encapsulation of Soft Computing Approaches within Itemset Mining a A Survey
Data Mining discovers patterns and trends by extracting knowledge from large databases. Soft Computing techniques such as fuzzy logic, neural networks, genetic algorithms, rough sets, etc. aims to reveal the tolerance for imprecision and uncertainty for achieving tractability, robustness and low-cost solutions. Fuzzy Logic and Rough sets are suitable for handling different types of uncertainty. Neural networks provide good learning and generalization. Genetic algorithms provide efficient search algorithms for selecting a model, from mixed media data. Data mining refers to information extraction while soft computing is used for information processing. For effective knowledge discovery from large databases, both Soft Computing and Data Mining can be merged. Association rule mining (ARM) and Itemset mining focus on finding most frequent item sets and corresponding association rules, extracting rare itemsets including temporal and fuzzy concepts in discovered patterns. This survey paper explores the usage of soft computing approaches in itemset utility mining
ReOnto: A Neuro-Symbolic Approach for Biomedical Relation Extraction
Relation Extraction (RE) is the task of extracting semantic relationships
between entities in a sentence and aligning them to relations defined in a
vocabulary, which is generally in the form of a Knowledge Graph (KG) or an
ontology. Various approaches have been proposed so far to address this task.
However, applying these techniques to biomedical text often yields
unsatisfactory results because it is hard to infer relations directly from
sentences due to the nature of the biomedical relations. To address these
issues, we present a novel technique called ReOnto, that makes use of neuro
symbolic knowledge for the RE task. ReOnto employs a graph neural network to
acquire the sentence representation and leverages publicly accessible
ontologies as prior knowledge to identify the sentential relation between two
entities. The approach involves extracting the relation path between the two
entities from the ontology. We evaluate the effect of using symbolic knowledge
from ontologies with graph neural networks. Experimental results on two public
biomedical datasets, BioRel and ADE, show that our method outperforms all the
baselines (approximately by 3\%).Comment: Accepted in ECML 202
- …