182,481 research outputs found
Knowledge Enhanced Graph Neural Networks
Graph data is omnipresent and has a large variety of applications such as
natural science, social networks or semantic web. Though rich in information,
graphs are often noisy and incomplete. Therefore, graph completion tasks such
as node classification or link prediction have gained attention. On the one
hand, neural methods such as graph neural networks have proven to be robust
tools for learning rich representations of noisy graphs. On the other hand,
symbolic methods enable exact reasoning on graphs. We propose KeGNN, a
neuro-symbolic framework for learning on graph data that combines both
paradigms and allows for the integration of prior knowledge into a graph neural
network model. In essence, KeGNN consists of a graph neural network as a base
on which knowledge enhancement layers are stacked with the objective of
refining predictions with respect to prior knowledge. We instantiate KeGNN in
conjunction with two standard graph neural networks: Graph Convolutional
Networks and Graph Attention Networks, and evaluate KeGNN on multiple benchmark
datasets for node classification
Deeper Text Understanding for IR with Contextual Neural Language Modeling
Neural networks provide new possibilities to automatically learn complex
language patterns and query-document relations. Neural IR models have achieved
promising results in learning query-document relevance patterns, but few
explorations have been done on understanding the text content of a query or a
document. This paper studies leveraging a recently-proposed contextual neural
language model, BERT, to provide deeper text understanding for IR. Experimental
results demonstrate that the contextual text representations from BERT are more
effective than traditional word embeddings. Compared to bag-of-words retrieval
models, the contextual language model can better leverage language structures,
bringing large improvements on queries written in natural languages. Combining
the text understanding ability with search knowledge leads to an enhanced
pre-trained BERT model that can benefit related search tasks where training
data are limited.Comment: In proceedings of SIGIR 201
Robust Minutiae Extractor: Integrating Deep Networks and Fingerprint Domain Knowledge
We propose a fully automatic minutiae extractor, called MinutiaeNet, based on
deep neural networks with compact feature representation for fast comparison of
minutiae sets. Specifically, first a network, called CoarseNet, estimates the
minutiae score map and minutiae orientation based on convolutional neural
network and fingerprint domain knowledge (enhanced image, orientation field,
and segmentation map). Subsequently, another network, called FineNet, refines
the candidate minutiae locations based on score map. We demonstrate the
effectiveness of using the fingerprint domain knowledge together with the deep
networks. Experimental results on both latent (NIST SD27) and plain (FVC 2004)
public domain fingerprint datasets provide comprehensive empirical support for
the merits of our method. Further, our method finds minutiae sets that are
better in terms of precision and recall in comparison with state-of-the-art on
these two datasets. Given the lack of annotated fingerprint datasets with
minutiae ground truth, the proposed approach to robust minutiae detection will
be useful to train network-based fingerprint matching algorithms as well as for
evaluating fingerprint individuality at scale. MinutiaeNet is implemented in
Tensorflow: https://github.com/luannd/MinutiaeNetComment: Accepted to International Conference on Biometrics (ICB 2018
Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
For recurrent neural networks trained on time series with target and
exogenous variables, in addition to accurate prediction, it is also desired to
provide interpretable insights into the data. In this paper, we explore the
structure of LSTM recurrent neural networks to learn variable-wise hidden
states, with the aim to capture different dynamics in multi-variable time
series and distinguish the contribution of variables to the prediction. With
these variable-wise hidden states, a mixture attention mechanism is proposed to
model the generative process of the target. Then we develop associated training
methods to jointly learn network parameters, variable and temporal importance
w.r.t the prediction of the target variable. Extensive experiments on real
datasets demonstrate enhanced prediction performance by capturing the dynamics
of different variables. Meanwhile, we evaluate the interpretation results both
qualitatively and quantitatively. It exhibits the prospect as an end-to-end
framework for both forecasting and knowledge extraction over multi-variable
data.Comment: Accepted to International Conference on Machine Learning (ICML), 201
Accelerating Deterministic and Stochastic Binarized Neural Networks on FPGAs Using OpenCL
Recent technological advances have proliferated the available computing
power, memory, and speed of modern Central Processing Units (CPUs), Graphics
Processing Units (GPUs), and Field Programmable Gate Arrays (FPGAs).
Consequently, the performance and complexity of Artificial Neural Networks
(ANNs) is burgeoning. While GPU accelerated Deep Neural Networks (DNNs)
currently offer state-of-the-art performance, they consume large amounts of
power. Training such networks on CPUs is inefficient, as data throughput and
parallel computation is limited. FPGAs are considered a suitable candidate for
performance critical, low power systems, e.g. the Internet of Things (IOT) edge
devices. Using the Xilinx SDAccel or Intel FPGA SDK for OpenCL development
environment, networks described using the high-level OpenCL framework can be
accelerated on heterogeneous platforms. Moreover, the resource utilization and
power consumption of DNNs can be further enhanced by utilizing regularization
techniques that binarize network weights. In this paper, we introduce, to the
best of our knowledge, the first FPGA-accelerated stochastically binarized DNN
implementations, and compare them to implementations accelerated using both
GPUs and FPGAs. Our developed networks are trained and benchmarked using the
popular MNIST and CIFAR-10 datasets, and achieve near state-of-the-art
performance, while offering a >16-fold improvement in power consumption,
compared to conventional GPU-accelerated networks. Both our FPGA-accelerated
determinsitic and stochastic BNNs reduce inference times on MNIST and CIFAR-10
by >9.89x and >9.91x, respectively.Comment: 4 pages, 3 figures, 1 tabl
Beyond Material Implication: An Empirical Study of Residuum in Knowledge Enhanced Neural Networks
openKnowledge Enchanced Neural Networks (KENN) is a neuro-symbolic architecture that exploits fuzzy logic for injecting prior knowledge, codified by propositional formulas, into a neural network. It works by adding a new layer at the end of a generic neural network that further elaborates the initial predictions accordingly to the knowledge.
In the existing KENN, according to material implication rule, a conditional statement is represented as a conjunctive normal form formula. The following work extends this interpretation of the implication by using the fuzzy logic's Residuum semantic and shows how it has been integrated into the original KENN architecture, while keeping it reproducible.
The Residuum integration allowed to evaluate KENN on MNIST Addition, a task that couldn't be approached by the original architecture, and the results obtained were comparable to others state of the art neuro-symbolic methods.
The extended architecture has subsequently been evaluated also on visual relationships detection, showing that it could improve the performance of the original one.Knowledge Enchanced Neural Networks (KENN) is a neuro-symbolic architecture that exploits fuzzy logic for injecting prior knowledge, codified by propositional formulas, into a neural network. It works by adding a new layer at the end of a generic neural network that further elaborates the initial predictions accordingly to the knowledge.
In the existing KENN, according to material implication rule, a conditional statement is represented as a conjunctive normal form formula. The following work extends this interpretation of the implication by using the fuzzy logic's Residuum semantic and shows how it has been integrated into the original KENN architecture, while keeping it reproducible.
The Residuum integration allowed to evaluate KENN on MNIST Addition, a task that couldn't be approached by the original architecture, and the results obtained were comparable to others state of the art neuro-symbolic methods.
The extended architecture has subsequently been evaluated also on visual relationships detection, showing that it could improve the performance of the original one
Knowledge Graph Construction in Power Distribution Networks
In this paper, we propose a method for knowledge graph construction in power
distribution networks. This method leverages entity features, which involve
their semantic, phonetic, and syntactic characteristics, in both the knowledge
graph of distribution network and the dispatching texts. An enhanced model
based on Convolutional Neural Network, is utilized for effectively matching
dispatch text entities with those in the knowledge graph. The effectiveness of
this model is evaluated through experiments in real-world power distribution
dispatch scenarios. The results indicate that, compared with the baselines, the
proposed model excels in linking a variety of entity types, demonstrating high
overall accuracy in power distribution knowledge graph construction task
- …