69 research outputs found
Developments in the theory of randomized shortest paths with a comparison of graph node distances
There have lately been several suggestions for parametrized distances on a
graph that generalize the shortest path distance and the commute time or
resistance distance. The need for developing such distances has risen from the
observation that the above-mentioned common distances in many situations fail
to take into account the global structure of the graph. In this article, we
develop the theory of one family of graph node distances, known as the
randomized shortest path dissimilarity, which has its foundation in statistical
physics. We show that the randomized shortest path dissimilarity can be easily
computed in closed form for all pairs of nodes of a graph. Moreover, we come up
with a new definition of a distance measure that we call the free energy
distance. The free energy distance can be seen as an upgrade of the randomized
shortest path dissimilarity as it defines a metric, in addition to which it
satisfies the graph-geodetic property. The derivation and computation of the
free energy distance are also straightforward. We then make a comparison
between a set of generalized distances that interpolate between the shortest
path distance and the commute time, or resistance distance. This comparison
focuses on the applicability of the distances in graph node clustering and
classification. The comparison, in general, shows that the parametrized
distances perform well in the tasks. In particular, we see that the results
obtained with the free energy distance are among the best in all the
experiments.Comment: 30 pages, 4 figures, 3 table
Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach
Knowledge base completion (KBC) aims to predict missing information in a
knowledge base.In this paper, we address the out-of-knowledge-base (OOKB)
entity problem in KBC:how to answer queries concerning test entities not
observed at training time. Existing embedding-based KBC models assume that all
test entities are available at training time, making it unclear how to obtain
embeddings for new entities without costly retraining. To solve the OOKB entity
problem without retraining, we use graph neural networks (Graph-NNs) to compute
the embeddings of OOKB entities, exploiting the limited auxiliary knowledge
provided at test time.The experimental results show the effectiveness of our
proposed model in the OOKB setting.Additionally, in the standard KBC setting in
which OOKB entities are not involved, our model achieves state-of-the-art
performance on the WordNet dataset. The code and dataset are available at
https://github.com/takuo-h/GNN-for-OOKBComment: This paper has been accepted by IJCAI1
Ridge Regression, Hubness, and Zero-Shot Learning
This paper discusses the effect of hubness in zero-shot learning, when ridge
regression is used to find a mapping between the example space to the label
space. Contrary to the existing approach, which attempts to find a mapping from
the example space to the label space, we show that mapping labels into the
example space is desirable to suppress the emergence of hubs in the subsequent
nearest neighbor search step. Assuming a simple data model, we prove that the
proposed approach indeed reduces hubness. This was verified empirically on the
tasks of bilingual lexicon extraction and image labeling: hubness was reduced
with both of these tasks and the accuracy was improved accordingly.Comment: To be presented at ECML/PKDD 201
Data-dependent Learning of Symmetric/Antisymmetric Relations for Knowledge Base Completion
Embedding-based methods for knowledge base completion (KBC) learn
representations of entities and relations in a vector space, along with the
scoring function to estimate the likelihood of relations between entities. The
learnable class of scoring functions is designed to be expressive enough to
cover a variety of real-world relations, but this expressive comes at the cost
of an increased number of parameters. In particular, parameters in these
methods are superfluous for relations that are either symmetric or
antisymmetric. To mitigate this problem, we propose a new L1 regularizer for
Complex Embeddings, which is one of the state-of-the-art embedding-based
methods for KBC. This regularizer promotes symmetry or antisymmetry of the
scoring function on a relation-by-relation basis, in accordance with the
observed data. Our empirical evaluation shows that the proposed method
outperforms the original Complex Embeddings and other baseline methods on the
FB15k dataset.Comment: In AAAI 201
Learning Decorrelated Representations Efficiently Using Fast Fourier Transform
Barlow Twins and VICReg are self-supervised representation learning models
that use regularizers to decorrelate features. Although these models are as
effective as conventional representation learning models, their training can be
computationally demanding if the dimension d of the projected embeddings is
high. As the regularizers are defined in terms of individual elements of a
cross-correlation or covariance matrix, computing the loss for n samples takes
O(n d^2) time. In this paper, we propose a relaxed decorrelating regularizer
that can be computed in O(n d log d) time by Fast Fourier Transform. We also
propose an inexpensive technique to mitigate undesirable local minima that
develop with the relaxation. The proposed regularizer exhibits accuracy
comparable to that of existing regularizers in downstream tasks, whereas their
training requires less memory and is faster for large d. The source code is
available.Comment: Accepted for CVPR 202
Action Class Relation Detection and Classification Across Multiple Video Datasets
The Meta Video Dataset (MetaVD) provides annotated relations between action
classes in major datasets for human action recognition in videos. Although
these annotated relations enable dataset augmentation, it is only applicable to
those covered by MetaVD. For an external dataset to enjoy the same benefit, the
relations between its action classes and those in MetaVD need to be determined.
To address this issue, we consider two new machine learning tasks: action class
relation detection and classification. We propose a unified model to predict
relations between action classes, using language and visual information
associated with classes. Experimental results show that (i) pre-trained recent
neural network models for texts and videos contribute to high predictive
performance, (ii) the relation prediction based on action label texts is more
accurate than based on videos, and (iii) a blending approach that combines
predictions by both modalities can further improve the predictive performance
in some cases.Comment: Accepted to Pattern Recognition Letters. 12 pages, 4 figure
Cross-Modal Perception in the Framework of Non-Riemannian Sensory Space
Though human sensations, such as the senses of hearing, sight, etc., are independent each other, the interference between two of them is sometimes observed, and is called cross-modal perception[1]. Hitherto we studied unimodal perception of visual sensation[2] and auditory sensation[3] respectively by differential geometry[4]. We interpreted the parallel alley and the distance alley as two geodesics under different conditions in a visual space, and depicted the trace of continuous vowel speech as the geodesics through phonemes on a vowel plane. In this work, cross-modal perception is similarly treated from the standpoint of non-Riemannian geometry, where each axis of a cross-modal sensory space represents unimodal sensation. The geometry allows us to treat asymmetric metric tensor and hence a non-Euclidean concept of anholonomic objects, representing unidirectional property of cross-modal perception. The McGurk effect in audiovisual perception[5] and ‘rubber hand’ illusion in visual tactile perception[6] can afford experimental evidence of torsion tensor. The origin of ‘bouncing balls’ illusion[7] is discussed from the standpoint of an audiovisual cross-modal sensory space in a qualitative manner
- …