6 research outputs found
Relationformer: A Unified Framework for Image-to-Graph Generation
A comprehensive representation of an image requires understanding objects and their mutual relationship, especially in image-to-graph generation, e.g., road network extraction, blood-vessel network extraction, or scene graph generation. Traditionally, image-to-graph generation is addressed with a two-stage approach consisting of object detection followed by a separate relation prediction, which prevents simultaneous object-relation interaction. This work proposes a unified one-stage transformer-based framework, namely Relationformer, that jointly predicts objects and their relations. We leverage direct set-based object prediction and incorporate the interaction among the objects to learn an object-relation representation jointly. In addition to existing [obj]-tokens, we propose a novel learnable token, namely [rln]-token. Together with [obj]-tokens, [rln]-token exploits local and global semantic reasoning in an image through a series of mutual associations. In combination with the pair-wise [obj]-token, the [rln]-token contributes to a computationally efficient relation prediction. We achieve state-of-the-art performance on multiple, diverse and multi-domain datasets that demonstrate our approach's effectiveness and generalizability
Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework
The heterogeneity in recently published knowledge graph embedding models'
implementations, training, and evaluation has made fair and thorough
comparisons difficult. In order to assess the reproducibility of previously
published results, we re-implemented and evaluated 21 interaction models in the
PyKEEN software package. Here, we outline which results could be reproduced
with their reported hyper-parameters, which could only be reproduced with
alternate hyper-parameters, and which could not be reproduced at all as well as
provide insight as to why this might be the case.
We then performed a large-scale benchmarking on four datasets with several
thousands of experiments and 24,804 GPU hours of computation time. We present
insights gained as to best practices, best configurations for each model, and
where improvements could be made over previously published best configurations.
Our results highlight that the combination of model architecture, training
approach, loss function, and the explicit modeling of inverse relations is
crucial for a model's performances, and not only determined by the model
architecture. We provide evidence that several architectures can obtain results
competitive to the state-of-the-art when configured carefully. We have made all
code, experimental configurations, results, and analyses that lead to our
interpretations available at https://github.com/pykeen/pykeen and
https://github.com/pykeen/benchmarkin