15 research outputs found
Graph Edit Distance Reward: Learning to Edit Scene Graph
Scene Graph, as a vital tool to bridge the gap between language domain and
image domain, has been widely adopted in the cross-modality task like VQA. In
this paper, we propose a new method to edit the scene graph according to the
user instructions, which has never been explored. To be specific, in order to
learn editing scene graphs as the semantics given by texts, we propose a Graph
Edit Distance Reward, which is based on the Policy Gradient and Graph Matching
algorithm, to optimize neural symbolic model. In the context of text-editing
image retrieval, we validate the effectiveness of our method in CSS and CRIR
dataset. Besides, CRIR is a new synthetic dataset generated by us, which we
will publish it soon for future use.Comment: 14 pages, 6 figures, ECCV camera ready versio