CORE
🇺🇦
make metadata, not war
Services
Services overview
Explore all CORE services
Access to raw data
API
Dataset
FastSync
Content discovery
Recommender
Discovery
OAI identifiers
OAI Resolver
Managing content
Dashboard
Bespoke contracts
Consultancy services
Support us
Support us
Membership
Sponsorship
Community governance
Advisory Board
Board of supporters
Research network
About
About us
Our mission
Team
Blog
FAQs
Contact us
Leveraging Past References for Robust Language Grounding
Authors
Michael Noseworthy
Daehyung Park
+3 more
Rohan Paul
Nicholas Roy
Subhro Roy
Publication date
3 May 2021
Publisher
'Association for Computational Linguistics (ACL)'
Doi
Cite
Abstract
© 2019 Association for Computational Linguistics. Grounding referring expressions to objects in an environment has traditionally been considered a one-off, ahistorical task. However, in realistic applications of grounding, multiple users will repeatedly refer to the same set of objects. As a result, past referring expressions for objects can provide strong signals for grounding subsequent referring expressions. We therefore reframe the grounding problem from the perspective of coreference detection and propose a neural network that detects when two expressions are referring to the same object. The network combines information from vision and past referring expressions to resolve which object is being referred to. Our experiments show that detecting referring expression coreference is an effective way to ground objects described by subtle visual properties, which standard visual grounding models have difficulty capturing. We also show the ability to detect object coreference allows the grounding model to perform well even when it encounters object categories not seen in the training data
Similar works
Full text
Available Versions
DSpace@MIT
See this paper in CORE
Go to the repository landing page
Download from data provider
oai:dspace.mit.edu:1721.1/1373...
Last time updated on 19/12/2021