163 research outputs found
Robust Visual Question Answering: Datasets, Methods, and Future Challenges
Visual question answering requires a system to provide an accurate natural
language answer given an image and a natural language question. However, it is
widely recognized that previous generic VQA methods often exhibit a tendency to
memorize biases present in the training data rather than learning proper
behaviors, such as grounding images before predicting answers. Therefore, these
methods usually achieve high in-distribution but poor out-of-distribution
performance. In recent years, various datasets and debiasing methods have been
proposed to evaluate and enhance the VQA robustness, respectively. This paper
provides the first comprehensive survey focused on this emerging fashion.
Specifically, we first provide an overview of the development process of
datasets from in-distribution and out-of-distribution perspectives. Then, we
examine the evaluation metrics employed by these datasets. Thirdly, we propose
a typology that presents the development process, similarities and differences,
robustness comparison, and technical features of existing debiasing methods.
Furthermore, we analyze and discuss the robustness of representative
vision-and-language pre-training models on VQA. Finally, through a thorough
review of the available literature and experimental analysis, we discuss the
key areas for future research from various viewpoints.Comment: IEEE TPAMI (Under Review
- …