3 research outputs found
Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge
Multimodal Named Entity Recognition (MNER) on social media aims to enhance
textual entity prediction by incorporating image-based clues. Existing studies
mainly focus on maximizing the utilization of pertinent image information or
incorporating external knowledge from explicit knowledge bases. However, these
methods either neglect the necessity of providing the model with external
knowledge, or encounter issues of high redundancy in the retrieved knowledge.
In this paper, we present PGIM -- a two-stage framework that aims to leverage
ChatGPT as an implicit knowledge base and enable it to heuristically generate
auxiliary knowledge for more efficient entity prediction. Specifically, PGIM
contains a Multimodal Similar Example Awareness module that selects suitable
examples from a small number of predefined artificial samples. These examples
are then integrated into a formatted prompt template tailored to the MNER and
guide ChatGPT to generate auxiliary refined knowledge. Finally, the acquired
knowledge is integrated with the original text and fed into a downstream model
for further processing. Extensive experiments show that PGIM outperforms
state-of-the-art methods on two classic MNER datasets and exhibits a stronger
robustness and generalization capability.Comment: Accepted to Findings of EMNLP 202