CORE
🇺🇦
make metadata, not war
Services
Services overview
Explore all CORE services
Access to raw data
API
Dataset
FastSync
Content discovery
Recommender
Discovery
OAI identifiers
OAI Resolver
Managing content
Dashboard
Bespoke contracts
Consultancy services
Support us
Support us
Membership
Sponsorship
Community governance
Advisory Board
Board of supporters
Research network
About
About us
Our mission
Team
Blog
FAQs
Contact us
Multimodal news analytics using measures of cross-modal entity and context consistency
Authors
Sebastian Diering
Ralph Ewerth
+4 more
Sherzod Hakimov
Maximilian Idahl
Eric Müller-Budack
Jonas Theiner
Publication date
1 January 2021
Publisher
London : Springer
Doi
Cite
Abstract
The World Wide Web has become a popular source to gather information and news. Multimodal information, e.g., supplement text with photographs, is typically used to convey the news more effectively or to attract attention. The photographs can be decorative, depict additional details, but might also contain misleading information. The quantification of the cross-modal consistency of entity representations can assist human assessors’ evaluation of the overall multimodal message. In some cases such measures might give hints to detect fake news, which is an increasingly important topic in today’s society. In this paper, we present a multimodal approach to quantify the entity coherence between image and text in real-world news. Named entity linking is applied to extract persons, locations, and events from news texts. Several measures are suggested to calculate the cross-modal similarity of the entities in text and photograph by exploiting state-of-the-art computer vision approaches. In contrast to previous work, our system automatically acquires example data from the Web and is applicable to real-world news. Moreover, an approach that quantifies contextual image-text relations is introduced. The feasibility is demonstrated on two datasets that cover different languages, topics, and domains. © 2021, The Author(s)
Similar works
Full text
Open in the Core reader
Download PDF
Available Versions
Sustaining member
Repositorium für Naturwissenschaften und Technik
See this paper in CORE
Go to the repository landing page
Download from data provider
oai:oa.tib.eu:123456789/7782
Last time updated on 23/07/2022
Institutionelles Repositorium der Leibniz Universität Hannover
See this paper in CORE
Go to the repository landing page
Download from data provider
oai:www.repo.uni-hannover.de:1...
Last time updated on 01/11/2022