20 research outputs found
Entity-sensitive attention and fusion network for entity-level multimodal sentiment classification
National Research Foundation (NRF) Singapor
VISIR : visual and semantic image label refinement
The social media explosion has populated the Internet with a wealth of images. There are two existing paradigms for image retrieval: 1) content-based image retrieval (CBIR), which has traditionally used visual features for similarity search (e.g., SIFT features), and 2) tag-based image retrieval (TBIR), which has relied on user tagging (e.g., Flickr tags). CBIR now gains semantic expressiveness by advances in deep-learning-based detection of visual labels. TBIR benefits from query-and-click logs to automatically infer more informative labels. However, learning-based tagging still yields noisy labels and is restricted to concrete objects, missing out on generalizations and abstractions. Click-based tagging is limited to terms that appear in the textual context of an image or in queries that lead to a click. This paper addresses the above limitations by semantically refining and expanding the labels suggested by learning-based object detection. We consider the semantic coherence between the labels for different objects, leverage lexical and commonsense knowledge, and cast the label assignment into a constrained optimization problem solved by an integer linear program. Experiments show that our method, called VISIR, improves the quality of the state-of-the-art visual labeling tools like LSDA and YOLO
Computational Emotion Analysis From Images: Recent Advances and Future Directions
Emotions are usually evoked in humans by images. Recently, extensive research
efforts have been dedicated to understanding the emotions of images. In this
chapter, we aim to introduce image emotion analysis (IEA) from a computational
perspective with the focus on summarizing recent advances and suggesting future
directions. We begin with commonly used emotion representation models from
psychology. We then define the key computational problems that the researchers
have been trying to solve and provide supervised frameworks that are generally
used for different IEA tasks. After the introduction of major challenges in
IEA, we present some representative methods on emotion feature extraction,
supervised classifier learning, and domain adaptation. Furthermore, we
introduce available datasets for evaluation and summarize some main results.
Finally, we discuss some open questions and future directions that researchers
can pursue.Comment: Accepted chapter in the book "Human Perception of Visual Information
Psychological and Computational Perspective
A Survey on Visual Analytics of Social Media Data
The unprecedented availability of social media data offers substantial opportunities for data owners, system operators, solution providers, and end users to explore and understand social dynamics. However, the exponential growth in the volume, velocity, and variability of social media data prevents people from fully utilizing such data. Visual analytics, which is an emerging research direction, ha..