1,285 research outputs found
A Quantum-Inspired Multimodal Sentiment Analysis Framework
Multimodal sentiment analysis aims to capture diversified sentiment information implied in data that are of different modalities (e.g., an image that is associated with a textual description or a set of textual labels). The key challenge is rooted on the “semantic gap” between different low-level content features and high-level semantic information. Existing approaches generally utilize a combination of multimodal features in a somehow heuristic way. However, how to employ and combine multiple information from different sources effectively is still an important yet largely unsolved problem. To address the problem, in this paper, we propose a Quantum-inspired Multimodal Sentiment Analysis (QMSA) framework. The framework consists of a Quantum-inspired Multimodal Representation (QMR) model (which aims to fill the “semantic gap” and model the correlations between different modalities via density matrix), and a Multimodal decision Fusion strategy inspired by Quantum Interference (QIMF) in the double-slit experiment (in which the sentiment label is analogous to a photon, and the data modalities are analogous to slits). Extensive experiments are conducted on two large scale datasets, which are collected from the Getty Images and Flickr photo sharing platform. The experimental results show that our approach significantly outperforms a wide range of baselines and state-of-the-art methods
Recommended from our members
A Quantum-like Multimodal Network Framework for Modeling Interaction Dynamics in Multiparty Conversational Sentiment Analysis
Sentiment analysis in conversations is an emerging yet challenging artificial intelligence (AI) task. It aims to discover the affective states and emotional changes of speakers involved in a conversation on the basis of their opinions, which are carried by different modalities of information (e.g., a video associated with a transcript). There exists a wealth of intra- and inter-utterance interaction information that affects the emotions of speakers in a complex and dynamic way. How to accurately and comprehensively model complicated interactions is the key problem of the field. To fill this gap, in this paper, we propose a novel and comprehensive framework for multimodal sentiment analysis in conversations, called a quantum-like multimodal network (QMN), which leverages the mathematical formalism of quantum theory (QT) and a long short-term memory (LSTM) network. Specifically, the QMN framework consists of a multimodal decision fusion approach inspired by quantum interference theory to capture the interactions within each utterance (i.e., the correlations between different modalities) and a strong-weak influence model inspired by quantum measurement theory to model the interactions between adjacent utterances (i.e., how one speaker influences another). Extensive experiments are conducted on two widely used conversational sentiment datasets: the MELD and IEMOCAP datasets. The experimental results show that our approach significantly outperforms a wide range of baselines and state-of-the-art models
A Survey of Quantum-Cognitively Inspired Sentiment Analysis Models
Quantum theory, originally proposed as a physical theory to describe the
motions of microscopic particles, has been applied to various non-physics
domains involving human cognition and decision-making that are inherently
uncertain and exhibit certain non-classical, quantum-like characteristics.
Sentiment analysis is a typical example of such domains. In the last few years,
by leveraging the modeling power of quantum probability (a non-classical
probability stemming from quantum mechanics methodology) and deep neural
networks, a range of novel quantum-cognitively inspired models for sentiment
analysis have emerged and performed well. This survey presents a timely
overview of the latest developments in this fascinating cross-disciplinary
area. We first provide a background of quantum probability and quantum
cognition at a theoretical level, analyzing their advantages over classical
theories in modeling the cognitive aspects of sentiment analysis. Then, recent
quantum-cognitively inspired models are introduced and discussed in detail,
focusing on how they approach the key challenges of the sentiment analysis
task. Finally, we discuss the limitations of the current research and highlight
future research directions
Quantum Cognitively Motivated Decision Fusion for Video Sentiment Analysis
Video sentiment analysis as a decision-making process is inherently complex,
involving the fusion of decisions from multiple modalities and the so-caused
cognitive biases. Inspired by recent advances in quantum cognition, we show
that the sentiment judgment from one modality could be incompatible with the
judgment from another, i.e., the order matters and they cannot be jointly
measured to produce a final decision. Thus the cognitive process exhibits
"quantum-like" biases that cannot be captured by classical probability
theories. Accordingly, we propose a fundamentally new, quantum cognitively
motivated fusion strategy for predicting sentiment judgments. In particular, we
formulate utterances as quantum superposition states of positive and negative
sentiment judgments, and uni-modal classifiers as mutually incompatible
observables, on a complex-valued Hilbert space with positive-operator valued
measures. Experiments on two benchmarking datasets illustrate that our model
significantly outperforms various existing decision level and a range of
state-of-the-art content-level fusion approaches. The results also show that
the concept of incompatibility allows effective handling of all combination
patterns, including those extreme cases that are wrongly predicted by all
uni-modal classifiers.Comment: The uploaded version is a preprint of the accepted AAAI-21 pape
Emotion Quantification Using Variational Quantum State Fidelity Estimation
Sentiment analysis has been instrumental in developing artificial intelligence when applied to various domains. However, most sentiments and emotions are temporal and often exist in a complex manner. Several emotions can be experienced at the same time. Instead of recognizing only categorical information about emotions, there is a need to understand and quantify the intensity of emotions. The proposed research intends to investigate a quantum-inspired approach for quantifying emotional intensities in runtime. The inspiration comes from manifesting human cognition and decision-making capabilities, which may adopt a brief explanation through quantum theory. Quantum state fidelity was used to characterize states and estimate emotion intensities rendered by subjects from the Amsterdam Dynamic Facial Expression Set (ADFES) dataset. The Quantum variational classifier technique was used to perform this experiment on the IBM Quantum Experience platform. The proposed method successfully quantifies the intensities of joy, sadness, contempt, anger, surprise, and fear emotions of labelled subjects from the ADFES dataset
A Survey of Quantum Theory Inspired Approaches to Information Retrieval
Since 2004, researchers have been using the mathematical framework of Quantum Theory (QT) in Information Retrieval (IR). QT offers a generalized probability and logic framework. Such a framework has been shown capable of unifying the representation, ranking and user cognitive aspects of IR, and helpful in developing more dynamic, adaptive and context-aware IR systems. Although Quantum-inspired IR is still a growing area, a wide array of work in different aspects of IR has been done and produced promising results. This paper presents a survey of the research done in this area, aiming to show the landscape of the field and draw a road-map of future directions
A Quantum Probability Driven Framework for Joint Multi-Modal Sarcasm, Sentiment and Emotion Analysis
Sarcasm, sentiment, and emotion are three typical kinds of spontaneous
affective responses of humans to external events and they are tightly
intertwined with each other. Such events may be expressed in multiple
modalities (e.g., linguistic, visual and acoustic), e.g., multi-modal
conversations. Joint analysis of humans' multi-modal sarcasm, sentiment, and
emotion is an important yet challenging topic, as it is a complex cognitive
process involving both cross-modality interaction and cross-affection
correlation. From the probability theory perspective, cross-affection
correlation also means that the judgments on sarcasm, sentiment, and emotion
are incompatible. However, this exposed phenomenon cannot be sufficiently
modelled by classical probability theory due to its assumption of
compatibility. Neither do the existing approaches take it into consideration.
In view of the recent success of quantum probability (QP) in modeling human
cognition, particularly contextual incompatible decision making, we take the
first step towards introducing QP into joint multi-modal sarcasm, sentiment,
and emotion analysis. Specifically, we propose a QUantum probabIlity driven
multi-modal sarcasm, sEntiment and emoTion analysis framework, termed QUIET.
Extensive experiments on two datasets and the results show that the
effectiveness and advantages of QUIET in comparison with a wide range of the
state-of-the-art baselines. We also show the great potential of QP in
multi-affect analysis
- …