3,230 research outputs found
LoRA-like Calibration for Multimodal Deception Detection using ATSFace Data
Recently, deception detection on human videos is an eye-catching techniques
and can serve lots applications. AI model in this domain demonstrates the high
accuracy, but AI tends to be a non-interpretable black box. We introduce an
attention-aware neural network addressing challenges inherent in video data and
deception dynamics. This model, through its continuous assessment of visual,
audio, and text features, pinpoints deceptive cues. We employ a multimodal
fusion strategy that enhances accuracy; our approach yields a 92\% accuracy
rate on a real-life trial dataset. Most important of all, the model indicates
the attention focus in the videos, providing valuable insights on deception
cues. Hence, our method adeptly detects deceit and elucidates the underlying
process. We further enriched our study with an experiment involving students
answering questions either truthfully or deceitfully, resulting in a new
dataset of 309 video clips, named ATSFace. Using this, we also introduced a
calibration method, which is inspired by Low-Rank Adaptation (LoRA), to refine
individual-based deception detection accuracy.Comment: 10 pages, 9 figure
Can lies be faked? Comparing low-stakes and high-stakes deception video datasets from a Machine Learning perspective
Despite the great impact of lies in human societies and a meager 54% human
accuracy for Deception Detection (DD), Machine Learning systems that perform
automated DD are still not viable for proper application in real-life settings
due to data scarcity. Few publicly available DD datasets exist and the creation
of new datasets is hindered by the conceptual distinction between low-stakes
and high-stakes lies. Theoretically, the two kinds of lies are so distinct that
a dataset of one kind could not be used for applications for the other kind.
Even though it is easier to acquire data on low-stakes deception since it can
be simulated (faked) in controlled settings, these lies do not hold the same
significance or depth as genuine high-stakes lies, which are much harder to
obtain and hold the practical interest of automated DD systems. To investigate
whether this distinction holds true from a practical perspective, we design
several experiments comparing a high-stakes DD dataset and a low-stakes DD
dataset evaluating their results on a Deep Learning classifier working
exclusively from video data. In our experiments, a network trained in
low-stakes lies had better accuracy classifying high-stakes deception than
low-stakes, although using low-stakes lies as an augmentation strategy for the
high-stakes dataset decreased its accuracy.Comment: 11 pages, 3 figure
Deception Detection in Group Video Conversations using Dynamic Interaction Networks
Detecting groups of people who are jointly deceptive in video conversations
is crucial in settings such as meetings, sales pitches, and negotiations. Past
work on deception in videos focuses on detecting a single deceiver and uses
facial or visual features only. In this paper, we propose the concept of
Face-to-Face Dynamic Interaction Networks (FFDINs) to model the interpersonal
interactions within a group of people. The use of FFDINs enables us to leverage
network relations in detecting group deception in video conversations for the
first time. We use a dataset of 185 videos from a deception-based game called
Resistance. We first characterize the behavior of individual, pairs, and groups
of deceptive participants and compare them to non-deceptive participants. Our
analysis reveals that pairs of deceivers tend to avoid mutual interaction and
focus their attention on non-deceivers. In contrast, non-deceivers interact
with everyone equally. We propose Negative Dynamic Interaction Networks to
capture the notion of missing interactions. We create the DeceptionRank
algorithm to detect deceivers from NDINs extracted from videos that are just
one minute long. We show that our method outperforms recent state-of-the-art
computer vision, graph embedding, and ensemble methods by at least 20.9% AUROC
in identifying deception from videos.Comment: The paper is published at ICWSM 2021. Dataset link:
https://snap.stanford.edu/data/comm-f2f-Resistance.htm
Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learning
Deception detection in conversations is a challenging yet important task,
having pivotal applications in many fields such as credibility assessment in
business, multimedia anti-frauds, and custom security. Despite this, deception
detection research is hindered by the lack of high-quality deception datasets,
as well as the difficulties of learning multimodal features effectively. To
address this issue, we introduce DOLOS\footnote {The name ``DOLOS" comes from
Greek mythology.}, the largest gameshow deception detection dataset with rich
deceptive conversations. DOLOS includes 1,675 video clips featuring 213
subjects, and it has been labeled with audio-visual feature annotations. We
provide train-test, duration, and gender protocols to investigate the impact of
different factors. We benchmark our dataset on previously proposed deception
detection approaches. To further improve the performance by fine-tuning fewer
parameters, we propose Parameter-Efficient Crossmodal Learning (PECL), where a
Uniform Temporal Adapter (UT-Adapter) explores temporal attention in
transformer-based architectures, and a crossmodal fusion module, Plug-in
Audio-Visual Fusion (PAVF), combines crossmodal information from audio-visual
features. Based on the rich fine-grained audio-visual annotations on DOLOS, we
also exploit multi-task learning to enhance performance by concurrently
predicting deception and audio-visual features. Experimental results
demonstrate the desired quality of the DOLOS dataset and the effectiveness of
the PECL. The DOLOS dataset and the source codes are available at
https://github.com/NMS05/Audio-Visual-Deception-Detection-DOLOS-Dataset-and-Parameter-Efficient-Crossmodal-Learning/tree/main.Comment: 11 pages, 6 figure
Machine Learning-based Lie Detector applied to a Novel Annotated Game Dataset
Lie detection is considered a concern for everyone in their day to day life
given its impact on human interactions. Thus, people normally pay attention to
both what their interlocutors are saying and also to their visual appearances,
including faces, to try to find any signs that indicate whether the person is
telling the truth or not. While automatic lie detection may help us to
understand this lying characteristics, current systems are still fairly
limited, partly due to lack of adequate datasets to evaluate their performance
in realistic scenarios. In this work, we have collected an annotated dataset of
facial images, comprising both 2D and 3D information of several participants
during a card game that encourages players to lie. Using our collected dataset,
We evaluated several types of machine learning-based lie detectors in terms of
their generalization, person-specific and cross-domain experiments. Our results
show that models based on deep learning achieve the best accuracy, reaching up
to 57\% for the generalization task and 63\% when dealing with a single
participant. Finally, we also highlight the limitation of the deep learning
based lie detector when dealing with cross-domain lie detection tasks
Exploiting Group Structures to Infer Social Interactions From Videos
In this thesis, we consider the task of inferring the social interactions between humans by analyzing multi-modal data. Specifically, we attempt to solve some of the problems in interaction analysis, such as long-term deception detection, political deception detection, and impression prediction. In this work, we emphasize the importance of using knowledge about the group structure of the analyzed interactions. Previous works on the matter mostly neglected this aspect and analyzed a single subject at a time. Using the new Resistance dataset, collected by our collaborators, we approach the problem of long-term deception detection by designing a class of histogram-based features and a novel class of meta-features we callLiarRank. We develop a LiarOrNot model to identify spies in Resistance videos. We achieve AUCs of over 0.70 outperforming our baselines by 3% and human judges by 12%. For the problem of political deception, we first collect a dataset of videos and transcripts of 76 politicians from 18 countries making truthful and deceptive statements. We call it the Global Political Deception Dataset. We then show how to analyze the statements in a broader context by building a Video-Article-Topic graph. From this graph, we create a novel class of features called Deception Score that captures how controversial each topic is and how it affects the truthfulness of each statement. We show that our approach achieves 0.775 AUC outperforming competing baselines. Finally, we use the Resistance data to solve the problem of dyadic impression prediction. Our proposed Dyadic Impression Prediction System (DIPS) contains four major innovations: a novel class of features called emotion ranks, sign imbalance features derived from signed graphs theory, a novel method to align the facial expressions of subjects, and finally, we propose the concept of a multilayered stochastic network we call Temporal Delayed Network. Our DIPS architecture beats eight baselines from the literature, yielding statistically significant improvements of 19.9-30.8% in AUC
Recent Trends in Deep Learning Based Personality Detection
Recently, the automatic prediction of personality traits has received a lot
of attention. Specifically, personality trait prediction from multimodal data
has emerged as a hot topic within the field of affective computing. In this
paper, we review significant machine learning models which have been employed
for personality detection, with an emphasis on deep learning-based methods.
This review paper provides an overview of the most popular approaches to
automated personality detection, various computational datasets, its industrial
applications, and state-of-the-art machine learning models for personality
detection with specific focus on multimodal approaches. Personality detection
is a very broad and diverse topic: this survey only focuses on computational
approaches and leaves out psychological studies on personality detection
- …