2,565 research outputs found
Automated Deception Detection from Videos: Using End-to-End Learning Based High-Level Features and Classification Approaches
Deception detection is an interdisciplinary field attracting researchers from
psychology, criminology, computer science, and economics. We propose a
multimodal approach combining deep learning and discriminative models for
automated deception detection. Using video modalities, we employ convolutional
end-to-end learning to analyze gaze, head pose, and facial expressions,
achieving promising results compared to state-of-the-art methods. Due to
limited training data, we also utilize discriminative models for deception
detection. Although sequence-to-class approaches are explored, discriminative
models outperform them due to data scarcity. Our approach is evaluated on five
datasets, including a new Rolling-Dice Experiment motivated by economic
factors. Results indicate that facial expressions outperform gaze and head
pose, and combining modalities with feature selection enhances detection
performance. Differences in expressed features across datasets emphasize the
importance of scenario-specific training data and the influence of context on
deceptive behavior. Cross-dataset experiments reinforce these findings. Despite
the challenges posed by low-stake datasets, including the Rolling-Dice
Experiment, deception detection performance exceeds chance levels. Our proposed
multimodal approach and comprehensive evaluation shed light on the potential of
automating deception detection from video modalities, opening avenues for
future research.Comment: 29 pages, 17 figures (19 if counting subfigures
Learning Grimaces by Watching TV
Differently from computer vision systems which require explicit supervision,
humans can learn facial expressions by observing people in their environment.
In this paper, we look at how similar capabilities could be developed in
machine vision. As a starting point, we consider the problem of relating facial
expressions to objectively measurable events occurring in videos. In
particular, we consider a gameshow in which contestants play to win significant
sums of money. We extract events affecting the game and corresponding facial
expressions objectively and automatically from the videos, obtaining large
quantities of labelled data for our study. We also develop, using benchmarks
such as FER and SFEW 2.0, state-of-the-art deep neural networks for facial
expression recognition, showing that pre-training on face verification data can
be highly beneficial for this task. Then, we extend these models to use facial
expressions to predict events in videos and learn nameable expressions from
them. The dataset and emotion recognition models are available at
http://www.robots.ox.ac.uk/~vgg/data/facevalueComment: British Machine Vision Conference (BMVC) 201
LoRA-like Calibration for Multimodal Deception Detection using ATSFace Data
Recently, deception detection on human videos is an eye-catching techniques
and can serve lots applications. AI model in this domain demonstrates the high
accuracy, but AI tends to be a non-interpretable black box. We introduce an
attention-aware neural network addressing challenges inherent in video data and
deception dynamics. This model, through its continuous assessment of visual,
audio, and text features, pinpoints deceptive cues. We employ a multimodal
fusion strategy that enhances accuracy; our approach yields a 92\% accuracy
rate on a real-life trial dataset. Most important of all, the model indicates
the attention focus in the videos, providing valuable insights on deception
cues. Hence, our method adeptly detects deceit and elucidates the underlying
process. We further enriched our study with an experiment involving students
answering questions either truthfully or deceitfully, resulting in a new
dataset of 309 video clips, named ATSFace. Using this, we also introduced a
calibration method, which is inspired by Low-Rank Adaptation (LoRA), to refine
individual-based deception detection accuracy.Comment: 10 pages, 9 figure
- …