Deception Detection in Videos

We present a system for covert automated deception detection in real-life courtroom trial videos. We study the importance of different modalities like vision, audio and text for this task. On the vision side, our system uses classifiers trained on low level video features which predict human micro-expressions. We show that predictions of high-level micro-expressions can be used as features for deception prediction. Surprisingly, IDT (Improved Dense Trajectory) features which have been widely used for action recognition, are also very good at predicting deception in videos. We fuse the score of classifiers trained on IDT features and high-level micro-expressions to improve performance. MFCC (Mel-frequency Cepstral Coefficients) features from the audio domain also provide a significant boost in performance, while information from transcripts is not very beneficial for our system. Using various classifiers, our automated system obtains an AUC of 0.877 (10-fold cross-validation) when evaluated on subjects which were not part of the training set. Even though state-of-the-art methods use human annotations of micro-expressions for deception detection, our fully automated approach outperforms them by 5%. When combined with human annotations of micro-expressions, our AUC improves to 0.922. We also present results of a user-study to analyze how well do average humans perform on this task, what modalities they use for deception detection and how they perform if only one modality is accessible. Our project page can be found at \url{https://doubaibai.github.io/DARE/}.Comment: AAAI 2018, project page: https://doubaibai.github.io/DARE

Detecting Micro-Expressions in Real Time Using High-Speed Video Sequences

Micro-expressions (ME) are brief, fast facial movements that occur in high-stake situations when people try to conceal their feelings, as a form of either suppression or repression. They are reliable sources of deceit detection and human behavior understanding. Automatic analysis of micro-expression is challenging because of their short duration (they occur as fast as 1/15–1/25 of a second) and their low movement amplitude. In this study, we report a fast and robust micro-expression detection framework, which analyzes the subtle movement variations that occur around the most prominent facial regions using two absolute frame differences and simple classifier to predict the micro-expression frames. The robustness of the system is increased by further processing the preliminary predictions of the classifier: the appropriate predicted micro-expression intervals are merged together and the intervals that are too short are filtered out

The truth-telling motor cortex: Response competition in M1 discloses deceptive behaviour

Neural circuits associated with response conflict are active during deception. Here we use transcranial magnetic stimulation to examine for the first time whether competing responses in primary motor cortex can be used to detect lies. Participants used their little finger or thumb to respond either truthfully or deceitfully regarding facial familiarity. Motor-evoked-potentials (MEPs) from muscles associated with both digits tracked the development of each motor plan. When preparing to deceive, the MEP of the non-responding digit (i.e. the plan corresponding to the truth) exceeds the MEP of the responding digit (i.e. the lie), whereas a mirror-reversed pattern occurs when telling the truth. This give away response conflict interacts with the time of stimulation during a speeded reaction period. Lies can even activate digit-specific cortical representations when only verbal responses are made. Our findings support neurobiological models which blend cognitive decision-making with motor programming, and suggest a novel index for discriminating between honest and intentionally false facial recognition

Detection of simple and complex deceits through facial micro-expressions: a comparison between human beings’ performances and machine learning techniques

Micro-expressions have gained increasing interest in the last few years, both in scientific and professional contexts. Theoretically, their emergence suggests ongoing concealments, making them arguably one of the most reliable cues for lie detection (e.g., Yan, Wang, Liu, Wu & Fu, 2014; Venkatesh, Ramachandra & Bours, 2019). Given their fast onset, they result almost imperceptible to the eye of an untrained subject, making it necessary to work on automatic detection tools. Machine learning models have shown promisingly results within this domain; thus, the aim of the study at hand, was to compare the performances human judges and machine learning models obtain on the same dataset of stimuli. Regrettably, machine learning performances have ended up being around the chance level, positing the question of why previous and a-like studies have collected better results. Briefly, insights on how to properly organize an experimental paradigm and collect a dataset for lie detection studies are discussed, while concluding that among other several necessary cues it is still crucial to consider micro-expressions when dealing with lie detection procedures

Automatic Recognition of Facial Displays of Unfelt Emotions

Humans modify their facial expressions in order to communicate their internal states and sometimes to mislead observers regarding their true emotional states. Evidence in experimental psychology shows that discriminative facial responses are short and subtle. This suggests that such behavior would be easier to distinguish when captured in high resolution at an increased frame rate. We are proposing SASE-FE, the first dataset of facial expressions that are either congruent or incongruent with underlying emotion states. We show that overall the problem of recognizing whether facial movements are expressions of authentic emotions or not can be successfully addressed by learning spatio-temporal representations of the data. For this purpose, we propose a method that aggregates features along fiducial trajectories in a deeply learnt space. Performance of the proposed model shows that on average, it is easier to distinguish among genuine facial expressions of emotion than among unfelt facial expressions of emotion and that certain emotion pairs such as contempt and disgust are more difficult to distinguish than the rest. Furthermore, the proposed methodology improves state of the art results on CK+ and OULU-CASIA datasets for video emotion recognition, and achieves competitive results when classifying facial action units on BP4D datase