Recognition of nonmanual markers in American Sign Language (ASL) using non-parametric adaptive 2D-3D face tracking

This paper addresses the problem of automatically recognizing linguistically significant nonmanual expressions in American Sign Language from video. We develop a fully automatic system that is able to track facial expressions and head movements, and detect and recognize facial events continuously from video. The main contributions of the proposed framework are the following: (1) We have built a stochastic and adaptive ensemble of face trackers to address factors resulting in lost face track; (2) We combine 2D and 3D deformable face models to warp input frames, thus correcting for any variation in facial appearance resulting from changes in 3D head pose; (3) We use a combination of geometric features and texture features extracted from a canonical frontal representation. The proposed new framework makes it possible to detect grammatically significant nonmanual expressions from continuous signing and to differentiate successfully among linguistically significant expressions that involve subtle differences in appearance. We present results that are based on the use of a dataset containing 330 sentences from videos that were collected and linguistically annotated at Boston University

Cognitive visual tracking and camera control

Cognitive visual tracking is the process of observing and understanding the behaviour of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision

Assentication: User Deauthentication and Lunchtime Attack Mitigation with Seated Posture Biometric

Biometric techniques are often used as an extra security factor in authenticating human users. Numerous biometrics have been proposed and evaluated, each with its own set of benefits and pitfalls. Static biometrics (such as fingerprints) are geared for discrete operation, to identify users, which typically involves some user burden. Meanwhile, behavioral biometrics (such as keystroke dynamics) are well suited for continuous, and sometimes more unobtrusive, operation. One important application domain for biometrics is deauthentication, a means of quickly detecting absence of a previously authenticated user and immediately terminating that user's active secure sessions. Deauthentication is crucial for mitigating so called Lunchtime Attacks, whereby an insider adversary takes over (before any inactivity timeout kicks in) authenticated state of a careless user who walks away from her computer. Motivated primarily by the need for an unobtrusive and continuous biometric to support effective deauthentication, we introduce PoPa, a new hybrid biometric based on a human user's seated posture pattern. PoPa captures a unique combination of physiological and behavioral traits. We describe a low cost fully functioning prototype that involves an office chair instrumented with 16 tiny pressure sensors. We also explore (via user experiments) how PoPa can be used in a typical workplace to provide continuous authentication (and deauthentication) of users. We experimentally assess viability of PoPa in terms of uniqueness by collecting and evaluating posture patterns of a cohort of users. Results show that PoPa exhibits very low false positive, and even lower false negative, rates. In particular, users can be identified with, on average, 91.0% accuracy. Finally, we compare pros and cons of PoPa with those of several prominent biometric based deauthentication techniques