Unobtrusive and pervasive video-based eye-gaze tracking

Eye-gaze tracking has long been considered a desktop technology that finds its use inside the traditional office setting, where the operating conditions may be controlled. Nonetheless, recent advancements in mobile technology and a growing interest in capturing natural human behaviour have motivated an emerging interest in tracking eye movements within unconstrained real-life conditions, referred to as pervasive eye-gaze tracking. This critical review focuses on emerging passive and unobtrusive video-based eye-gaze tracking methods in recent literature, with the aim to identify different research avenues that are being followed in response to the challenges of pervasive eye-gaze tracking. Different eye-gaze tracking approaches are discussed in order to bring out their strengths and weaknesses, and to identify any limitations, within the context of pervasive eye-gaze tracking, that have yet to be considered by the computer vision community.peer-reviewe

A 3D Face Modelling Approach for Pose-Invariant Face Recognition in a Human-Robot Environment

Face analysis techniques have become a crucial component of human-machine interaction in the fields of assistive and humanoid robotics. However, the variations in head-pose that arise naturally in these environments are still a great challenge. In this paper, we present a real-time capable 3D face modelling framework for 2D in-the-wild images that is applicable for robotics. The fitting of the 3D Morphable Model is based exclusively on automatically detected landmarks. After fitting, the face can be corrected in pose and transformed back to a frontal 2D representation that is more suitable for face recognition. We conduct face recognition experiments with non-frontal images from the MUCT database and uncontrolled, in the wild images from the PaSC database, the most challenging face recognition database to date, showing an improved performance. Finally, we present our SCITOS G5 robot system, which incorporates our framework as a means of image pre-processing for face analysis

Fitting 3D Morphable Models using Local Features

In this paper, we propose a novel fitting method that uses local image features to fit a 3D Morphable Model to 2D images. To overcome the obstacle of optimising a cost function that contains a non-differentiable feature extraction operator, we use a learning-based cascaded regression method that learns the gradient direction from data. The method allows to simultaneously solve for shape and pose parameters. Our method is thoroughly evaluated on Morphable Model generated data and first results on real data are presented. Compared to traditional fitting methods, which use simple raw features like pixel colour or edge maps, local features have been shown to be much more robust against variations in imaging conditions. Our approach is unique in that we are the first to use local features to fit a Morphable Model. Because of the speed of our method, it is applicable for realtime applications. Our cascaded regression framework is available as an open source library (https://github.com/patrikhuber).Comment: Submitted to ICIP 2015; 4 pages, 4 figure

Semantic Morphable Models

In this thesis we discuss how computers can automatically interpret images of human faces. The applications of face image analysis systems range from image description, face analysis, interpretation, human-computer interaction, forensics to image manipulation. The analysis of faces in unconstrained scenes is a challenging task. Faces appear in images in a high variety of shape and texture and factors influencing the image formation process like illumination, 3D pose and the scene itself. A face is only a component of a scene and can be occluded by glasses or various other objects in front of the face. We propose an attribute-based image description framework for the analysis of unconstrained face images. The core of the framework are copula Morphable Models to jointly model facial shape, color and attributes in a generative statistical way. A set of model parameters for a face image directly holds facial attributes as image description. We estimate the model parameters for a new image in an Analysis-by-Synthesis setting. In this process, we include a semantic segmentation of the target image into semantic regions to be targeted by their associated models. Different models compete to explain the image pixels. We focus on face image analysis and use a face, a beard and a non-face model to explain different parts of input images. This semantic Morphable Model framework leads to better face explanation since only pixels belonging to the face have to be explained by the face model. We include occlusions or beards as semantic regions and model them as separated classes in the implemented application of the proposed framework. A main cornerstone for the Analysis-by-Synthesis process is illumination estimation. Illumination dominates facial appearance and varies strongly in natural images. We explicitly estimate the illumination condition robust to occlusions and outliers. This thesis combines copula Morphable Models, semantic model adaptation, image segmentation and robust illumination estimation which are necessary to build the overall semantic Morphable Model framework

Open-Source Face Recognition Frameworks: A Review of the Landscape

publishedVersio

Detecting human engagement propensity in human-robot interaction

Elaborazione di immagini ricavate dal flusso di una semplice videocamera RGB di un robot al fine di stimare la propensione all'interazione di una persona in situazioni di interazione uomo-robot. Per calcolare la stima finale, tecniche basate su deep learning sono usate per estrarre alcune informazioni ausiliarie come: stima della posa di una persona, quale tipo di posa, orientamento del corpo, orientamento della testa, come appaiono le mani.Processing of images retrieved from a simple robot RGB camera stream in order to estimate the engagement propensity of a person in human-robot interaction scenarios. To compute the final estimation, deep learning based technique are used to extract some auxiliary information as: estimation of the pose of a person, which type of pose, body orientation, head orientation, how hands appear