Measuring and modeling the perception of natural and unconstrained gaze in humans and machines

Humans are remarkably adept at interpreting the gaze direction of other individuals in their surroundings. This skill is at the core of the ability to engage in joint visual attention, which is essential for establishing social interactions. How accurate are humans in determining the gaze direction of others in lifelike scenes, when they can move their heads and eyes freely, and what are the sources of information for the underlying perceptual processes? These questions pose a challenge from both empirical and computational perspectives, due to the complexity of the visual input in real-life situations. Here we measure empirically human accuracy in perceiving the gaze direction of others in lifelike scenes, and study computationally the sources of information and representations underlying this cognitive capacity. We show that humans perform better in face-to-face conditions compared with recorded conditions, and that this advantage is not due to the availability of input dynamics. We further show that humans are still performing well when only the eyes-region is visible, rather than the whole face. We develop a computational model, which replicates the pattern of human performance, including the finding that the eyes-region contains on its own, the required information for estimating both head orientation and direction of gaze. Consistent with neurophysiological findings on task-specific face regions in the brain, the learned computational representations reproduce perceptual effects such as the Wollaston illusion, when trained to estimate direction of gaze, but not when trained to recognize objects or faces.This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF – 1231216

Unobtrusive and pervasive video-based eye-gaze tracking

Eye-gaze tracking has long been considered a desktop technology that finds its use inside the traditional office setting, where the operating conditions may be controlled. Nonetheless, recent advancements in mobile technology and a growing interest in capturing natural human behaviour have motivated an emerging interest in tracking eye movements within unconstrained real-life conditions, referred to as pervasive eye-gaze tracking. This critical review focuses on emerging passive and unobtrusive video-based eye-gaze tracking methods in recent literature, with the aim to identify different research avenues that are being followed in response to the challenges of pervasive eye-gaze tracking. Different eye-gaze tracking approaches are discussed in order to bring out their strengths and weaknesses, and to identify any limitations, within the context of pervasive eye-gaze tracking, that have yet to be considered by the computer vision community.peer-reviewe

Detección robusta de la orientación de la cabeza del usuario a partir de una cámara RGBZ

La localización de caras es una característica ampliamente utilizada actualmente en diferentes productos software. Además, con la aparición de sensores RGBZ (como la Kinect o la RealSense) se ha añadido la capacidad no sólo detectar dónde hay una cabeza, si no de obtener información tridimensional sobre la misma. En este proyecto se diseña, desarrolla y analiza un software que permita obtener, mediante el uso de las cámaras RGBZ anteriormente mencionadas, la orientación 3D de la cabeza del usuario que esté delante de ellas, es decir, los ángulos que determinan hacia qué dirección está mirando el usuario. Para ello se ha diseñado un algoritmo basado en el método Iterative Closest Point, de forma que por cada imagen capturada por la cámara se detecte qué ángulos presenta la cabeza. También se ha desarrollado una plataforma externa utilizando un servomotor y un microcontrolador Arduino, permitiendo realizar pruebas de los diferentes parámetros del algoritmo para validar sus resultados, mediante una plataforma giratoria sobre la que se puede orientar con precisión una reproducción a escala de una cabeza 3D.La localització de cares es una característica àmpliament utilitzada en diferents productes software actualment. A més, amb l’aparició de sensors RGBZ (com la Kinect o la RealSense) s’ha afegit la capacitat de, no només detectar a on hi ha una cara, si no d’obtenir la informació tridimensional d’aquesta. En aquest projecte es dissenya, desenvolupa i s’analitza un software que permeti obtenir, mitjançant l’ús de les càmeres RGBZ anteriorment nombrades, la orientació del cap de l’usuari que es trobi davant d’elles, és a dir, dels angles que defineixen cap a quina direcció està mirant l’usuari. Per aconseguir-ho s’ha dissenyat un algoritme basat en el mètode Iterative Closest Point, de manera que per cada imatge capturada per la càmera es detecti quins angles presenta el cap. També s’ha desenvolupat una plataforma externa utilitzant un motor i un microcontrolador Arduino, a on es poden realitzar proves dels diferents paràmetres de l’algoritme per validar els resultats mitjançant una plataforma giratòria sobre la qual s’ha col·locat una reproducció a escala d’un cap en tres dimensions que es pot orientar amb precisió.Face localization has become a hugely demanded feature in many different sofware products. In addition, with the appearence of RGBZ sensors (such as the Kinect and the RealSense) the capacity of not only detecting where the face is located but also obtaining the 3D orientation of the face has been added. In this project we aim to design, develop and test a software able to, using the RGBZ sensors, detect the pose of the head of a user in front of the camera, that is, extract the three angles that define the direction of the head. To do that, we developed an algorithm based on the Iterative Closest Point family. For each image provided by the camera, the angles are detected. An external platform was also developed using a servomotor and an Arduino microcontroller, able to perform tests of the different parameters of the algorithm to validate the results using a rotating base that can turn precisely a reproduction of a real-size 3D printed head

Detection of Head Pose and Gaze Direction for Human-Computer Interaction

Abstract. In this contribution we extend existing methods for head pose estimation and investigate the use of local image phase for gaze de-tection. Moreover we describe how a small database of face images with given ground truth for head pose and gaze direction was acquired. With this database we compare two different computational approaches for ex-tracting the head pose. We demonstrate that a simple implementation of the proposed methods without extensive training sessions or calibration is sufficient to accurately detect the head pose for human-computer in-teraction. Furthermore, we propose how eye gaze can be extracted based on the outcome of local filter responses and the detected head pose. In all, we present a framework where different approaches are combined to a single system for extracting information about the attentional state of a person

Detection of Head Pose and Gaze Direction for Human-Computer Interaction

Abstract. In this contribution we extend existing methods for head pose estimation and investigate the use of local image phase for gaze detection. Moreover we describe how a small database of face images with given ground truth for head pose and gaze direction was acquired. With this database we compare two different computational approaches for extracting the head pose. We demonstrate that a simple implementation of the proposed methods without extensive training sessions or calibration is sufficient to accurately detect the head pose for human-computer interaction. Furthermore, we propose how eye gaze can be extracted based on the outcome of local filter responses and the detected head pose. In all, we present a framework where different approaches are combined to a single system for extracting information about the attentional state of a person

Verbesserung der Störsicherheit bei der Mimikanalyse in mono- und binokularen Farbbildsequenzen durch Auswertung geometrischer und dynamischer Merkmale

Magdeburg, Univ., Fak. für Elektrotechnik und Informationstechnik, Diss., 2010Robert Nies