A Bayesian Approach to Tracking Learning Detection

n/

Face pose estimation with automatic 3D model creation for a driver inattention monitoring application

Texto en inglés y resumen en inglés y españolRecent studies have identified inattention (including distraction and drowsiness) as the main cause of accidents, being responsible of at least 25% of them. Driving distraction has been less studied, since it is more diverse and exhibits a higher risk factor than fatigue. In addition, it is present over half of the inattention involved crashes. The increased presence of In Vehicle Information Systems (IVIS) adds to the potential distraction risk and modifies driving behaviour, and thus research on this issue is of vital importance. Many researchers have been working on different approaches to deal with distraction during driving. Among them, Computer Vision is one of the most common, because it allows for a cost effective and non-invasive driver monitoring and sensing. Using Computer Vision techniques it is possible to evaluate some facial movements that characterise the state of attention of a driver. This thesis presents methods to estimate the face pose and gaze direction of a person in real-time, using a stereo camera as a basic for assessing driver distractions. The methods are completely automatic and user-independent. A set of features in the face are identified at initialisation, and used to create a sparse 3D model of the face. These features are tracked from frame to frame, and the model is augmented to cover parts of the face that may have been occluded before. The algorithm is designed to work in a naturalistic driving simulator, which presents challenging low light conditions. We evaluate several techniques to detect features on the face that can be matched between cameras and tracked with success. Well-known methods such as SURF do not return good results, due to the lack of salient points in the face, as well as the low illumination of the images. We introduce a novel multisize technique, based on Harris corner detector and patch correlation. This technique benefits from the better performance of small patches under rotations and illumination changes, and the more robust correlation of the bigger patches under motion blur. The head rotates in a range of ±90º in the yaw angle, and the appearance of the features change noticeably. To deal with these changes, we implement a new re-registering technique that captures new textures of the features as the face rotates. These new textures are incorporated to the model, which mixes the views of both cameras. The captures are taken at regular angle intervals for rotations in yaw, so that each texture is only used in a range of ±7.5º around the capture angle. Rotations in pitch and roll are handled using affine patch warping. The 3D model created at initialisation can only take features in the frontal part of the face, and some of these may occlude during rotations. The accuracy and robustness of the face tracking depends on the number of visible points, so new points are added to the 3D model when new parts of the face are visible from both cameras. Bundle adjustment is used to reduce the accumulated drift of the 3D reconstruction. We estimate the pose from the position of the features in the images and the 3D model using POSIT or Levenberg-Marquardt. A RANSAC process detects incorrectly tracked points, which are not considered for pose estimation. POSIT is faster, while LM obtains more accurate results. Using the model extension and the re-registering technique, we can accurately estimate the pose in the full head rotation range, with error levels that improve the state of the art. A coarse eye direction is composed with the face pose estimation to obtain the gaze and driver's fixation area, parameter which gives much information about the distraction pattern of the driver. The resulting gaze estimation algorithm proposed in this thesis has been tested on a set of driving experiments directed by a team of psychologists in a naturalistic driving simulator. This simulator mimics conditions present in real driving, including weather changes, manoeuvring and distractions due to IVIS. Professional drivers participated in the tests. The driver?s fixation statistics obtained with the proposed system show how the utilisation of IVIS influences the distraction pattern of the drivers, increasing reaction times and affecting the fixation of attention on the road and the surroundings

Face pose estimation with automatic 3D model creation for a driver inattention monitoring application

Texto en inglés y resumen en inglés y españolRecent studies have identified inattention (including distraction and drowsiness) as the main cause of accidents, being responsible of at least 25% of them. Driving distraction has been less studied, since it is more diverse and exhibits a higher risk factor than fatigue. In addition, it is present over half of the inattention involved crashes. The increased presence of In Vehicle Information Systems (IVIS) adds to the potential distraction risk and modifies driving behaviour, and thus research on this issue is of vital importance. Many researchers have been working on different approaches to deal with distraction during driving. Among them, Computer Vision is one of the most common, because it allows for a cost effective and non-invasive driver monitoring and sensing. Using Computer Vision techniques it is possible to evaluate some facial movements that characterise the state of attention of a driver. This thesis presents methods to estimate the face pose and gaze direction of a person in real-time, using a stereo camera as a basic for assessing driver distractions. The methods are completely automatic and user-independent. A set of features in the face are identified at initialisation, and used to create a sparse 3D model of the face. These features are tracked from frame to frame, and the model is augmented to cover parts of the face that may have been occluded before. The algorithm is designed to work in a naturalistic driving simulator, which presents challenging low light conditions. We evaluate several techniques to detect features on the face that can be matched between cameras and tracked with success. Well-known methods such as SURF do not return good results, due to the lack of salient points in the face, as well as the low illumination of the images. We introduce a novel multisize technique, based on Harris corner detector and patch correlation. This technique benefits from the better performance of small patches under rotations and illumination changes, and the more robust correlation of the bigger patches under motion blur. The head rotates in a range of ±90º in the yaw angle, and the appearance of the features change noticeably. To deal with these changes, we implement a new re-registering technique that captures new textures of the features as the face rotates. These new textures are incorporated to the model, which mixes the views of both cameras. The captures are taken at regular angle intervals for rotations in yaw, so that each texture is only used in a range of ±7.5º around the capture angle. Rotations in pitch and roll are handled using affine patch warping. The 3D model created at initialisation can only take features in the frontal part of the face, and some of these may occlude during rotations. The accuracy and robustness of the face tracking depends on the number of visible points, so new points are added to the 3D model when new parts of the face are visible from both cameras. Bundle adjustment is used to reduce the accumulated drift of the 3D reconstruction. We estimate the pose from the position of the features in the images and the 3D model using POSIT or Levenberg-Marquardt. A RANSAC process detects incorrectly tracked points, which are not considered for pose estimation. POSIT is faster, while LM obtains more accurate results. Using the model extension and the re-registering technique, we can accurately estimate the pose in the full head rotation range, with error levels that improve the state of the art. A coarse eye direction is composed with the face pose estimation to obtain the gaze and driver's fixation area, parameter which gives much information about the distraction pattern of the driver. The resulting gaze estimation algorithm proposed in this thesis has been tested on a set of driving experiments directed by a team of psychologists in a naturalistic driving simulator. This simulator mimics conditions present in real driving, including weather changes, manoeuvring and distractions due to IVIS. Professional drivers participated in the tests. The driver?s fixation statistics obtained with the proposed system show how the utilisation of IVIS influences the distraction pattern of the drivers, increasing reaction times and affecting the fixation of attention on the road and the surroundings

BTLD+:A BAYESIAN APPROACH TO TRACKING LEARNING DETECTION BY PARTS

The contribution proposed in this thesis focuses on this particular instance of the visual tracking problem, referred as Adaptive Ap- iv \ufffcpearance Tracking. We proposed different approaches based on the Tracking Learning Detection (TLD) decomposition proposed in [55]. TLD decomposes visual tracking into three components, namely the tracker, the learner and detector. The tracker and the detector are two competitive processes for target localization based on comple- mentary sources of informations. The former searches for local fea- tures between consecutive frames in order to localize the target; the latter exploits an on-line appearance model to detect confident hy- pothesis over the entire image. The learner selects the final solution among the provided hypothesis. It updates the target appearance model, if necessary, reinitialize the tracker and bootstraps the detec- tor\u2019s appearance model. In particular, we investigated different ap- proaches to enforce the TLD stability. First, we replaced the tracker component with a novel one based on mcmc particle filtering; after- wards, we proposed a robust appearance modeling component able to characterize deformable objects in static images; after all, we inte- grated a modeling component able to integrate local visual features learning into the whole approach, lying to a couple layered represen- tation of the target appearance

Layered graphical models for tracking partially-occluded moving objects in video (PhD thesis)

Tracking multiple targets using fixed cameras with non-overlapping views is a challenging problem. One of the challenges is predicting and tracking through occlusions caused by other targets or by fixed objects in the scene. Considerable effort has been devoted toward developing appearance models that are robust to partial occlusions, tracking algorithms that cope with short-term loss of observations, and algorithms that learn static occlusion maps. In this thesis we consider scenarios where it is impossible to learn a static occlusion map. This is often the case when the scene consists of both people and large objects whose position is not permanently fixed. These objects may enter, leave or relocate within the scene during a short time span. We call such objects "relocatable objects" or "relocatable occluders." We develop a representation for scenes containing relocatable objects that can cause partial occlusions of people in a camera's field of view. In many practical applications, relocatable objects tend to appear often; therefore, models for them can be learned off-line and stored in a database. We formulate an occluder-centric representation, called a graphical model layer, where a person's motion in the ground plane is defined as a first-order Markov process on activity zones, while image evidence is aggregated in 2D observation regions that are depth-ordered with respect to the occlusion mask of the relocatable object. We represent real-world scenes as a composition of depth-ordered, interacting graphical model layers, and account for image evidence in a way that handles mutual overlap of the observation regions and their occlusions by the relocatable objects. These layers interact: proximate ground plane zones of different model instances are linked to allow a person to move between the layers, and image evidence is shared between the observation regions of these models. We demonstrate our formulation in tracking low-resolution, partially-occluded pedestrians in the vicinity of parked vehicles. In these scenarios some tracking formulations that rely on part-based person detectors may fail completely. Our pedestrian tracker fares well and compares favorably with the state-of-the-art pedestrian detectors---lowering false positives by twenty-nine percent and false negatives by forty-two percent---and a deformable-contour--based tracker

Face tracking with active models for a driver monitoring application

La falta de atención durante la conducción es una de las principales causas de accidentes de tráfico. La \ud \ud monitorización del conductor para detectar inatención es un problema complejo, que incluye elementos fisiológicos y de \ud \ud comportamiento. Un sistema de Visión Computacional para detección de inatención se compone de varios etapas de procesado, y \ud \ud esta tesis se centra en el seguimiento de la cara del conductor. La tesis doctoral propone un nuevo conjunto de vídeos de \ud \ud conductores, grabados en un vehículo real y en dos simuladores realistas, que contienen la mayoría de los comportamientos \ud \ud presentes en la conducción, incluyendo gestos, giros de cabeza, interacción con el sistema de sonido y otras distracciones, \ud \ud y somnolencia. Esta base de datos, RS-DMV, se emplea para evaluar el rendimiento de los métodos que propone la tesis y \ud \ud otros del estado del arte. La tesis analiza el rendimiento de los Modelos Activos de Forma (ASM), y de los Modelos Locales \ud \ud Restringidos (CLM), por considerarlos a priori de interés. En concreto, se ha evaluado el método Stacked Trimmed ASM \ud \ud (STASM), que integra una serie de mejoras sobre el ASM original, mostrando una alta precisión en todas las pruebas cuando \ud \ud la cara es frontal a la cámara, si bien no funciona con la cara girada y su velocidad de ejecución es muy baja. CLM es \ud \ud capaz de ejecutarse con mayor rapidez, pero tiene una precisión mucho menor en todos los casos. El tercer método a evaluar \ud \ud es el Modelado y Seguimiento Simultáneo (SMAT), que caracteriza la forma y la textura de manera incremental, a partir de \ud \ud muestras encontradas previamente. La textura alrededor de cada punto de la forma que define la cara se modela mediante un \ud \ud conjunto de grupos (clusters) de muestras pasadas. El trabajo de tesis propone 3 métodos de clustering alternativos al \ud \ud original para la textura, y un modelo de forma entrenado off-line con una función de ajuste robusta. Los métodos \ud \ud alternativos propuestos obtienen una amplia mejora tanto en la precisión del seguimiento como en la robustez de éste frente \ud \ud a giros de cabeza, oclusiones, gestos y cambios de iluminación. Los métodos propuestos tienen, además, una baja carga \ud \ud computacional, y son capaces de ejecutarse a velocidades en torno a 100 imágenes por segundo en un computador de sobremesa