3D Face tracking and gaze estimation using a monocular camera

Estimating a user’s gaze direction, one of the main novel user interaction technologies, will eventually be used for numerous applications where current methods are becoming less effective. In this paper, a new method is presented for estimating the gaze direction using Canonical Correlation Analysis (CCA), which finds a linear relationship between two datasets defining the face pose and the corresponding facial appearance changes. Afterwards, iris tracking is performed by blob detection using a 4-connected component labeling algorithm. Finally, a gaze vector is calculated based on gathered eye properties. Results obtained from datasets and real-time input confirm the robustness of this metho

Kinect Range Sensing: Structured-Light versus Time-of-Flight Kinect

Recently, the new Kinect One has been issued by Microsoft, providing the next generation of real-time range sensing devices based on the Time-of-Flight (ToF) principle. As the first Kinect version was using a structured light approach, one would expect various differences in the characteristics of the range data delivered by both devices. This paper presents a detailed and in-depth comparison between both devices. In order to conduct the comparison, we propose a framework of seven different experimental setups, which is a generic basis for evaluating range cameras such as Kinect. The experiments have been designed with the goal to capture individual effects of the Kinect devices as isolatedly as possible and in a way, that they can also be adopted, in order to apply them to any other range sensing device. The overall goal of this paper is to provide a solid insight into the pros and cons of either device. Thus, scientists that are interested in using Kinect range sensing cameras in their specific application scenario can directly assess the expected, specific benefits and potential problem of either device.Comment: 58 pages, 23 figures. Accepted for publication in Computer Vision and Image Understanding (CVIU

Statistical Region Based Segmentation of Ultrasound Images

Segmentation of ultrasound images is a challenging problem due to speckle, which corrupts the image and can result in weak or missing image boundaries, poor signal to noise ratio, and diminished contrast resolution. Speckle is a random interference pattern that is characterized by an asymmetric distribution as well as significant spatial correla- tion. These attributes of speckle are challenging to model in a segmentation approach, so many previous ultrasound segmentation methods simplify the problem by assuming that the speckle is white and/or Gaussian distributed. Unlike these methods, in this paper we present an ultrasound-specific segmentation approach that addresses both the spatial correlation of the data as well as its intensity distribution. We first decorrelate the image and then apply a region-based active contour whose motion is derived from an appropri- ate parametric distribution for maximum likelihood image segmentation. We consider zero-mean complex Gaussian, Rayleigh, and Fisher-Tippett flows, which are designed to model fully formed speckle in the in-phase/quadrature (IQ), envelope detected, and display (log compressed) images, respectively. We present experimental results demon- strating the effectiveness of our method, and compare the results to other parametric and non-parametric active contours

Method and apparatus for predicting the direction of movement in machine vision

A computer-simulated cortical network is presented. The network is capable of computing the visibility of shifts in the direction of movement. Additionally, the network can compute the following: (1) the magnitude of the position difference between the test and background patterns; (2) localized contrast differences at different spatial scales analyzed by computing temporal gradients of the difference and sum of the outputs of paired even- and odd-symmetric bandpass filters convolved with the input pattern; and (3) the direction of a test pattern moved relative to a textured background. The direction of movement of an object in the field of view of a robotic vision system is detected in accordance with nonlinear Gabor function algorithms. The movement of objects relative to their background is used to infer the 3-dimensional structure and motion of object surfaces

Three Dimensional Shape Reconstruction with Dual-camera Measurement Fusion

Recently, three-dimensional (3D) shape measurement technologies have been extensively researched in the fields such as computer science and medical engineering. They have been applied in various industries and commercial uses, including robot navigation, reverser engineering and face and gesture recognition. Optical 3D shape measurement is one of the most popular methods, which can be divided into two categories: passive 3D shape reconstruction and active 3D shape imaging. Passive 3D shape measurement techniques use cameras to capture the object with only ambient light. Stereo vision (SV) is one of the typical methods in passive 3D measurement approaches. This method uses two cameras to take photos of the scene from different viewpoints and extract the 3D information by establishing the correspondence between the photos captured. To translate the correspondence to the depth map, epipolar geometry is applied to determine the depth of each pixel. Active 3D shape imaging methods add diverse active light sources to project on the object and use the camera to capture the scene with pre-defined patterns on the object’s surface. The fringe projection profilometry (FPP) is a representative technique among active 3D reconstruction methods. It replaces one of the cameras in stereo vision with a projector, and projects the fringe patterns onto the object before the camera captures it. The depth map can be built via triangulations by analysing the phase difference between patterns distorted by the object’s surface and the original one. Those two mainstream techniques work alone in different scenarios and have various advantages and disadvantages. Active stereo vision (ASV) has excellent dynamic performance, yet its accuracy and spatial resolution are limited. On the other hand, 3D shape measurement methods like FPP have higher accuracy and speed; however, their dynamic performance varies depending on the codification schemes chosen. This thesis presents the research on developing a fusion method that contains both passive and active 3D shape reconstruction algorithms in one system to combine their advantages and reduce the budget of building a high-precision 3D shape measurement system with good dynamic performance. Specifically, in the thesis, we propose a fusion method that combines the epipolar geometry in ASV and triangulations in the FPP system by a specially designed cost function. This way, the information obtained from each system alone is combined, leading to better accuracy. Furthermore, the correlation of object surface is exploited with the autoregressive model to improve the precision of the fusion system. In addition, the expectation maximization framework is employed to address the issue of estimating variables with unknown parameters introduced by AR. Moreover, the fusion cost function derived before is embedded into the EM framework. Next, the message passing algorithm is applied to implement the EM efficiently on large image sizes. A factor graph is derived from fitting the EM approach. To implement belief propagation to solve the problem, it is divided into two sub-graphs: the E-Step factor graph and the M-Step factor graph. Based on two factor graphs, belief propagation is implemented on each of them to estimate the unknown parameters and EM messages. In the last iteration, the height of the object surface can be obtained with the forward and backward messages. Due to the consideration of the object’s surface correlation, the fusion system’s precision is further improved. Simulation and experimental results are presented at last to examine the performance of the proposed system. It is found that the accuracy of the depth map of the fusion method is improved compared to fringe projection profilometry or stereo vision system alone. The limitations of the current study are discussed, and potential future work is presented

Automatic segmentation of the left ventricle cavity and myocardium in MRI data

A novel approach for the automatic segmentation has been developed to extract the epi-cardium and endo-cardium boundaries of the left ventricle (lv) of the heart. The developed segmentation scheme takes multi-slice and multi-phase magnetic resonance (MR) images of the heart, transversing the short-axis length from the base to the apex. Each image is taken at one instance in the heart's phase. The images are segmented using a diffusion-based filter followed by an unsupervised clustering technique and the resulting labels are checked to locate the (lv) cavity. From cardiac anatomy, the closest pool of blood to the lv cavity is the right ventricle cavity. The wall between these two blood-pools (interventricular septum) is measured to give an approximate thickness for the myocardium. This value is used when a radial search is performed on a gradient image to find appropriate robust segments of the epi-cardium boundary. The robust edge segments are then joined using a normal spline curve. Experimental results are presented with very encouraging qualitative and quantitative results and a comparison is made against the state-of-the art level-sets method