Static scene illumination estimation from video with applications

We present a system that automatically recovers scene geometry and illumination from a video, providing a basis for various applications. Previous image based illumination estimation methods require either user interaction or external information in the form of a database. We adopt structure-from-motion and multi-view stereo for initial scene reconstruction, and then estimate an environment map represented by spherical harmonics (as these perform better than other bases). We also demonstrate several video editing applications that exploit the recovered geometry and illumination, including object insertion (e.g., for augmented reality), shadow detection, and video relighting

Evaluation of MoG Video Segmentation on GPU-based HPC System

Automated and intelligent video surveillance systems play an important role in the modern world. Since the number of various video streams that must be analyzed concurrently grows, such systems can assist humans in performing tiresome tasks. In order to be effective, video surveillance systems have to meet several requirements: they must be accurate and able to process the received video stream in real-time. A robust system should not depend on lighting conditions, illumination changes and other sources of scene variation. A common component of surveillance systems is a module that performs background estimation and foreground segmentation. The MoG (Mixture of Gaussians) algorithm is a widely used statistical technique of video segmentation. The estimation process is time-consuming, especially for complex mixture models containing many components. The work presented here focuses on the performance evaluation of MoG algorithm aiming to assess feasibility of OpenCL-based processing of high resolution video on GPU accelerated platforms

A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos

This paper presents a comparative evaluation of methods for remote heart rate estimation using face videos, i.e., given a video sequence of the face as input, methods to process it to obtain a robust estimation of the subjects heart rate at each moment. Four alternatives from the literature are tested, three based in hand crafted approaches and one based on deep learning. The methods are compared using RGB videos from the COHFACE database. Experiments show that the learning-based method achieves much better accuracy than the hand crafted ones. The low error rate achieved by the learning based model makes possible its application in real scenarios, e.g. in medical or sports environments.Comment: Accepted in "IEEE International Workshop on Medical Computing (MediComp) 2020

Empirical mode decomposition-based facial pose estimation inside video sequences

We describe a new pose-estimation algorithm via integration of the strength in both empirical mode decomposition (EMD) and mutual information. While mutual information is exploited to measure the similarity between facial images to estimate poses, EMD is exploited to decompose input facial images into a number of intrinsic mode function (IMF) components, which redistribute the effect of noise, expression changes, and illumination variations as such that, when the input facial image is described by the selected IMF components, all the negative effects can be minimized. Extensive experiments were carried out in comparisons to existing representative techniques, and the results show that the proposed algorithm achieves better pose-estimation performances with robustness to noise corruption, illumination variation, and facial expressions