Eulerian video magnification for revealing subtle changes in the world

Our goal is to reveal temporal variations in videos that are difficult or impossible to see with the naked eye and display them in an indicative manner. Our method, which we call Eulerian Video Magnification, takes a standard video sequence as input, and applies spatial decomposition, followed by temporal filtering to the frames. The resulting signal is then amplified to reveal hidden information. Using our method, we are able to visualize the flow of blood as it fills the face and also to amplify and reveal small motions. Our technique can run in real time to show phenomena occurring at the temporal frequencies selected by the user.United States. Defense Advanced Research Projects Agency (DARPA SCENICC program)National Science Foundation (U.S.) (NSF CGV-1111415)Quanta Computer (Firm)Nvidia Corporation (Graduate Fellowship

Video Acceleration Magnification

The ability to amplify or reduce subtle image changes over time is useful in contexts such as video editing, medical video analysis, product quality control and sports. In these contexts there is often large motion present which severely distorts current video amplification methods that magnify change linearly. In this work we propose a method to cope with large motions while still magnifying small changes. We make the following two observations: i) large motions are linear on the temporal scale of the small changes; ii) small changes deviate from this linearity. We ignore linear motion and propose to magnify acceleration. Our method is pure Eulerian and does not require any optical flow, temporal alignment or region annotations. We link temporal second-order derivative filtering to spatial acceleration magnification. We apply our method to moving objects where we show motion magnification and color magnification. We provide quantitative as well as qualitative evidence for our method while comparing to the state-of-the-art.Comment: Accepted paper at CVPR 2017. Project webpage: http://acceleration-magnification.github.io

Micro-expression Recognition using Spatiotemporal Texture Map and Motion Magnification

Micro-expressions are short-lived, rapid facial expressions that are exhibited by individuals when they are in high stakes situations. Studying these micro-expressions is important as these cannot be modified by an individual and hence offer us a peek into what the individual is actually feeling and thinking as opposed to what he/she is trying to portray. The spotting and recognition of micro-expressions has applications in the fields of criminal investigation, psychotherapy, education etc. However due to micro-expressions’ short-lived and rapid nature; spotting, recognizing and classifying them is a major challenge. In this paper, we design a hybrid approach for spotting and recognizing micro-expressions by utilizing motion magnification using Eulerian Video Magnification and Spatiotemporal Texture Map (STTM). The validation of this approach was done on the spontaneous micro-expression dataset, CASMEII in comparison with the baseline. This approach achieved an accuracy of 80% viz. an increase by 5% as compared to the existing baseline by utilizing 10-fold cross validation using Support Vector Machines (SVM) with a linear kernel

Efficient denoising approach based eulerian video magnification for colour and motion variations

Digital video magnification is a computer-based microscope, which is useful to detect subtle changes to human eyes in recorded videos. This technology can be employed in several areas such as medical, biological, mechanical and physical applications. Eulerian is the most popular approach in video magnification. However, amplifying the subtle changes in video produces amplifying the subtle noise. This paper proposes an approach to reduce amplified noise in magnified video for both type of changes amplifications, color and motion. The proposed approach processes the resulted video from Eulerian algorithm whether linear or phase based in order to noise cancellation. The approach utilizes wavelet denoising method to localize the frequencies of distributed noise over the different frequency bands. Subsequently, the energy of the coefficients under localized frequencies are attenuated by attenuating the amplitude of these coefficients. The experimental results of the proposed approach show its superiority over conventional linear and phase based Eulerian video magnification approaches in terms of quality of the resulted magnified videos. This allows to amplify the videos by larger amplification factor, so that several new applications can be added to the list of Eulerian video magnification users. Furthermore, the processing time does not significantly increase, the increment is only less than 3% of the overall processing compare to conventional Eulerian video magnification

Using Eulerian video magnification to enhance detection of fasciculations in people with amyotrophic lateral sclerosis

Introduction: This study seeks to determine whether the use of Eulerian video magnification (EVM) increases the detection of muscle fasciculations in people with amyotrophic lateral sclerosis (PALS) compared with direct clinical observation (DCO). Methods: Thirty-second-long video recordings were taken of 9 body regions of 7 PALS and 7 controls, and fasciculations were counted by DCO during the same 30-s period. The video recordings were then motion magnified and reviewed by 2 independent assessors. Results: In PALS, median fasciculation count per body region was 1 by DCO (range 0–10) and 3 in the EVM recordings (range 0–15; P < 0.0001). EVM revealed more fasciculations than DCO in 61% of recordings. In controls, median fasciculation count was 0 for both DCO and EVM. Discussion: Compared with DCO, EVM significantly increased the detection of fasciculations in body regions of PALS. When it is used to supplement clinical examination, EVM has the potential to facilitate the diagnosis of ALS. Muscle Nerve 56: 1063–1067, 201

Phase-based video motion processing

We introduce a technique to manipulate small movements in videos based on an analysis of motion in complex-valued image pyramids. Phase variations of the coefficients of a complex-valued steerable pyramid over time correspond to motion, and can be temporally processed and amplified to reveal imperceptible motions, or attenuated to remove distracting changes. This processing does not involve the computation of optical flow, and in comparison to the previous Eulerian Video Magnification method it supports larger amplification factors and is significantly less sensitive to noise. These improved capabilities broaden the set of applications for motion processing in videos. We demonstrate the advantages of this approach on synthetic and natural video sequences, and explore applications in scientific analysis, visualization and video enhancement.Shell ResearchUnited States. Defense Advanced Research Projects Agency. Soldier Centric Imaging via Computational CamerasNational Science Foundation (U.S.) (CGV-1111415)Cognex CorporationMicrosoft Research (PhD Fellowship)American Society for Engineering Education. National Defense Science and Engineering Graduate Fellowshi

Eulerian Phase-based Motion Magnification for High-Fidelity Vital Sign Estimation with Radar in Clinical Settings

Efficient and accurate detection of subtle motion generated from small objects in noisy environments, as needed for vital sign monitoring, is challenging, but can be substantially improved with magnification. We developed a complex Gabor filter-based decomposition method to amplify phases at different spatial wavelength levels to magnify motion and extract 1D motion signals for fundamental frequency estimation. The phase-based complex Gabor filter outputs are processed and then used to train machine learning models that predict respiration and heart rate with greater accuracy. We show that our proposed technique performs better than the conventional temporal FFT-based method in clinical settings, such as sleep laboratories and emergency departments, as well for a variety of human postures.Comment: Accepted in IEEE Sensors 202