Riesz pyramids for fast phase-based video magnification

We present a new compact image pyramid representation, the Riesz pyramid, that can be used for real-time phase-based motion magnification. Our new representation is less overcomplete than even the smallest two orientation, octave-bandwidth complex steerable pyramid, and can be implemented using compact, efficient linear filters in the spatial domain. Motion-magnified videos produced with this new representation are of comparable quality to those produced with the complex steerable pyramid. When used with phase-based video magnification, the Riesz pyramid phase-shifts image features along only their dominant orientation rather than every orientation like the complex steerable pyramid.Quanta Computer (Firm)Shell ResearchNational Science Foundation (U.S.) (CGV-1111415)Microsoft Research (PhD Fellowship)Massachusetts Institute of Technology. Department of MathematicsNational Science Foundation (U.S.). Graduate Research Fellowship (Grant 1122374

Applications of Phase-Based Motion Processing

Image pyramids provide useful information in determining structural response at low cost using commercially available cameras. The current effort applies previous work on the complex steerable pyramid to analyze and identify imperceptible linear motions in video. Instead of implicitly computing motion spectra through phase analysis of the complex steerable pyramid and magnifying the associated motions, instead present a visual technique and the necessary software to display the phase changes of high frequency signals within video. The present technique quickly identifies regions of largest motion within a video with a single phase visualization and without the artifacts of motion magnification, but requires use of the computationally intensive Fourier transform. While Riesz pyramids present an alternative to the computationally intensive complex steerable pyramid for motion magnification, the Riesz formulation contains significant noise, and motion magnification still presents large amounts of data that cannot be quickly assessed by the human eye. Thus, user-friendly software is presented for quickly identifying structural response through optical flow and phase visualization in both Python and MATLAB

Developed approach for phase-based Eulerian video magnification

This paper proposes a modification approach for phased-based EVM in order to reduce the processing time without effect the quality of the magnified video. The proposed approach applies a resizing process on the input video using Lanczos-3 algorithm. Then, it decomposes video frames using steerable pyramid to obtain multi-scale frame with its orientation. Subsequently, the resulted frames are filtered by temporal filters for specific bands and the filtered frames are multiplied by a magnification factor. Now, both the magnified regions and the unmagnified regions for each frame are added together. Finally, reconstructing the produced magnified multi-scale frames using the inverse steerable pyramid. The experimental results show that superiority of the proposed approach compares to the conventional phase-based EVM in processing time, where the processing time reduction about 60-65%. Furthermore, this approach does not affect on the video quality, which maintain it in the boundary of the conventional Phase-based EVM

Video Magnification for Structural Analysis Testing

The goal of this thesis is to allow a user to see minute motion of an object at different frequencies, using a computer program, to aid in vibration testing analysis without the use of complex setups of accelerometers or expensive laser vibrometers. MIT’s phase-based video motion processing ­was modified to enable modal determination of structures in the field using a cell phone camera. The algorithm was modified by implementing a stabilization algorithm and permitting the magnification filter to operate on multiple frequency ranges to enable visualization of the natural frequencies of structures in the field. To implement multiple frequency ranges a new function was developed to implement the magnification filter at each relevant frequency range within the original video. The stabilization algorithm would allow for a camera to be hand-held instead of requiring a tripod mount. The following methods for stabilization were tested: fixed point video stabilization and image registration. Neither method removed the global motion from the hand-held video, even after masking was implemented, which resulted in poor results. Specifically, fixed point did not remove much motion or created sharp motions and image registration introduced a pulsing effect. The best results occurred when the object being observed had contrast from the background, was the largest feature in the video frame, and the video was captured from a tripod at an appropriate angle. The final program can amplify the motion in user selected frequency bands and can be used as an aid in structural analysis testing

Video Acceleration Magnification

The ability to amplify or reduce subtle image changes over time is useful in contexts such as video editing, medical video analysis, product quality control and sports. In these contexts there is often large motion present which severely distorts current video amplification methods that magnify change linearly. In this work we propose a method to cope with large motions while still magnifying small changes. We make the following two observations: i) large motions are linear on the temporal scale of the small changes; ii) small changes deviate from this linearity. We ignore linear motion and propose to magnify acceleration. Our method is pure Eulerian and does not require any optical flow, temporal alignment or region annotations. We link temporal second-order derivative filtering to spatial acceleration magnification. We apply our method to moving objects where we show motion magnification and color magnification. We provide quantitative as well as qualitative evidence for our method while comparing to the state-of-the-art.Comment: Accepted paper at CVPR 2017. Project webpage: http://acceleration-magnification.github.io

Quaternionic Representation of the Riesz Pyramid for Video Magnification

Recently, we presented a new image pyramid, called the Riesz pyramid, that uses the Riesz transform to manipulate the phase in non-oriented sub-bands of an image sequence to produce real-time motion-magnified videos. In this report we give a quaternionic formulation of the Riesz pyramid, and show how several seemingly heuristic choices in how to use the Riesz transform for phase-based video magnification fall out of this formulation in a natural and principled way. We intend this report to accompany the original paper on the Riesz pyramid for video magnification

Simultaneous magnification of subtle motions and color variations in videos using riesz pyramids

Videos often contain subtle motions and color variations that cannot be easily observed. Examples include, for instance, head motion and changes in skin face color due to blood flow controlled by the heart pumping rhythm. A few techniques have been developed to magnify these subtle signals. However, they are not easily applied to many applications. First of all, previous techniques were targeted specifically towards magnification of either motion or color variations. Trying to magnify both aspects applying two of these tech niques in sequence does not produce good results. We present a method for magnifying subtle motions and color variations in videos simultaneously. Our approach is based on the Riesz pyramid, which was previously used only for motion magnification. Besides modifying the local phases of the coefficients of this pyramid, we show how altering its local amplitudes and its residue can be used to produce intensity (color) magnification. We demonstrate the effectiveness of our technique in multiple videos by revealing both subtle signals simultaneously. Finally, we also developed an Android application as a proof-of-concept that can be used for magnifying either motion or color changes

Efficient denoising approach based eulerian video magnification for colour and motion variations

Digital video magnification is a computer-based microscope, which is useful to detect subtle changes to human eyes in recorded videos. This technology can be employed in several areas such as medical, biological, mechanical and physical applications. Eulerian is the most popular approach in video magnification. However, amplifying the subtle changes in video produces amplifying the subtle noise. This paper proposes an approach to reduce amplified noise in magnified video for both type of changes amplifications, color and motion. The proposed approach processes the resulted video from Eulerian algorithm whether linear or phase based in order to noise cancellation. The approach utilizes wavelet denoising method to localize the frequencies of distributed noise over the different frequency bands. Subsequently, the energy of the coefficients under localized frequencies are attenuated by attenuating the amplitude of these coefficients. The experimental results of the proposed approach show its superiority over conventional linear and phase based Eulerian video magnification approaches in terms of quality of the resulted magnified videos. This allows to amplify the videos by larger amplification factor, so that several new applications can be added to the list of Eulerian video magnification users. Furthermore, the processing time does not significantly increase, the increment is only less than 3% of the overall processing compare to conventional Eulerian video magnification

An experimental study of the feasibility of phase‐based video magnification for damage detection and localisation in operational deflection shapes

Optical measurements from high‐speed, high‐definition video recordings can be used to define the full‐field dynamics of a structure. By comparing the dynamic responses resulting from both damaged and undamaged elements, structural health monitoring can be carried out, similarly as with mounted transducers. Unlike the physical sensors, which provide point‐wise measurements and a limited number of output channels, high‐quality video recording allows very spatially dense information. Moreover, video acquisition is a noncontact technique. This guarantees that any anomaly in the dynamic behaviour can be more easily correlated to damage and not to added mass or stiffness due to the installed sensors. However, in real‐life scenarios, the vibrations due to environmental input are often so small that they are indistinguishable from measurement noise if conventional image‐based techniques are applied. In order to improve the signal‐to‐noise ratio in low‐amplitude measurements, phase‐based motion magnification has been recently proposed. This study intends to show that model‐based structural health monitoring can be performed on modal data and time histories processed with phase‐based motion magnification, whereas unamplified vibrations would be too small for being successfully exploited. All the experiments were performed on a multidamaged box beam with different damage sizes and angles