1 research outputs found
Lagrangian Motion Magnification with Double Sparse Optical Flow Decomposition
Motion magnification techniques aim at amplifying and hence revealing subtle
motion in videos. There are basically two main approaches to reach this goal,
namely via Eulerian or Lagrangian techniques. While the first one magnifies
motion implicitly by operating directly on image pixels, the Lagrangian
approach uses optical flow techniques to extract and amplify pixel
trajectories. Microexpressions are fast and spatially small facial expressions
that are difficult to detect. In this paper, we propose a novel approach for
local Lagrangian motion magnification of facial micromovements. Our
contribution is three-fold: first, we fine-tune the recurrent all-pairs field
transforms for optical flows (RAFT) deep learning approach for faces by adding
ground truth obtained from the variational dense inverse search (DIS) for
optical flow algorithm applied to the CASME II video set of faces. This enables
us to produce optical flows of facial videos in an efficient and sufficiently
accurate way. Second, since facial micromovements are both local in space and
time, we propose to approximate the optical flow field by sparse components
both in space and time leading to a double sparse decomposition. Third, we use
this decomposition to magnify micro-motions in specific areas of the face,
where we introduce a new forward warping strategy using a triangular splitting
of the image grid and barycentric interpolation of the RGB vectors at the
corners of the transformed triangles. We demonstrate the very good performance
of our approach by various examples