Context-aware Synthesis for Video Frame Interpolation

Video frame interpolation algorithms typically estimate optical flow or its variations and then use it to guide the synthesis of an intermediate frame between two consecutive original frames. To handle challenges like occlusion, bidirectional flow between the two input frames is often estimated and used to warp and blend the input frames. However, how to effectively blend the two warped frames still remains a challenging problem. This paper presents a context-aware synthesis approach that warps not only the input frames but also their pixel-wise contextual information and uses them to interpolate a high-quality intermediate frame. Specifically, we first use a pre-trained neural network to extract per-pixel contextual information for input frames. We then employ a state-of-the-art optical flow algorithm to estimate bidirectional flow between them and pre-warp both input frames and their context maps. Finally, unlike common approaches that blend the pre-warped frames, our method feeds them and their context maps to a video frame synthesis neural network to produce the interpolated frame in a context-aware fashion. Our neural network is fully convolutional and is trained end to end. Our experiments show that our method can handle challenging scenarios such as occlusion and large motion and outperforms representative state-of-the-art approaches.Comment: CVPR 2018, http://graphics.cs.pdx.edu/project/ctxsy

Parallel numerical modeling of hybrid-dimensional compositional non-isothermal Darcy flows in fractured porous media

This paper introduces a new discrete fracture model accounting for non-isothermal compositional multiphase Darcy flows and complex networks of fractures with intersecting, immersed and non immersed fractures. The so called hybrid-dimensional model using a 2D model in the fractures coupled with a 3D model in the matrix is first derived rigorously starting from the equi-dimensional matrix fracture model. Then, it is dis-cretized using a fully implicit time integration combined with the Vertex Approximate Gradient (VAG) finite volume scheme which is adapted to polyhedral meshes and anisotropic heterogeneous media. The fully coupled systems are assembled and solved in parallel using the Single Program Multiple Data (SPMD) paradigm with one layer of ghost cells. This strategy allows for a local assembly of the discrete systems. An efficient preconditioner is implemented to solve the linear systems at each time step and each Newton type iteration of the simulation. The numerical efficiency of our approach is assessed on different meshes, fracture networks, and physical settings in terms of parallel scalability, nonlinear convergence and linear convergence

Video Frame Interpolation via Adaptive Separable Convolution

Standard video frame interpolation methods first estimate optical flow between input frames and then synthesize an intermediate frame guided by motion. Recent approaches merge these two steps into a single convolution process by convolving input frames with spatially adaptive kernels that account for motion and re-sampling simultaneously. These methods require large kernels to handle large motion, which limits the number of pixels whose kernels can be estimated at once due to the large memory demand. To address this problem, this paper formulates frame interpolation as local separable convolution over input frames using pairs of 1D kernels. Compared to regular 2D kernels, the 1D kernels require significantly fewer parameters to be estimated. Our method develops a deep fully convolutional neural network that takes two input frames and estimates pairs of 1D kernels for all pixels simultaneously. Since our method is able to estimate kernels and synthesizes the whole video frame at once, it allows for the incorporation of perceptual loss to train the neural network to produce visually pleasing frames. This deep neural network is trained end-to-end using widely available video data without any human annotation. Both qualitative and quantitative experiments show that our method provides a practical solution to high-quality video frame interpolation.Comment: ICCV 2017, http://graphics.cs.pdx.edu/project/sepconv

Adding memory to pressure-sensitive phosphors

Mechanoluminescence (ML) is the phenomenon describing the emission of light during mechanical action on a solid, leading to applications such as pressure sensing, damage detection and visualization of stress distributions. In most cases, this mechanical action releases energy that was previously stored in the crystal lattice of the phosphor by means of trapped charge carriers. A drawback is the need to record the ML emission during a pressure event. In this work, we provide a method for adding a memory function to these pressure-sensitive phosphors, allowing an optical readout of the location and intensity of a pressure event in excess of 72 h after the event. This is achieved in the BaSi2O2N2:Eu2+ phosphor, where a broad trap depth distribution essential for the process is present. By merging optically stimulated luminescence (OSL), thermoluminescence (TL) and ML measurements, the influence of light, heat and pressure on the trap depth distribution is carefully analysed. This analysis demonstrates that mechanical action can not only lead to direct light emission but also to a reshuffling of trap occupations. This memory effect not only is expected to lead to new pressure sensing applications but also offers an approach to study charge carrier transitions in energy storage phosphors

Multiple-sensor integration for efficient reverse engineering of geometry

This paper describes a multi-sensor measuring system for reverse engineering applications. A sphere-plate artefact is developed for data unification of the hybrid system. With the coordinate data acquired using the optical system, intelligent feature recognition and segmentation algorithms can be applied to extract the global surface information of the object. The coordinate measuring machine (CMM) is used to re-measure the geometric features with a small amount of sampling points and the obtained information can be subsequently used to compensate the point data patches which are measured by optical system. Then the optimized point data can be exploited for accurate reverse engineering of CAD model. The limitations of each measurement system are compensated by the other. Experimental results validate the accuracy and effectiveness of this data optimization approach