A Convolutional Neural Network Approach for Half-Pel Interpolation in Video Coding

Motion compensation is a fundamental technology in video coding to remove the temporal redundancy between video frames. To further improve the coding efficiency, sub-pel motion compensation has been utilized, which requires interpolation of fractional samples. The video coding standards usually adopt fixed interpolation filters that are derived from the signal processing theory. However, as video signal is not stationary, the fixed interpolation filters may turn out less efficient. Inspired by the great success of convolutional neural network (CNN) in computer vision, we propose to design a CNN-based interpolation filter (CNNIF) for video coding. Different from previous studies, one difficulty for training CNNIF is the lack of ground-truth since the fractional samples are actually not available. Our solution for this problem is to derive the "ground-truth" of fractional samples by smoothing high-resolution images, which is verified to be effective by the conducted experiments. Compared to the fixed half-pel interpolation filter for luma in High Efficiency Video Coding (HEVC), our proposed CNNIF achieves up to 3.2% and on average 0.9% BD-rate reduction under low-delay P configuration.Comment: International Symposium on Circuits and Systems (ISCAS) 201

Statistical framework for video decoding complexity modeling and prediction

Video decoding complexity modeling and prediction is an increasingly important issue for efficient resource utilization in a variety of applications, including task scheduling, receiver-driven complexity shaping, and adaptive dynamic voltage scaling. In this paper we present a novel view of this problem based on a statistical framework perspective. We explore the statistical structure (clustering) of the execution time required by each video decoder module (entropy decoding, motion compensation, etc.) in conjunction with complexity features that are easily extractable at encoding time (representing the properties of each module's input source data). For this purpose, we employ Gaussian mixture models (GMMs) and an expectation-maximization algorithm to estimate the joint execution-time - feature probability density function (PDF). A training set of typical video sequences is used for this purpose in an offline estimation process. The obtained GMM representation is used in conjunction with the complexity features of new video sequences to predict the execution time required for the decoding of these sequences. Several prediction approaches are discussed and compared. The potential mismatch between the training set and new video content is addressed by adaptive online joint-PDF re-estimation. An experimental comparison is performed to evaluate the different approaches and compare the proposed prediction scheme with related resource prediction schemes from the literature. The usefulness of the proposed complexity-prediction approaches is demonstrated in an application of rate-distortion-complexity optimized decoding

Maxwell-compensated design of asymmetric gradient waveforms for tensor-valued diffusion encoding

Purpose: Asymmetric gradient waveforms are attractive for diffusion encoding due to their superior efficiency, however, the asymmetry may cause a residual gradient moment at the end of the encoding. Depending on the experiment setup, this residual moment may cause significant signal bias and image artifacts. The purpose of this study was to develop an asymmetric gradient waveform design for tensor-valued diffusion encoding that is not affected by concomitant gradient. Methods: The Maxwell index was proposed as a scalar invariant that captures the effect of concomitant gradients and was constrained in the numerical optimization to 100 (mT/m)$^2$ms to yield Maxwell-compensated waveforms. The efficacy of this design was tested in an oil phantom, and in a healthy human brain. For reference, waveforms from literature were included in the analysis. Simulations were performed to investigate if the design was valid for a wide range of experiments and if it could predict the signal bias. Results: Maxwell-compensated waveforms showed no signal bias in oil or in the brain. By contrast, several waveforms from literature showed gross signal bias. In the brain, the bias was large enough to markedly affect both signal and parameter maps, and the bias could be accurately predicted by theory. Conclusion: Constraining the Maxwell index in the optimization of asymmetric gradient waveforms yields efficient tensor-valued encoding with concomitant gradients that have a negligible effect on the signal. This waveform design is especially relevant in combination with strong gradients, long encoding times, thick slices, simultaneous multi-slice acquisition and large/oblique FOVs

Lagrangian Structure Functions in Turbulence: A Quantitative Comparison between Experiment and Direct Numerical Simulation

A detailed comparison between data from experimental measurements and numerical simulations of Lagrangian velocity structure functions in turbulence is presented. By integrating information from experiments and numerics, a quantitative understanding of the velocity scaling properties over a wide range of time scales and Reynolds numbers is achieved. The local scaling properties of the Lagrangian velocity increments for the experimental and numerical data are in good quantitative agreement for all time lags. The degree of intermittency changes when measured close to the Kolmogorov time scales or at larger time lags. This study resolves apparent disagreements between experiment and numerics.Comment: 13 RevTeX pages (2 columns) + 8 figures include

A Framework for Temperature Imaging using the Change in Backscattered Ultrasonic Signals

Hyperthermia is a cancer treatment that elevates tissue temperature to 40 to 43oC. It would benefit from a non-invasive, safe, inexpensive and convenient thermometry to monitor heating patterns. Ultrasound is a modality that meets these requirements. In our initial work, using both prediction and experimental data, we showed that the change in the backscattered energy: CBE) is a potential parameter for TI. CBE, however, was computed in a straightforward yet ad hoc manner. In this work, we developed and exploited a mathematical representation for our approach to TI to optimize temperature accuracy. Non-thermal effects of noise and motion confound the use of CBE. Assuming additive white Gaussian noise, we applied signal averaging and thresholding to reduce noise effects. Our motion compensation algorithms were also applied to images with known motion to evaluate factors affecting the compensation performance. In the framework development, temperature imaging was modeled as a problem of estimating temperature from the random processes resulting from thermal changes in signals. CBE computation was formalized as a ratio between two random variables. Mutual information: MI) was studied as an example of possible parameters for temperature imaging based on the joint distributions. Furthermore, a maximum likelihood estimator: MLE) was developed. Both simulations and experimental results showed that noise effects were reduced by signal averaging. The motion compensation algorithms proved to be able to compensate for motion in images and were improved by choosing appropriate interpolation methods and sample rates. For images of uniformly distributed scatterers, CBE and MI can be computed independent of SNR to improve the temperature accuracy. The application of the MLE also showed improvements in temperature accuracy compared to the energy ratio from the signal mean in simulations. The application of the framework to experimental data requires more work to implement noise reduction approaches in 3D heating experiments. The framework identified ways in which we were able to reduce the effects of both noise and motion. The framework formalized our approaches to temperature imaging, improved temperature accuracy in simulations, and can be applied to experimental data if the noise reduction approaches can be implemented for 3D experiments