Sparse optical flow regularisation for real-time visual tracking

Optical flow can greatly improve the robustness of visual tracking algorithms. While dense optical flow algorithms have various applications, they can not be used for real-time solutions without resorting to GPU calculations. Furthermore, most optical flow algorithms fail in challenging lighting environments due to the violation of the brightness constraint. We propose a simple but effective iterative regularisation scheme for real-time, sparse optical flow algorithms, that is shown to be robust to sudden illumination changes and can handle large displacements. The algorithm proves to outperform well known techniques in real life video sequences, while being much faster to calculate. Our solution increases the robustness of a real-time particle filter based tracking application, consuming only a fraction of the available CPU power. Furthermore, a new and realistic optical flow dataset with annotated ground truth is created and made freely available for research purposes

A Content-Adaptive Side Information Generation Method for Distributed Video Coding

AbstractIn this paper, a content-adaptive method to generate side information at the block level is presented. First, motion compensated temporal interpolation (MCTI) algorithm is used between the reconstructed key frames at the decoder to acquire initial motion vectors. Second, the image is segmented and the edge of moving region is detected by obtained the residual frame between two consecutive key frames. Furthermore, hierarchical motion estimation (HME) and motion vector filter (MVF) are adopted for edge region and an adaptive motion vector filter (AMVF) is introduced in non-edge region to correct the false estimated motion vectors. The proposal is tested and compared with the results of the state-of-the-art DISCOVER codec and RD improvements on the set of test sequences are observed

Video Deinterlacing using Control Grid Interpolation Frameworks

abstract: Video deinterlacing is a key technique in digital video processing, particularly with the widespread usage of LCD and plasma TVs. This thesis proposes a novel spatio-temporal, non-linear video deinterlacing technique that adaptively chooses between the results from one dimensional control grid interpolation (1DCGI), vertical temporal filter (VTF) and temporal line averaging (LA). The proposed method performs better than several popular benchmarking methods in terms of both visual quality and peak signal to noise ratio (PSNR). The algorithm performs better than existing approaches like edge-based line averaging (ELA) and spatio-temporal edge-based median filtering (STELA) on fine moving edges and semi-static regions of videos, which are recognized as particularly challenging deinterlacing cases. The proposed approach also performs better than the state-of-the-art content adaptive vertical temporal filtering (CAVTF) approach. Along with the main approach several spin-off approaches are also proposed each with its own characteristics.Dissertation/ThesisM.S. Electrical Engineering 201

Contextual cropping and scaling of TV productions

This is the author's accepted manuscript. The final publication is available at Springer via http://dx.doi.org/10.1007/s11042-011-0804-3. Copyright @ Springer Science+Business Media, LLC 2011.In this paper, an application is presented which automatically adapts SDTV (Standard Definition Television) sports productions to smaller displays through intelligent cropping and scaling. It crops regions of interest of sports productions based on a smart combination of production metadata and systematic video analysis methods. This approach allows a context-based composition of cropped images. It provides a differentiation between the original SD version of the production and the processed one adapted to the requirements for mobile TV. The system has been comprehensively evaluated by comparing the outcome of the proposed method with manually and statically cropped versions, as well as with non-cropped versions. Envisaged is the integration of the tool in post-production and live workflows

Adaptive deinterlacing of video sequences using motion data

In this work an efficient motion adaptive deinterlacing method with considerable improvement in picture quality is proposed. A temporal deinterlacing method has a high performance in static images while a spatial method has a better performance in dynamic parts. In the proposed deinterlacing method, a motion adaptive interpolator combines the results of a spatial method and a temporal method based on motion activity level of video sequence. A high performance and low complexity algorithm for motion detection is introduced. This algorithm uses five consecutive interlaced video fields for motion detection. It is able to capture a wide range of motions from slow to fast. The algorithm benefits from a hierarchal structure. It starts with detecting motion in large partitions of a given field. Depending on the detected motion activity level for that partition, the motion detection algorithm might recursively be applied to sub-blocks of the original partition. Two different low pass filters are used during the motion detection to increase the algorithm accuracy. The result of motion detection is then used in the proposed motion adaptive interpolator. The performance of the proposed deinterlacing algorithm is compared to previous methods in the literature. Experimenting with several standard video sequences, the method proposed in this work shows excellent results for motion detection and deinterlacing performance

Analysis of Optical Flow Algorithms for Denoising

When a video sequence is recorded in low-light conditions, the image often become noisy. Standard methods for noise reduction have difficulties with motion. But the interesting parts in a video is often the ones that are moving, for instance a burglar captured in a surveillance video. One approach for denoising video sequences is to use temporal filtering controlled by optical flow, which describes how pixels move between two image frames. Today, there exists few studies comparing how different optical flow algorithms perform on noisy video sequences. Four different algorithms have been analyzed in the thesis. Moreover, a comparison on how well they can be used to improve the result of a temporal noise filter has been done. The conclusion of the comparison is that optical flow is useful for noise reduction. Algorithms based on patch matching and edge consistency perform better than algorithms based on color consistency. A recommendation for future work is to combine the best parts of each algorithm to develop a new optical flow algorithm, specialized on noisy image sequences. Furthermore, develop and implement a sophisticated optical flow based noise filter in camera hardware