A Parallel framework for video super-resolution

Advisor: Mylène C.Q. Farias. Date and location of PhD thesis defense: 19 February 2013, University of BrasíliaIn this work we propose a framework for increasing the processing efficiency of super-resolution algorithms. The framework is targeted at super-resolution video processing algorithms, that require a large amount of data processing. We propose a set of strategies that use a combination of data simplification and parallel processing. The simplification strategies are used to decrease the amount of complex data and, consequently, decrease the processing time. The parallel processing strategies are designed so that major modifications of the super-resolution algorithms are not required. As presented in this work, the framework is fast and makes the video resolution increase timely

A Parallel framework for video super-resolution

In this work we propose a framework for increasing the processing efficiency of super-resolution algorithms. The framework is targeted at super-resolution video processing algorithms, that require a large amount of data processing. We propose a set of strategies that use a combination of data simplification and parallel processing. The simplification strategies are used to decrease the amount of complex data and, consequently, decrease the processing time. The parallel processing strategies are designed so that major modifications of the super-resolution algorithms are not required. As presented in this work, the framework is fast and makes the video resolution increase timely

PROCESSING VIDEOS USING PARALLEL COMPUTING: A NOVEL APPROACH

In this paper, the proposed framework is presented that supports acquiring high-resolution video’s from the low-resolution. The high-resolution videos could be used in tagging, identifying and tracking people. We concentrate on two aspects. One, data simplification method as the algorithm required for conversion large amount of data processing which is run in parallel. Second, is building a parallel video processing techniques as pipeline for analyzing image modules such as face detection, recognition and tracking so that multiple people can be identified more efficiently and smoothly with increased performance and computational efficiency. Parallel processing techniques makes the use of super resolution algorithm obsolete for major modification in generating high-resolution video images. Recognition of multiple people with super resolution can be tracked from real time live videos or it could be recorded on

A new bandwidth adaptive non-local kernel regression algorithm for image/video restoration and its GPU realization

This paper presents a new bandwidth adaptive nonlocal kernel regression (BA-NLKR) algorithm for image and video restoration. NLKR is a recent approach for improving the performance of conventional steering kernel regression (SKR) and local polynomial regression (LPR) in image/video processing. Its bandwidth, which controls the amount of smoothing, however is chosen empirically. The proposed algorithm incorporates the intersecting confidence intervals (ICI) bandwidth selection method into the framework of NLKR to facilitate automatic bandwidth selection so as to achieve better performance. A parallel implementation of the proposed algorithm is also introduced to reduce significantly its computation time. The effectiveness of the proposed algorithm is illustrated by experimental results on both single image and videos super resolution and denoising.published_or_final_versio

Um framework para processamento paralelo de algoritmos de aumento de resolução de vídeos

Dissertação (mestrado)—Universidade de Brasília, Instituto de Ciências Exatas, Departamento de Ciência da Computação, 2013.O aumento dimensional de sinais visuais consiste na alteração do tamanho de uma imagem ou de um vídeo para dimensões espaciais maiores, utilizando técnicas de processa- mento digital de sinais. Geralmente, esse aumento é feito com a utilização de técnicas de interpolação. Contudo, essas técnicas de interpolação produzem distorções nas imagens au- mentadas. Tais distorções ocorrem porque a imagem aumentada possui apenas as amostras da imagem original, de dimensões menores, que são insu cientes para reconstrução exata do sinal, o que gera efeitos de aliasing. Assim sendo, as técnicas de interpolação apenas estimam os coe cientes não-amostrados do sinal, o que muitas vezes produz resultados insatisfatórios para muitas aplicações, necessitando de outras técnicas para reconstituir os coe cientes não-amostrados com maior precisão. Para melhorar a aproximação de uma imagem estimada com relação à imagem origi- nal, existem técnicas que reconstroem os coe cientes não-amostrados. Essas técnicas são chamadas de super-resolução. Elas consistem em aumentar a resolução utilizando, geral- mente, informações de outras imagens em baixa ou alta-resolução para estimar a informação faltante na imagem que se deseja ampliar. Super-resolução é um processo computacionalmente intenso, onde a complexidade dos algoritmos são, geralmente, de ordem exponencial no tempo em função do bloco ou do fa- tor de ampliação. Portanto, quando essas técnicas são aplicadas para vídeos, é necessário que o algoritmo seja extremamente rápido. O problema é que os algoritmos mais com- putacionalmente e cientes, nem sempre são aqueles que produzem os melhores resultados visuais. Sendo assim, este trabalho propõe um framework para melhorar o desempenho de diversos algoritmos de super-resolução através de estratégias de processamento seletivo e paralelo. Para isso, nesta dissertação são examinadas as propriedades dos resultados produzidos pelos algoritmos de super-resolução e os resultados produzidos utilizando-se técnicas de interpolação. Com essas propriedades, é encontrado um critério para classi car as regiões em que os resultados produzidos sejam visualmente equivalentes, não importando o método utilizado para ampliação. Nessas regiões de equivalência utiliza-se um algoritmo de interpolação, que é muito mais veloz do que os computacionalmente complexos de super-resolução. Assim, consegue-se reduzir o tempo de processamento sem prejudicar a qualidade visual do vídeo ampliado. Além dessa abordagem, este trabalho também propõe uma estratégia de divisão de dados entre diferentes tarefas para que a operação de aumento de resolução seja realizada de forma paralela. Um resultado interessante do modelo proposto é que ele desacopla a abstração de distribuição de carga da função de aumento dimensional. Em outras palavras, diferentes métodos de super-resolução podem explorar os recursos do framework sem que para isso seus algoritmos precisem ser modi cados para obtenção do paralelismo. Isso torna o framework portável, escalável e reusável por diferentes métodos de super-resolução. ______________________________________________________________________________ ABSTRACTThe magni cation of visual signals consists of changing the size of an image or a video to larger spatial dimensions, using digital signal processing techniques. Usually, this mag- ni cation is done using numerical interpolation methods. However, these interpolation methods tend to produce some distortions in the increased images. Such distortions oc- cours because the interpolated image is reconstructed using only the original image samples, which are insu cients for the accurate signal reconstruction, generating aliasing e ects. These interpolation techniques only approximate the non-sampled signal coe cients, pro- ducing unsatisfactory results for many applications. Thus, for these applications, others techniques to estimate the non-sampled coe cients are needed. To improve the estimation accuracy of an image with respect to the original, the super- resolution techniques are used to reconstruct the non-sampled coe cients. Generally, these super-resolution techniques enhance the increased image using information of other images to estimate the missing information. Super-resolution is a computationally intensive process, where the algorithms com- plexity are, generally, exponential in time as function of the block size or magni cation factor. Therefore, when these techniques are applied for videos, it is required that the super-resolution algorithm be extremely fast. However, more computationally e cient algorithms are not always those that produce the best visual results. Therefore, this work proposes a framework to improve the performance of various super- resolution algorithms using selective processing and parallel processing strategies. Thus, this dissertation examines the properties of the results produced by the super-resolution algorithms and the results produced by using interpolation techniques. From these proper- ties, is achieved a criterion to classify regions wherein the results produced are equivalent (using both super-resolution or interpolation). In these regions of equivalence, the in- terpolation algorithms are used to increase the dimensions. In the anothers regions, the super-resolution algorithms are used. As interpolation algorithms are faster than the com- putationally complex super-resolution algorithms, the idea is decrease the processing time without a ecting the visual quality of ampli ed video. Besides this approach, this paper also proposes a strategy to divide the data among various processes to perform the super-resolution operation in parallel. An interesting re- sult of the proposed model is the decoupling of the super-resolution algorithm and the parallel processing strategy. In other words, di erent super-resolution algorithms can ex- plore the features of the proposed framework without algorithmic modi cations to achieve the parallelism. Thus, the framework is portable, scalable and can be reusable by di erent super-resolution methods

Advanced Restoration Techniques for Images and Disparity Maps

With increasing popularity of digital cameras, the field of Computa- tional Photography emerges as one of the most demanding areas of research. In this thesis we study and develop novel priors and op- timization techniques to solve inverse problems, including disparity estimation and image restoration. The disparity map estimation method proposed in this thesis incor- porates multiple frames of a stereo video sequence to ensure temporal coherency. To enforce smoothness, we use spatio-temporal connec- tions between the pixels of the disparity map to constrain our solution. Apart from smoothness, we enforce a consistency constraint for the disparity assignments by using connections between the left and right views. These constraints are then formulated in a graphical model, which we solve using mean-field approximation. We use a filter-based mean-field optimization that perform efficiently by updating the dis- parity variables in parallel. The parallel updates scheme, however, is not guaranteed to converge to a stationary point. To compare and demonstrate the effectiveness of our approach, we developed a new optimization technique that uses sequential updates, which runs ef- ficiently and guarantees convergence. Our empirical results indicate that with proper initialization, we can employ the parallel update scheme and efficiently optimize our disparity maps without loss of quality. Our method ranks amongst the state of the art in common benchmarks, and significantly reduces the temporal flickering artifacts in the disparity maps. In the second part of this thesis, we address several image restora- tion problems such as image deblurring, demosaicing and super- resolution. We propose to use denoising autoencoders to learn an approximation of the true natural image distribution. We parametrize our denoisers using deep neural networks and show that they learn the gradient of the smoothed density of natural images. Based on this analysis, we propose a restoration technique that moves the so- lution towards the local extrema of this distribution by minimizing the difference between the input and output of our denoiser. Weii demonstrate the effectiveness of our approach using a single trained neural network in several restoration tasks such as deblurring and super-resolution. In a more general framework, we define a new Bayes formulation for the restoration problem, which leads to a more efficient and robust estimator. The proposed framework achieves state of the art performance in various restoration tasks such as deblurring and demosaicing, and also for more challenging tasks such as noise- and kernel-blind image deblurring. Keywords. disparity map estimation, stereo matching, mean-field optimization, graphical models, image processing, linear inverse prob- lems, image restoration, image deblurring, image denoising, single image super-resolution, image demosaicing, deep neural networks, denoising autoencoder

Steered mixture-of-experts for light field images and video : representation and coding

Research in light field (LF) processing has heavily increased over the last decade. This is largely driven by the desire to achieve the same level of immersion and navigational freedom for camera-captured scenes as it is currently available for CGI content. Standardization organizations such as MPEG and JPEG continue to follow conventional coding paradigms in which viewpoints are discretely represented on 2-D regular grids. These grids are then further decorrelated through hybrid DPCM/transform techniques. However, these 2-D regular grids are less suited for high-dimensional data, such as LFs. We propose a novel coding framework for higher-dimensional image modalities, called Steered Mixture-of-Experts (SMoE). Coherent areas in the higher-dimensional space are represented by single higher-dimensional entities, called kernels. These kernels hold spatially localized information about light rays at any angle arriving at a certain region. The global model consists thus of a set of kernels which define a continuous approximation of the underlying plenoptic function. We introduce the theory of SMoE and illustrate its application for 2-D images, 4-D LF images, and 5-D LF video. We also propose an efficient coding strategy to convert the model parameters into a bitstream. Even without provisions for high-frequency information, the proposed method performs comparable to the state of the art for low-to-mid range bitrates with respect to subjective visual quality of 4-D LF images. In case of 5-D LF video, we observe superior decorrelation and coding performance with coding gains of a factor of 4x in bitrate for the same quality. At least equally important is the fact that our method inherently has desired functionality for LF rendering which is lacking in other state-of-the-art techniques: (1) full zero-delay random access, (2) light-weight pixel-parallel view reconstruction, and (3) intrinsic view interpolation and super-resolution