Contributions in image and video coding

Orientador: Max Henrique Machado CostaTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: A comunidade de codificação de imagens e vídeo vem também trabalhando em inovações que vão além das tradicionais técnicas de codificação de imagens e vídeo. Este trabalho é um conjunto de contribuições a vários tópicos que têm recebido crescente interesse de pesquisadores na comunidade, nominalmente, codificação escalável, codificação de baixa complexidade para dispositivos móveis, codificação de vídeo de múltiplas vistas e codificação adaptativa em tempo real. A primeira contribuição estuda o desempenho de três transformadas 3-D rápidas por blocos em um codificador de vídeo de baixa complexidade. O codificador recebeu o nome de Fast Embedded Video Codec (FEVC). Novos métodos de implementação e ordens de varredura são propostos para as transformadas. Os coeficiente 3-D são codificados por planos de bits pelos codificadores de entropia, produzindo um fluxo de bits (bitstream) de saída totalmente embutida. Todas as implementações são feitas usando arquitetura com aritmética inteira de 16 bits. Somente adições e deslocamentos de bits são necessários, o que reduz a complexidade computacional. Mesmo com essas restrições, um bom desempenho em termos de taxa de bits versus distorção pôde ser obtido e os tempos de codificação são significativamente menores (em torno de 160 vezes) quando comparados ao padrão H.264/AVC. A segunda contribuição é a otimização de uma recente abordagem proposta para codificação de vídeo de múltiplas vistas em aplicações de video-conferência e outras aplicações do tipo "unicast" similares. O cenário alvo nessa abordagem é fornecer vídeo com percepção real em 3-D e ponto de vista livre a boas taxas de compressão. Para atingir tal objetivo, pesos são atribuídos a cada vista e mapeados em parâmetros de quantização. Neste trabalho, o mapeamento ad-hoc anteriormente proposto entre pesos e parâmetros de quantização é mostrado ser quase-ótimo para uma fonte Gaussiana e um mapeamento ótimo é derivado para fonte típicas de vídeo. A terceira contribuição explora várias estratégias para varredura adaptativa dos coeficientes da transformada no padrão JPEG XR. A ordem de varredura original, global e adaptativa do JPEG XR é comparada com os métodos de varredura localizados e híbridos propostos neste trabalho. Essas novas ordens não requerem mudanças nem nos outros estágios de codificação e decodificação, nem na definição da bitstream A quarta e última contribuição propõe uma transformada por blocos dependente do sinal. As transformadas hierárquicas usualmente exploram a informação residual entre os níveis no estágio da codificação de entropia, mas não no estágio da transformada. A transformada proposta neste trabalho é uma técnica de compactação de energia que também explora as similaridades estruturais entre os níveis de resolução. A idéia central da técnica é incluir na transformada hierárquica um número de funções de base adaptativas derivadas da resolução menor do sinal. Um codificador de imagens completo foi desenvolvido para medir o desempenho da nova transformada e os resultados obtidos são discutidos neste trabalhoAbstract: The image and video coding community has often been working on new advances that go beyond traditional image and video architectures. This work is a set of contributions to various topics that have received increasing attention from researchers in the community, namely, scalable coding, low-complexity coding for portable devices, multiview video coding and run-time adaptive coding. The first contribution studies the performance of three fast block-based 3-D transforms in a low complexity video codec. The codec has received the name Fast Embedded Video Codec (FEVC). New implementation methods and scanning orders are proposed for the transforms. The 3-D coefficients are encoded bit-plane by bit-plane by entropy coders, producing a fully embedded output bitstream. All implementation is performed using 16-bit integer arithmetic. Only additions and bit shifts are necessary, thus lowering computational complexity. Even with these constraints, reasonable rate versus distortion performance can be achieved and the encoding time is significantly smaller (around 160 times) when compared to the H.264/AVC standard. The second contribution is the optimization of a recent approach proposed for multiview video coding in videoconferencing applications or other similar unicast-like applications. The target scenario in this approach is providing realistic 3-D video with free viewpoint video at good compression rates. To achieve such an objective, weights are computed for each view and mapped into quantization parameters. In this work, the previously proposed ad-hoc mapping between weights and quantization parameters is shown to be quasi-optimum for a Gaussian source and an optimum mapping is derived for a typical video source. The third contribution exploits several strategies for adaptive scanning of transform coefficients in the JPEG XR standard. The original global adaptive scanning order applied in JPEG XR is compared with the localized and hybrid scanning methods proposed in this work. These new orders do not require changes in either the other coding and decoding stages or in the bitstream definition. The fourth and last contribution proposes an hierarchical signal dependent block-based transform. Hierarchical transforms usually exploit the residual cross-level information at the entropy coding step, but not at the transform step. The transform proposed in this work is an energy compaction technique that can also exploit these cross-resolution-level structural similarities. The core idea of the technique is to include in the hierarchical transform a number of adaptive basis functions derived from the lower resolution of the signal. A full image codec is developed in order to measure the performance of the new transform and the obtained results are discussed in this workDoutoradoTelecomunicações e TelemáticaDoutor em Engenharia Elétric

Compressive sensing based image processing and energy-efficient hardware implementation with application to MRI and JPG 2000

In the present age of technology, the buzzwords are low-power, energy-efficient and compact systems. This directly leads to the date processing and hardware techniques employed in the core of these devices. One of the most power-hungry and space-consuming schemes is that of image/video processing, due to its high quality requirements. In current design methodologies, a point has nearly been reached in which physical and physiological effects limit the ability to just encode data faster. These limits have led to research into methods to reduce the amount of acquired data without degrading image quality and increasing the energy consumption. Compressive sensing (CS) has emerged as an efficient signal compression and recovery technique, which can be used to efficiently reduce the data acquisition and processing. It exploits the sparsity of a signal in a transform domain to perform sampling and stable recovery. This is an alternative paradigm to conventional data processing and is robust in nature. Unlike the conventional methods, CS provides an information capturing paradigm with both sampling and compression. It permits signals to be sampled below the Nyquist rate, and still allowing optimal reconstruction of the signal. The required measurements are far less than those of conventional methods, and the process is non-adaptive, making the sampling process faster and universal. In this thesis, CS methods are applied to magnetic resonance imaging (MRI) and JPEG 2000, which are popularly used imaging techniques in clinical applications and image compression, respectively. Over the years, MRI has improved dramatically in both imaging quality and speed. This has further revolutionized the field of diagnostic medicine. However, imaging speed, which is essential to many MRI applications still remains a major challenge. The specific challenge addressed in this work is the use of non-Fourier based complex measurement-based data acquisition. This method provides the possibility of reconstructing high quality MRI data with minimal measurements, due to the high incoherence between the two chosen matrices. Similarly, JPEG2000, though providing a high compression, can be further improved upon by using compressive sampling. In addition, the image quality is also improved. Moreover, having a optimized JPEG 2000 architecture reduces the overall processing, and a faster computation when combined with CS. Considering the requirements, this thesis is presented in two parts. In the first part: (1) A complex Hadamard matrix (CHM) based 2D and 3D MRI data acquisition with recovery using a greedy algorithm is proposed. The CHM measurement matrix is shown to satisfy the necessary condition for CS, known as restricted isometry property (RIP). The sparse recovery is done using compressive sampling matching pursuit (CoSaMP); (2) An optimized matrix and modified CoSaMP is presented, which enhances the MRI performance when compared with the conventional sampling; (3) An energy-efficient, cost-efficient hardware design based on field programmable gate array (FPGA) is proposed, to provide a platform for low-cost MRI processing hardware. At every stage, the design is proven to be superior with other commonly used MRI-CS methods and is comparable with the conventional MRI sampling. In the second part, CS techniques are applied to image processing and is combined with JPEG 2000 coder. While CS can reduce the encoding time, the effect on the overall JPEG 2000 encoder is not very significant due to some complex JPEG 2000 algorithms. One problem encountered is the big-level operations in JPEG 2000 arithmetic encoding (AE), which is completely based on bit-level operations. In this work, this problem is tackled by proposing a two-symbol AE with an efficient FPGA based hardware design. Furthermore, this design is energy-efficient, fast and has lower complexity when compared to conventional JPEG 2000 encoding

The Space and Earth Science Data Compression Workshop

This document is the proceedings from a Space and Earth Science Data Compression Workshop, which was held on March 27, 1992, at the Snowbird Conference Center in Snowbird, Utah. This workshop was held in conjunction with the 1992 Data Compression Conference (DCC '92), which was held at the same location, March 24-26, 1992. The workshop explored opportunities for data compression to enhance the collection and analysis of space and Earth science data. The workshop consisted of eleven papers presented in four sessions. These papers describe research that is integrated into, or has the potential of being integrated into, a particular space and/or Earth science data information system. Presenters were encouraged to take into account the scientists's data requirements, and the constraints imposed by the data collection, transmission, distribution, and archival system

Entropy coders and 3D-Hadamard coefficients sequency scan order for a fast embedded color video codec

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)This work compares the performances of two Golomb family entropy coders applied to a video codec system named FHVC (fast hadamard video codec). The entropy coders considered have different operation modes and specific adaptation strategies. The work also presents a new 3D-transform coefficient scan order developed for the FHVC. This scan process is based on the multiplication of the three-dimensional sequency numbers of each coefficient. The FHVC (which is also described in this work) is a fast embedded color video codec developed to be implemented in a video set-top box used in a fiber optics network. The focus is on more reduced execution times, and not on higher compression rates. Low computational complexity and use of meager computational resources are also required. All the multiplications and divisions operations are performed by binary shifts and the system is implemented exclusively with 16-bit integer arithmetic. Even with these constraints, good distortion versus bit-rate results were achieved. The Hadamard transform is used in a three-dimensional fashion, in order to reduce spatial and temporal correlation and to avoid costly motion estimation and compensation techniques. The proposed scan procedure allows the transform coefficient reading in an idealistic "decreasing in the average" order. After the scan procedure, the encoding of the bit sequence of the 3D-Hadamard coefficients is carried out, bit-plane-by-bit-plane, with an adaptive Golomb run-length entropy coder, which produces a fully embedded output bitstream. Two entropy coders were considered. The first one uses an empirical, but fast and efficient, adaptation strategy that shows good results on non-stationary data. The second one has an adaptation strategy that is nearly optimum, in a maximum-likelihood sense, for independent Bernoulli identically distributed data. (C) 2008 Elsevier Ltd. All rights reserved.364SI676690Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES

Three-dimensional Transforms And Entropy Coders For A Fast Embedded Color Video Codec

This work compares the performances of two fast 3-D transforms and two adaptive Golomb entropy coders applied to a video codec system named FEVC (Fast Embedded Video Codec). The compared transforms are Hadamard (4×4×4 and 8×8×8) and H.264/AVC integer DCT (4×4×4). The compared adaptive Golomb entropy coders have different operation modes and adaptation strategies. New 3-D implementation methods for the transforms are presented. After the scan procedure, the encoding of the 3-D coefficients is done, bit-plane-by-bit-plane, by the entropy coders, producing a fully embedded output bitstream. The FEVC (also described here) was developed to be implemented each of a large number of set-top boxes used in a fiber optics network. For that reason, it is focused on reduced complexity and execution time, not on high compression rates. The use of meager computational resources is also required. Even with these constraints, good distortion versus rate results were achieved. © 2008 IEEE.147154Jain, A.K., (1989) Fundamentals of Digital Image Processing, , Prentice Hall, Englewood Cliffs, NJ, USAMalvar, H., Hallapuro, A., Karczewicz, M., Kerofsky, L., Low-Complexity Transform and Quantization with 16-bit Arithmetic for H.26L (2002) Proceedings of the International Conference on Image Processing - ICIP, pp. 489-492Oliveira, F.C., Costa, M.H.M., Embedded DCT Image Encoding (2002) International Telecommunications Symposium - ITS-2002, , Natal, Brazil, SeptCosta, M.H.M., Malvar, H.S., Efficient Run-Length Encoding of Binary Sources with Unknown Statistics (2004) Proceedings of the Data Compression Conference, pp. 534-544. , Snowbird, UT, USA, ppChan, R.K.W., Lee, M.C., 3D-DCT Quantization as a Compression Technique for Video Sequences (1997) Proceedings of the International Conference On Virtual Systems And Multimedia, pp. 188-196. , Geneva, Switzerland, ppChan, R.K.W., Lee, M.C., Quantization of 3D-DCT Coefficients and Scan Order for Video Compression (1997) Journal of Visual Communication and Image Representation, 8 (4), pp. 405-422Testoni, V., Costa, M.H.M., 3D-Hadamard Coefficients Sequency Scan Order for a Fast Embedded Color Video Coded (2007) Proceedings of the International Conference on Signal Processing and Communication Systems, pp. 75-82. , Gold Coast, Australia, ppSullivan, G., Estrop, S., (2003) Video Rendering with 8-bit YUVFormats, , Microsoft Digital Media DivisionSullivan, G., Topiwala, P., Luthra, A., The H.264/ AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions (2004) Conference on Applications of Digital Image ProcessingShen, K., DeIp, E.J., Wavelet Based Rate Scalable Video Compression (1999) IEEE Transactions on Circuits and Systems for Video Technology, 9 (1), pp. 109-122Kim, B., Xiong, Z., Pearlman, W.A., Low Bit Rate Scalable Video Coding with 3D Set Partitioning in Hierarchical Trees (3D SPIHT) (2000) IEEE Transactions on Circuits and Systems for Video Technology, 10 (8), pp. 1374-1387H.264/AVC reference software version JM 11.0, , http://iphome.hhi.de/suehring/tml, Downloaded in Dec. 200