Distributed Compressed Representation of Correlated Image Sets

Vision sensor networks and video cameras find widespread usage in several applications that rely on effective representation of scenes or analysis of 3D information. These systems usually acquire multiple images of the same 3D scene from different viewpoints or at different time instants. Therefore, these images are generally correlated through displacement of scene objects. Efficient compression techniques have to exploit this correlation in order to efficiently communicate the 3D scene information. Instead of joint encoding that requires communication between the cameras, in this thesis we concentrate on distributed representation, where the captured images are encoded independently, but decoded jointly to exploit the correlation between images. One of the most important and challenging tasks relies in estimation of the underlying correlation from the compressed correlated images for effective reconstruction or analysis in the joint decoder. This thesis focuses on developing efficient correlation estimation algorithms and joint representation of multiple correlated images captured by various sensing methodologies, e.g., planar, omnidirectional and compressive sensing (CS) sensors. The geometry of the 2D visual representation and the acquisition complexity vary for each sensor type. Therefore, we need to carefully consider the specific geometric nature of the captured images while developing distributed representation algorithms. In this thesis we propose robust algorithms in different scene analysis and reconstruction scenarios. We first concentrate on the distributed representation of omnidirectional images captured by catadioptric sensors. The omnidirectional images are captured from different viewpoints and encoded independently with a balanced rate distribution among the different cameras. They are mapped on the sphere which captures the plenoptic function in its radial form without Euclidean discrepancies. We propose a transform-based distributed coding algorithm, where the spherical images initially undergo a multi-resolution decomposition. The visual information is then split into two correlated partitions. The encoder transmits one partition after entropy coding, as well as the syndrome bits resulting from the Slepian-Wolf encoding of the other partition. The joint decoder estimates a disparity image to take benefit of the correlation between views and uses the syndrome bits to decode the missing information. Such a strategy proves to be beneficial with respect to the independent processing of images and shows only a small performance loss compared to the joint encoding of different views. The encoding complexity in the previous approach is non-negligible due to the visual information processing based on Slepian-Wolf coding and its associated rate parameter estimation. We therefore discard the Slepian-Wolf encoding and propose a distributed coding solution, where the correlated images are encoded independently using transform-based coding solutions (e.g., SPIHT). The central decoder now builds a correlation model from the compressed images, which is used to jointly decode a pair of images. Experimental results demonstrate that the proposed distributed coding solution improves the rate-distortion performance of the separate coding results for both planar and omnidirectional images. However, this improvement is significant only at medium to high bit rates. We therefore propose a rate allocation scheme that identifies and transmits the necessary visual information from each image to improve the correlation estimation accuracy at low bit rate. Experimental results show that for a given bit budget the proposed encoding scheme permits to compute an accurate correlation estimation comparing to the one obtained with SPIHT, JPEG 2000 or JPEG coding schemes. We show however that the improvement in the correlation estimation comes at the price of penalizing the image reconstruction quality; therefore there exists an interesting trade-off between the accurate correlation estimation and image reconstruction as encoding optimization objectives are different in both cases. Next, we further simplify the encoding complexity by replacing the classical imaging sensors with the simple CS sensors, that directly acquire the compressed images in the form of quantized linear measurements. We now concentrate on the particular problem, where one image is selected as the reference and it is used as a side information for the correlation estimation. We propose a geometry-based model to describe the correlation between the visual information in a pair of images. The joint decoder first captures the most prominent visual features in the reconstructed reference image using geometric functions. Since the images are correlated, these features are likely to be present in the other images too, possibly with geometric transformations. Hence, we propose to estimate the correlation model with a regularized optimization problem that locates these features in the compressed images. The regularization terms enforce smoothness of the transformation field, and consistency between the estimated images and the quantized measurements. Experimental results show that the proposed scheme is able to efficiently estimate the correlation between images for several multi-view and video datasets. The proposed scheme is finally shown to outperform DSC schemes based on unsupervised disparity (or motion) learning, as well as independent coding solutions based on JPEG 2000. We then extend the previous scenario to a symmetric decoding problem, where we are interested to estimate the correlation model directly from the quantized linear measurements without explicitly reconstructing the reference images. We first show that the motion field that represents the main source of correlation between images can be described as a linear operator. We further derive a linear relationship between the correlated measurements in the compressed domain. We then derive a regularized cost function to estimate the correlation model directly in the compressed domain using graph-based optimization algorithms. Experimental results show that the proposed scheme estimates an accurate correlation model among images in both multi-view and video imaging scenarios. We then propose a robust data fidelity term that improves the quality of the correlation estimation when the measurements are quantized. Finally, we show by experiments that the proposed compressed correlation estimation scheme is able to compete the solution of a scheme that estimates a correlation model from the reconstructed images without the complexity of image reconstruction. Finally, we study the benefit of using the correlation information while jointly reconstructing the images from the compressed linear measurements. We consider both the asymmetric and symmetric scenarios described previously. We propose joint reconstruction methodologies based on a constrained optimization problem which is solved using effective proximal splitting methods. The constraints included in our framework enforce the reconstructed images to satisfy both the correlation and the quantized measurements consistency objectives. Experimental results demonstrate that the proposed joint reconstruction scheme improves the quality of the decoded images, when compared to a scheme where the images are handled independently. In this thesis we build efficient distributed scene representation algorithms for the multiple correlated images captured in planar, omnidirectional and CS cameras. The coding rate in our symmetric distributed coding solution stays balanced between the encoders and stays close to the joint encoding solutions. Our novel algorithms lead to effective correlation estimation in different sensing and coding scenarios. In addition, we provide innovative solutions for robust correlation estimation from highly compressed images in simple sensing frameworks. Our CS-based joint reconstruction frameworks effectively exploit the inter-view correlation, that permits to achieve high compression gains compared to state-of-the-art independent and distributed coding solutions

Адаптивное комбинированное кодирование изображений с прогнозированием объема арифметического кода

The problem of increasing the efficiency of coding of halftone images in the space of bit planes of differences in pixel values obtained using differential coding (DPCM – Differential pulse-code modulation) is considered. For a compact representation of DPCM pixel values, it is proposed to use a combined compression encoder that implements arithmetic coding and run-length coding. An arithmetic encoder provides high compression ratios, but has high computational complexity and significant encoding overhead. This makes it effective primarily for compressing the mean-value bit-planes of DPCM pixel values. Run-length coding is extremely simple and outperforms arithmetic coding in compressing long sequences of repetitive symbols that often occur in the upper bit planes of DPCM pixel values. For DPCM bit planes of pixel values of any image, a combination of simple run length coders and complex arithmetic coders can be selected that provides the maximum compression ratio for each bit plane and all planes in general with the least computational complexity. As a result, each image has its own effective combined encoder structure, which depends on the distribution of bits in the bit planes of the DPCM pixel values. To adapt the structure of the combined encoder to the distribution of bits in the bit planes of DPCM pixel values, the article proposes to use prediction of the volume of arithmetic code based on entropy and comparison of the obtained predicted value with the volume of run length code. The entropy is calculated based on the values of the number of repetitions of ones and zero symbols, which are obtained as intermediate results of the run length encoding. This does not require additional computational costs. It was found that in comparison with the adaptation of the combined encoder structure using direct determination of the arithmetic code volume of each bit plane of DPCM pixel values, the proposed encoder structure provides a significant reduction in computational complexity while maintaining high image compression ratios.Рассматривается задача повышения эффективности кодирования полутоновых изображений в пространстве битовых плоскостей разностей значений пикселей, полученных с помощью дифференциального кодирования (DPCM – Differential pulse-code modulation). Для компактного представления DPCM-значений пикселей предлагается использовать комбинированный кодер сжатия, реализующий арифметическое кодирование и кодирование длин серий. Арифметический кодер обеспечивает высокие коэффициенты сжатия, но имеет высокую вычислительную сложность и значительные накладные расходы на кодирование, что делает его эффективным в основном для сжатия средних по значимости битовых плоскостей DPCM-значений пикселей. Кодирование длин серий является предельно простым и превосходит арифметическое кодирование в сжатии длинных последовательностей повторяющихся символов, часто встречающихся в старших битовых плоскостях DPCM-значений пикселей. Для битовых плоскостей DPCM-значений пикселей любого изображения может быть подобрана комбинация простых кодеров длин серий и сложных арифметических кодеров, обеспечивающая максимальный коэффициент сжатия каждой битовой плоскости и всех плоскостей в целом при наименьшей вычислительной сложности. В результате каждому изображению соответствует своя эффективная структура комбинированного кодера, зависящая от распределения бит в битовых плоскостях DPCM-значений пикселей. Для адаптации структуры комбинированного кодера к распределению бит в битовых плоскостях DPCM-значений пикселей в статье предлагается использовать прогнозирование объема арифметического кода на основе энтропии и сравнение полученного прогнозного значения с объемом кода длин серий. Вычисление энтропии осуществляется на основе значений количества повторов единичных и нулевых символов, получаемых в качестве промежуточных результатов кодирования длин серий, что не требует дополнительных вычислительных затрат. Установлено, что в сравнении с адаптацией структуры комбинированного кодера с использованием прямого определения объема арифметического кода каждой битовой плоскости DPCM-значений пикселей предложенная структура кодера обеспечивает существенное снижение вычислительной сложности при сохранении высоких коэффициентов сжатия изображений

Anuário Científico – 2008 Resumos de Artigos, Comunicações, Teses e Livros

A divulgação do conhecimento resultante da Ciência, Investigação e Actividade Profissional de mérito reconhecido são indissociáveis e necessários numa sociedade em evolução, sem descurar a vertente pedagógica, numa Instituição de Ensino Superior. Verificou-se que durante este período se assistiu a um incremento das publicações científicas dos docentes do ISEL. Por outro lado, existiu um maior envolvimento em projectos de investigação e um acréscimo na conclusão do grau de Doutor. Assim, o anuário científico de 2008 constitui um documento de divulgação desta actividade no Instituto Superior de Engenharia de Lisboa em parceria com outros Politécnicos, Universidades e Centros de Investigação nacionais e internacionais. Numa altura em que se avizinham mudanças estruturais no Ensino Superior, esperamos que o poder político avalie as instituições pelo trabalho desenvolvido e pela qualidade dos engenheiros que estas formam

Anuário Científico – 2009 & 2010 Resumos de Artigos, Comunicações, Teses, Patentes, Livros e Monografias de Mestrado

O Conselho Técnico-Científico do Instituto Superior de Engenharia de Lisboa (ISEL), na senda da consolidação da divulgação do conhecimento e da ciência desenvolvidos pelo nosso corpo docente, propõe-se publicar mais uma edição do Anuário Científico, relativa à produção científica de 2009 e 2010. A investigação, enquanto vertente estratégica do Instituto Superior de Engenharia de Lisboa (ISEL), tem concorrido para o seu reconhecimento nacional e internacional como instituição de referência e de qualidade na área do ensino das engenharias. É também nesta vertente que o ISEL consubstancia a sua ligação à sociedade portuguesa e internacional através da transferência de tecnologia e de conhecimento, resultantes da sua atividade científica e pedagógica, contribuindo para o seu desenvolvimento e crescimento de forma sustentada. São parte integrante do Anuário Científico todos os conteúdos com afiliação ISEL resultantes de resumos de artigos publicados em livros, revistas e atas de congressos que os docentes do ISEL apresentaram em fóruns e congressos nacionais e internacionais, bem como teses e patentes. Desde 2002, ano da publicação da primeira edição, temos assistido a uma evolução crescente do número de publicações de conteúdos científicos, fruto do trabalho desenvolvido pelos docentes que se têm empenhado com afinco e perseverança. Contudo, nestes dois anos (2009 e 2010) constatou-se um decréscimo no número de publicações, principalmente em 2010. Uma das causas poderá estar diretamente relacionada com a redução do financiamento ao ensino superior uma vez que limita toda a investigação no âmbito da atividade de I&D e da produção científica. Na sequência da implementação do Processo de Bolonha em 2006, o ISEL promoveu a criação de cursos de Mestrado disponibilizando uma oferta educativa mais completa e diversificada aos seus alunos, mas também de outras instituições, dotando-os de competências inovadoras apropriadas ao mercado de trabalho que hoje se carateriza mais competitivo e dinâmico. Terminados os períodos escolar e de execução das monografias dos alunos, os resumos destas são igualmente parte integrante deste Anuário, no que concerne à conclusão dos Mestrados em 2009 e 2010.A fim de permitir uma maior acessibilidade à comunidade científica e à sociedade civil, o Anuário Científico será editado de ora avante em formato eletrónico. Excecionalmente esta edição contempla publicações referentes a dois anos – 2009 e 2010