Light field image coding with flexible viewpoint scalability and random access

This paper proposes a novel light field image compression approach with viewpoint scalability and random access functionalities. Although current state-of-the-art image coding algorithms for light fields already achieve high compression ratios, there is a lack of support for such functionalities, which are important for ensuring compatibility with different displays/capturing devices, enhanced user interaction and low decoding delay. The proposed solution enables various encoding profiles with different flexible viewpoint scalability and random access capabilities, depending on the application scenario. When compared to other state-of-the-art methods, the proposed approach consistently presents higher bitrate savings (44% on average), namely when compared to pseudo-video sequence coding approach based on HEVC. Moreover, the proposed scalable codec also outperforms MuLE and WaSP verification models, achieving average bitrate saving gains of 37% and 47%, respectively. The various flexible encoding profiles proposed add fine control to the image prediction dependencies, which allow to exploit the tradeoff between coding efficiency and the viewpoint random access, consequently, decreasing the maximum random access penalties that range from 0.60 to 0.15, for lenslet and HDCA light fields.info:eu-repo/semantics/acceptedVersio

Scalable light field representation and coding

This Thesis aims to advance the state-of-the-art in light field representation and coding. In this context, proposals to improve functionalities like light field random access and scalability are also presented. As the light field representation constrains the coding approach to be used, several light field coding techniques to exploit the inherent characteristics of the most popular types of light field representations are proposed and studied, which are normally based on micro-images or sub-aperture-images. To encode micro-images, two solutions are proposed, aiming to exploit the redundancy between neighboring micro-images using a high order prediction model, where the model parameters are either explicitly transmitted or inferred at the decoder, respectively. In both cases, the proposed solutions are able to outperform low order prediction solutions. To encode sub-aperture-images, an HEVC-based solution that exploits their inherent intra and inter redundancies is proposed. In this case, the light field image is encoded as a pseudo video sequence, where the scanning order is signaled, allowing the encoder and decoder to optimize the reference picture lists to improve coding efficiency. A novel hybrid light field representation coding approach is also proposed, by exploiting the combined use of both micro-image and sub-aperture-image representation types, instead of using each representation individually. In order to aid the fast deployment of the light field technology, this Thesis also proposes scalable coding and representation approaches that enable adequate compatibility with legacy displays (e.g., 2D, stereoscopic or multiview) and with future light field displays, while maintaining high coding efficiency. Additionally, viewpoint random access, allowing to improve the light field navigation and to reduce the decoding delay, is also enabled with a flexible trade-off between coding efficiency and viewpoint random access.Esta Tese tem como objetivo avançar o estado da arte em representação e codificação de campos de luz. Neste contexto, são também apresentadas propostas para melhorar funcionalidades como o acesso aleatório ao campo de luz e a escalabilidade. Como a representação do campo de luz limita a abordagem de codificação a ser utilizada, são propostas e estudadas várias técnicas de codificação de campos de luz para explorar as características inerentes aos seus tipos mais populares de representação, que são normalmente baseadas em micro-imagens ou imagens de sub-abertura. Para codificar as micro-imagens, são propostas duas soluções, visando explorar a redundância entre micro-imagens vizinhas utilizando um modelo de predição de alta ordem, onde os parâmetros do modelo são explicitamente transmitidos ou inferidos no decodificador, respetivamente. Em ambos os casos, as soluções propostas são capazes de superar as soluções de predição de baixa ordem. Para codificar imagens de sub-abertura, é proposta uma solução baseada em HEVC que explora a inerente redundância intra e inter deste tipo de imagens. Neste caso, a imagem do campo de luz é codificada como uma pseudo-sequência de vídeo, onde a ordem de varrimento é sinalizada, permitindo ao codificador e decodificador otimizar as listas de imagens de referência para melhorar a eficiência da codificação. Também é proposta uma nova abordagem de codificação baseada na representação híbrida do campo de luz, explorando o uso combinado dos tipos de representação de micro-imagem e sub-imagem, em vez de usar cada representação individualmente. A fim de facilitar a rápida implantação da tecnologia de campo de luz, esta Tese também propõe abordagens escaláveis de codificação e representação que permitem uma compatibilidade adequada com monitores tradicionais (e.g., 2D, estereoscópicos ou multivista) e com futuros monitores de campo de luz, mantendo ao mesmo tempo uma alta eficiência de codificação. Além disso, o acesso aleatório de pontos de vista, permitindo melhorar a navegação no campo de luz e reduzir o atraso na descodificação, também é permitido com um equilíbrio flexível entre eficiência de codificação e acesso aleatório de pontos de vista

Unified representation of monitoring information across federated cloud infrastructures

Nowadays one of the issues hindering the potential of federating cloud-based infrastructures to reach much larger scales is their standard management and monitoring. In particular, this is true in cases where these federated infrastructures provide emerging Future Internet and Smart Cities-oriented services, such as the Internet of Things (IoT), that benefit from cloud services. The contribution of this paper is the introduction of a unified monitoring architecture for federated cloud infrastructures accompanied by the adoption of a uniform representation of measurement data. The presented solution is capable of providing multi-domain compatibility, scalability, as well as the ability to analyze large amounts of monitoring data, collected from datacenters and offered through open and standardized APIs. The solution described herein has been deployed and is currently running on a community of 5 infrastructures within the framework of the European Project XIFI, to be extended to 12 more infrastructures

Light field image coding based on hybrid data representation

This paper proposes a novel efficient light field coding approach based on a hybrid data representation. Current state-of-the-art light field coding solutions either operate on micro-images or sub-aperture images. Consequently, the intrinsic redundancy that exists in light field images is not fully exploited, as is demonstrated. This novel hybrid data representation approach allows to simultaneously exploit four types of redundancies: i) sub-aperture image intra spatial redundancy, ii) sub-aperture image inter-view redundancy, iii) intra-micro-image redundancy, and iv) inter-micro-image redundancy between neighboring micro-images. The proposed light field coding solution allows flexibility for several types of baselines, by adaptively exploiting the most predominant type of redundancy on a coding block basis. To demonstrate the efficiency of using a hybrid representation, this paper proposes a set of efficient pixel prediction methods combined with a pseudo-video sequence coding approach, based on the HEVC standard. Experimental results show consistent average bitrate savings when the proposed codec is compared to relevant state-of-the-art benchmarks. For lenslet light field content, the proposed coding algorithm outperforms the HEVC-based pseudo-video sequence coding benchmark by an average bitrate savings of 23%. It is shown for the same light field content that the proposed solution outperforms JPEG Pleno verification models MuLE and WaSP, as these codecs are only able to achieve 11% and -14% bitrate savings over the same HEVC-based benchmark, respectively. The performance of the proposed coding approach is also validated for light fields with wider baselines, captured with high-density camera arrays, being able to outperform both the HEVC-based benchmark, as well as MuLE and WaSP.info:eu-repo/semantics/publishedVersio

Comparison of model-based and sensor-based detection of thermal runaway in Li-ion battery modules for automotive application

In recent years, research on lithium–ion (Li-ion) battery safety and fault detection has become an important topic, providing a broad range of methods for evaluating the cell state based on voltage and temperature measurements. However, other measurement quantities and close-to-application test setups have only been sparsely considered, and there has been no comparison in between methods. In this work, the feasibility of a multi-sensor setup for the detection of Thermal Runaway failure of automotive-size Li-ion battery modules have been investigated in comparison to a model-based approach. For experimental validation, Thermal Runaway tests were conducted in a close-to-application configuration of module and battery case—triggered by external heating with two different heating rates. By two repetitions of each experiment, a high accordance of characteristics and results has been achieved and the signal feasibility for fault detection has been discussed. The model-based method, that had previously been published, recognised the thermal fault in the fastest way—significantly prior to the required 5 min pre-warning time. This requirement was also achieved with smoke and gas sensors in most test runs. Additional criteria for evaluating detection approaches besides detection time have been discussed to provide a good starting point for choosing a suitable approach that is dependent on application defined requirements, e.g., acceptable complexity

Light field image coding with flexible viewpoint scalability and random access

This paper proposes a novel light field image compression approach with viewpoint scalability and random access functionalities. Although current state-of-the-art image coding algorithms for light fields already achieve high compression ratios, there is a lack of support for such functionalities, which are important for ensuring compatibility with different displays/capturing devices, enhanced user interaction and low decoding delay. The proposed solution enables various encoding profiles with different flexible viewpoint scalability and random access capabilities, depending on the application scenario. When compared to other state-of-the-art methods, the proposed approach consistently presents higher bitrate savings (44% on average), namely when compared to pseudo-video sequence coding approach based on HEVC. Moreover, the proposed scalable codec also outperforms MuLE and WaSP verification models, achieving average bitrate saving gains of 37% and 47%, respectively. The various flexible encoding profiles proposed add fine control to the image prediction dependencies, which allow to exploit the tradeoff between coding efficiency and the viewpoint random access, consequently, decreasing the maximum random access penalties that range from 0.60 to 0.15, for lenslet and HDCA light fields.info:eu-repo/semantics/publishedVersio

Scalable Coding Framework for a View-Dependent Streaming of Digital Holograms

Unlike conventional images and videos, digital holograms contain large amounts of data with very low redundancy. Consequently, current communication networks may be not able to meet the bandwidth requirements for hologram transmission in reasonable time. To enable practical streaming of holographic contents, we propose a progressive coding method that combines quality scalability with viewpoint scalability. From a Gabor wavelets decomposition of the hologram, the server starts by selecting the coefficients corresponding to the user's viewpoint. Then, the selected coefficients are encoded progressively according to their importance for the reconstructed view. Experimental results reveal that our approach outperforms conventional scalable codecs and enables the streaming of holographic data with a better quality of experience

Towards practical hologram streaming using progressive coding

Digital holography is an emerging technology for 3D visualization which is expected to dethrone conventional stereoscopic devices in the future. Aside from their specific signal properties, high quality holograms with broad viewing angles contain massive amount of data. For a reasonable transmission time, efficient scalable compression schemes are needed to bridge the gap between the overwhelming volume of data and the limited bandwidth of the communication channels. The viewpoint scalability is a powerful property since it allows to encode and transmit only the information corresponding to the observer's view. However, this approach imposes an online encoding at the server which may increase the latency of the transmission chain. To overcome this hurdle, we propose a scalable compression framework based on Gabor-wavelets decomposition, where the whole hologram is encoded offline. First, the observer plane is divided into spatial blocks. Then, the Gabor atoms are assigned to these blocks by exploiting the duality between Gabor wavelets and light rays. The atoms of each block are then classified into different layers according to their importance for the reconstruction and encoded in packets. At the decoder side, the atoms' packets are progressively decoded based on the viewer's position. Then, the corresponding sub-hologram is generated using a GPU implementation. Results show that our approach enables a practical progressive streaming of digital holograms with a low latency

Advanced three-dimensional multi-view video coding and evaluation techniques.

3D video services constitute the next step in multimedia services, as they give the chance of more natural visualisation and provide a sense of "being there". A lot of research effort has been put towards to realising 3D video services, especially in the context of stereoscopic video. 3D multi-view video is a step beyond stereoscopic video, creating much wider scene navigation range and improved user interaction, despite the higher source data size. With the aid of the extracted scene geometry and depth information, any arbitrary viewpoint can be reconstructed. The level of research in multi-view video is not as mature as the level of research in stereoscopic video, although there is a lot of ongoing work towards the realisation of practical multi-view video based 3D video applications. This thesis addresses compression and quality assessment related aspects of 3D multi-view video, for reduced bandwidth usage and more reliable evaluation of perceived quality. In the first part of the thesis, efficient compression algorithms for multi-view video with depth information that take into account several constraints are studied. These include the ease of viewpoint scalability and fast viewpoint random access. To be standards conformant, the proposed methods are implemented on the multi-view codec standard. The second part of the thesis studies processing and block based coding approaches for depth map video sequences, taking into account their special characteristics and effects on the 3D scene reconstruction process. Some state-of-the-art compression approaches, like reduced resolution coding, are extended to exploit scene texture and geometry information for improved performance. The last part of the thesis is devoted to the quality assessment problem for synthesized camera viewpoints, a core element of multi-view based free-viewpoint video applications. Depth based rendering related aspects are exploited to quantify the objective quality of synthesized scenes in an improved way, by extending the state-of-the-art 2D video quality assessment tools