Electronic Imaging & the Visual Arts. EVA 2018 Florence

The Publication is following the yearly Editions of EVA FLORENCE. The State of Art is presented regarding the Application of Technologies (in particular of digital type) to Cultural Heritage. The more recent results of the Researches in the considered Area are presented. Information Technologies of interest for Culture Heritage are presented: multimedia systems, data-bases, data protection, access to digital content, Virtual Galleries. Particular reference is reserved to digital images (Electronic Imaging & the Visual Arts), regarding Cultural Institutions (Museums, Libraries, Palace - Monuments, Archaeological Sites). The International Conference includes the following Sessions: Strategic Issues; New Sciences and Culture Developments and Applications; New Technical Developments & Applications; Museums - Virtual Galleries and Related Initiatives; Art and Humanities Ecosystem & Applications; Access to the Culture Information. Two Workshops regard: Innovation and Enterprise; the Cloud Systems connected to the Culture (eCulture Cloud) in the Smart Cities context. The more recent results of the Researches at national and international are reported in the Area of Technologies and Culture Heritage, also with experimental demonstrations of developed Activities

Random access prediction structures for light field video coding with MV-HEVC

Computational imaging and light field technology promise to deliver the required six-degrees-of-freedom for natural scenes in virtual reality. Already existing extensions of standardized video coding formats, such as multi-view coding and multi-view plus depth, are the most conventional light field video coding solutions at the moment. The latest multi-view coding format, which is a direct extension of the high efficiency video coding (HEVC) standard, is called multi-view HEVC (or MV-HEVC). MV-HEVC treats each light field view as a separate video sequence, and uses syntax elements similar to standard HEVC for exploiting redundancies between neighboring views. To achieve this, inter-view and temporal prediction schemes are deployed with the aim to find the most optimal trade-off between coding performance and reconstruction quality. The number of possible prediction structures is unlimited and many of them are proposed in the literature. Although some of them are efficient in terms of compression ratio, they complicate random access due to the dependencies on previously decoded pixels or frames. Random access is an important feature in video delivery, and a crucial requirement in multi-view video coding. In this work, we propose and compare different prediction structures for coding light field video using MV-HEVC with a focus on both compression efficiency and random accessibility. Experiments on three different short-baseline light field video sequences show the trade-off between bit-rate and distortion, as well as the average number of decoded views/frames, necessary for displaying any random frame at any time instance. The findings of this work indicate the most appropriate prediction structure depending on the available bandwidth and the required degree of random access

End-to-end 3D video communication over heterogeneous networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Three-dimensional technology, more commonly referred to as 3D technology, has revolutionised many fields including entertainment, medicine, and communications to name a few. In addition to 3D films, games, and sports channels, 3D perception has made tele-medicine a reality. By the year 2015, 30% of the all HD panels at home will be 3D enabled, predicted by consumer electronics manufacturers. Stereoscopic cameras, a comparatively mature technology compared to other 3D systems, are now being used by ordinary citizens to produce 3D content and share at a click of a button just like they do with the 2D counterparts via sites like YouTube. But technical challenges still exist, including with autostereoscopic multiview displays. 3D content requires many complex considerations--including how to represent it, and deciphering what is the best compression format--when considering transmission or storage, because of its increased amount of data. Any decision must be taken in the light of the available bandwidth or storage capacity, quality and user expectations. Free viewpoint navigation also remains partly unsolved. The most pressing issue getting in the way of widespread uptake of consumer 3D systems is the ability to deliver 3D content to heterogeneous consumer displays over the heterogeneous networks. Optimising 3D video communication solutions must consider the entire pipeline, starting with optimisation at the video source to the end display and transmission optimisation. Multi-view offers the most compelling solution for 3D videos with motion parallax and freedom from wearing headgear for 3D video perception. Optimising multi-view video for delivery and display could increase the demand for true 3D in the consumer market. This thesis focuses on an end-to-end quality optimisation in 3D video communication/transmission, offering solutions for optimisation at the compression, transmission, and decoder levels.Brunel University - Isambard Research Scholarshi

Methods for Light Field Display Profiling and Scalable Super-Multiview Video Coding

Light field 3D displays reproduce the light field of real or synthetic scenes, as observed by multiple viewers, without the necessity of wearing 3D glasses. Reproducing light fields is a technically challenging task in terms of optical setup, content creation, distributed rendering, among others; however, the impressive visual quality of hologramlike scenes, in full color, with real-time frame rates, and over a very wide field of view justifies the complexity involved. Seeing objects popping far out from the screen plane without glasses impresses even those viewers who have experienced other 3D displays before.Content for these displays can either be synthetic or real. The creation of synthetic (rendered) content is relatively well understood and used in practice. Depending on the technique used, rendering has its own complexities, quite similar to the complexity of rendering techniques for 2D displays. While rendering can be used in many use-cases, the holy grail of all 3D display technologies is to become the future 3DTVs, ending up in each living room and showing realistic 3D content without glasses. Capturing, transmitting, and rendering live scenes as light fields is extremely challenging, and it is necessary if we are about to experience light field 3D television showing real people and natural scenes, or realistic 3D video conferencing with real eye-contact.In order to provide the required realism, light field displays aim to provide a wide field of view (up to 180°), while reproducing up to ~80 MPixels nowadays. Building gigapixel light field displays is realistic in the next few years. Likewise, capturing live light fields involves using many synchronized cameras that cover the same display wide field of view and provide the same high pixel count. Therefore, light field capture and content creation has to be well optimized with respect to the targeted display technologies. Two major challenges in this process are addressed in this dissertation.The first challenge is how to characterize the display in terms of its capabilities to create light fields, that is how to profile the display in question. In clearer terms this boils down to finding the equivalent spatial resolution, which is similar to the screen resolution of 2D displays, and angular resolution, which describes the smallest angle, the color of which the display can control individually. Light field is formalized as 4D approximation of the plenoptic function in terms of geometrical optics through spatiallylocalized and angularly-directed light rays in the so-called ray space. Plenoptic Sampling Theory provides the required conditions to sample and reconstruct light fields. Subsequently, light field displays can be characterized in the Fourier domain by the effective display bandwidth they support. In the thesis, a methodology for displayspecific light field analysis is proposed. It regards the display as a signal processing channel and analyses it as such in spectral domain. As a result, one is able to derive the display throughput (i.e. the display bandwidth) and, subsequently, the optimal camera configuration to efficiently capture and filter light fields before displaying them.While the geometrical topology of optical light sources in projection-based light field displays can be used to theoretically derive display bandwidth, and its spatial and angular resolution, in many cases this topology is not available to the user. Furthermore, there are many implementation details which cause the display to deviate from its theoretical model. In such cases, profiling light field displays in terms of spatial and angular resolution has to be done by measurements. Measurement methods that involve the display showing specific test patterns, which are then captured by a single static or moving camera, are proposed in the thesis. Determining the effective spatial and angular resolution of a light field display is then based on an automated analysis of the captured images, as they are reproduced by the display, in the frequency domain. The analysis reveals the empirical limits of the display in terms of pass-band both in the spatial and angular dimension. Furthermore, the spatial resolution measurements are validated by subjective tests confirming that the results are in line with the smallest features human observers can perceive on the same display. The resolution values obtained can be used to design the optimal capture setup for the display in question.The second challenge is related with the massive number of views and pixels captured that have to be transmitted to the display. It clearly requires effective and efficient compression techniques to fit in the bandwidth available, as an uncompressed representation of such a super-multiview video could easily consume ~20 gigabits per second with today’s displays. Due to the high number of light rays to be captured, transmitted and rendered, distributed systems are necessary for both capturing and rendering the light field. During the first attempts to implement real-time light field capturing, transmission and rendering using a brute force approach, limitations became apparent. Still, due to the best possible image quality achievable with dense multi-camera light field capturing and light ray interpolation, this approach was chosen as the basis of further work, despite the massive amount of bandwidth needed. Decompression of all camera images in all rendering nodes, however, is prohibitively time consuming and is not scalable. After analyzing the light field interpolation process and the data-access patterns typical in a distributed light field rendering system, an approach to reduce the amount of data required in the rendering nodes has been proposed. This approach, on the other hand, requires rectangular parts (typically vertical bars in case of a Horizontal Parallax Only light field display) of the captured images to be available in the rendering nodes, which might be exploited to reduce the time spent with decompression of video streams. However, partial decoding is not readily supported by common image / video codecs. In the thesis, approaches aimed at achieving partial decoding are proposed for H.264, HEVC, JPEG and JPEG2000 and the results are compared.The results of the thesis on display profiling facilitate the design of optimal camera setups for capturing scenes to be reproduced on 3D light field displays. The developed super-multiview content encoding also facilitates light field rendering in real-time. This makes live light field transmission and real-time teleconferencing possible in a scalable way, using any number of cameras, and at the spatial and angular resolution the display actually needs for achieving a compelling visual experience

Algorithm/Architecture Co-Exploration of Visual Computing: Overview and Future Perspectives

Concurrently exploring both algorithmic and architectural optimizations is a new design paradigm. This survey paper addresses the latest research and future perspectives on the simultaneous development of video coding, processing, and computing algorithms with emerging platforms that have multiple cores and reconfigurable architecture. As the algorithms in forthcoming visual systems become increasingly complex, many applications must have different profiles with different levels of performance. Hence, with expectations that the visual experience in the future will become continuously better, it is critical that advanced platforms provide higher performance, better flexibility, and lower power consumption. To achieve these goals, algorithm and architecture co-design is significant for characterizing the algorithmic complexity used to optimize targeted architecture. This paper shows that seamless weaving of the development of previously autonomous visual computing algorithms and multicore or reconfigurable architectures will unavoidably become the leading trend in the future of video technology

Quality of Experience in Immersive Video Technologies

Over the last decades, several technological revolutions have impacted the television industry, such as the shifts from black & white to color and from standard to high-definition. Nevertheless, further considerable improvements can still be achieved to provide a better multimedia experience, for example with ultra-high-definition, high dynamic range & wide color gamut, or 3D. These so-called immersive technologies aim at providing better, more realistic, and emotionally stronger experiences. To measure quality of experience (QoE), subjective evaluation is the ultimate means since it relies on a pool of human subjects. However, reliable and meaningful results can only be obtained if experiments are properly designed and conducted following a strict methodology. In this thesis, we build a rigorous framework for subjective evaluation of new types of image and video content. We propose different procedures and analysis tools for measuring QoE in immersive technologies. As immersive technologies capture more information than conventional technologies, they have the ability to provide more details, enhanced depth perception, as well as better color, contrast, and brightness. To measure the impact of immersive technologies on the viewersâ QoE, we apply the proposed framework for designing experiments and analyzing collected subjectsâ ratings. We also analyze eye movements to study human visual attention during immersive content playback. Since immersive content carries more information than conventional content, efficient compression algorithms are needed for storage and transmission using existing infrastructures. To determine the required bandwidth for high-quality transmission of immersive content, we use the proposed framework to conduct meticulous evaluations of recent image and video codecs in the context of immersive technologies. Subjective evaluation is time consuming, expensive, and is not always feasible. Consequently, researchers have developed objective metrics to automatically predict quality. To measure the performance of objective metrics in assessing immersive content quality, we perform several in-depth benchmarks of state-of-the-art and commonly used objective metrics. For this aim, we use ground truth quality scores, which are collected under our subjective evaluation framework. To improve QoE, we propose different systems for stereoscopic and autostereoscopic 3D displays in particular. The proposed systems can help reducing the artifacts generated at the visualization stage, which impact picture quality, depth quality, and visual comfort. To demonstrate the effectiveness of these systems, we use the proposed framework to measure viewersâ preference between these systems and standard 2D & 3D modes. In summary, this thesis tackles the problems of measuring, predicting, and improving QoE in immersive technologies. To address these problems, we build a rigorous framework and we apply it through several in-depth investigations. We put essential concepts of multimedia QoE under this framework. These concepts not only are of fundamental nature, but also have shown their impact in very practical applications. In particular, the JPEG, MPEG, and VCEG standardization bodies have adopted these concepts to select technologies that were proposed for standardization and to validate the resulting standards in terms of compression efficiency

Evolution of the media market and its legal framework in Bosnia and Herzegovina since the independence: special focus on defamation

Encara que a Bòsnia i Hercegovina hi hagi un marc legislatiu que ofereix disposicions legals per a garantir la llibertat d’expressió, la seva aplicació real és molt dificultosa a causa dels comunistes i dels llegats de guerra. I és que la divisió ètnica i les pressions polítiques dins del sistema judicial, així com la influència política en els mitjans de comunicació, en perjudiquen l’exercici autèntic. Malgrat que en l’era actual, els mitjans de comunicació digitals predominin com a principal font per al consum de les notícies, fet que afavoreix la llibertat d’expressió i al pluralisme, la classe política coacciona aquest dret humà i genera un estat d’alerta i por mitjançant l’obertura de procediments legals contra els mitjans de comunicació, els periodistes i altres actors. La tesi analitza el panorama dels mitjans a Bòsnia i Hercegovina i tracta de temes relatius al finançament i a la propietat dels mitjans de comunicació, l’estret vincle entre la premsa i els partits polítics, la falta de coordinació entre els diversos suports mediàtics arreu del país, el marc legislatiu i sobre els organismes reguladors i autoreguladors dels mitjans. Un dels aspectes més rellevants en relació al marc legislatiu dels mitjans de comunicació va ser la despenalització de la difamació, que va ser transferida del Codi Penal al Codi Civil. Però malgrat aquest canvi, encara s’observa l’efecte dissuasiu per al lliure exercici del dret a la llibertat d’expressió, que s’evidencia en les altes compensacions exigides i en un nombre molt elevat de demandes presentades per polítics, tant en contra de periodistes com en contra de l’oposició. Aquest fet, ha cridat l’atenció d’aquesta investigació. Al mateix temps, els polítics no esgoten tots els recursos per a minimitzar el dany que les lleis els permet sobre la difamació. I és que no sol·liciten una retractació pública o disculpa, ni acudeixen als òrgans d’autoregulació, sinó que es dirigeixen directament als tribunals. L'estudi realitza una investigació extensa sobre la literatura local i internacional i sobre els casos de difamació davant els tribunals a Sarajevo i Banjaluka, així com dels casos presentats davant l'organisme d'autoregulació i el Consell de Premsa, amb l'objectiu d'oferir una explicació sobre el concepte de difamació i una anàlisi sobre les tendències mundials i les tendències actuals a Bòsnia i Hercegovina en relació a l'exercici de la llibertat d'expressió. Fins ara, aquest dret no ha estat objecte d’estudi ni a investigacions ni a anàlisis, possiblement, perquè els mitjans de comunicació es mostren reticents a informar sobre temes vinculats a personatges públics i perquè, en general, s’ha desestimat la Llei de Mitjans a B-H. Així, és un tema que encara ha de guanyar força com a àrea d’investigació.Aunque en Bosnia y Herzegovina exista un marco legislativo que ofrece disposiciones generales para garantizar la libertad de expresión, la implementación real de este derecho resulta problemática debido a los comunistas y legados de guerra. Y es que la división étnica y las presiones políticas dentro del sistema judicial, así como la influencia política en los medios de comunicación, perjudican su ejercicio (pleno) en el país. En la era actual, los medios de comunicación online predominan en el consumo de las noticias, hecho que favorece a la libre expresión de ideas y al pluralismo. Aun así, en distintas ocasiones, la clase política coacciona este derecho humano y genera un estado de miedo mediante la apertura de procedimientos legales contra los medios de comunicación, periodistas y otros actores. La tesis analiza el panorama de los medios en Bosnia y Herzegovina y abarca problemas relativos a su financiación y propiedad, el vínculo entre la prensa y los partidos políticos, la falta de coordinación entre los distintos soportes mediáticos en las diversas partes del país, el marco legislativo para los media y sobre los organismos reguladores y de autorregulación. El aspecto más relevante en cuanto a la legislación sobre medios de comunicación fue, entre otros, la despenalización de la difamación, que fue transferida del Código Penal al Código Civil. A pesar de este cambio, todavía se observa un efecto disuasorio para con el libre ejercicio del derecho a la libertad de expresión, que se refleja en las altas compensaciones exigidas y un elevado número de demandas. Esta investigación llama la atención del incremento de demandas presentadas por políticos, tanto en contra de periodistas como de oponentes políticos. Al mismo tiempo, los políticos no agotan todos los recursos para minimizar el daño que permiten las leyes sobre difamación. No solicitan una retractación pública o disculpa, ni acuden a los órganos de autorregulación, sino que acuden directamente a los tribunales. El estudio realiza una investigación extensa sobre la literatura local e internacional y los casos de difamación ante los tribunales en Sarajevo y Banjaluka así como de los asuntos presentados ante el organismo de autorregulación y el Consejo de Prensa, con el fin de ofrecer una explicación sobre el concepto de difamación y un análisis de las tendencias mundiales y de las tendencias actuales en Bosnia y Herzegovina sobre el ejercicio de la libertad de expresión. Hasta ahora, este tema no ha sido objeto de estudio en investigaciones y análisis, posiblemente, porque los medios de comunicación son reacios a informar sobre temas relacionados con personajes públicos y porque, en general, se ha desestimado la Ley de Medios en B-H. Es un tema que aún debe hacerse espacio en las áreas de investigación.Legislative framework in Bosnia and Herzegovina offers provisions that ensure the right to freedom of expression; however, their implementation is problematic due to communist and war legacies. Divisions along ethnic lines, political pressures on judiciary and political influence on the media landscape plague the exercise of freedom of expression in the country. In the era when online media are increasingly taking over the leading position of news consumption there are more options for enjoying freedom of expression, and better opportunities to have pluralism in media. However, in Bosnia and Herzegovina, political figures exercise pressures on enjoying this human right and create chilling effect by bringing numerous lawsuits against media outlets, journalists, and others. The thesis discussed the media landscape in Bosnia and Herzegovina, and troubles regarding financing of media outlets, problematic media ownership, lack of coordination among outlets from different parts of the country, affiliations to political parties, legislative framework for media and regulatory and self-regulatory bodies. The most important issue in terms of media laws was among other the decriminalization of defamation thus defamation was transferred from criminal into civil law, but today the chilling effect is still reflected in high compensations and a great number of lawsuits in general. The thesis notices that political figures bring most defamation lawsuits before courts, either against journalists or against their political opponents. At the same time, political figures do not exhaust all other remedies of minimizing damages which they could do based on defamation laws, they do not ask for a retraction or apology, nor do they turn to the self-regulatory body, they go directly to court instead. The study conducted an extensive research of local and international literature, defamation cases before the courts in Sarajevo and Banjaluka as well as the cases before the self-regulatory body, the Press Council, in order to offer explanation of the concept of defamation, analysis of global trends and of the current trends in Bosnia and Herzegovina regarding the exercise of freedom of expression. This complex topic has not so far been the topic of many researches or analyses possibly because media outlets are often reluctant to report about these issues related to public figures and because media law in B-H in general has been a rather neglected field and it is yet to gain momentum as a field of research

Scalable light field representation and coding

This Thesis aims to advance the state-of-the-art in light field representation and coding. In this context, proposals to improve functionalities like light field random access and scalability are also presented. As the light field representation constrains the coding approach to be used, several light field coding techniques to exploit the inherent characteristics of the most popular types of light field representations are proposed and studied, which are normally based on micro-images or sub-aperture-images. To encode micro-images, two solutions are proposed, aiming to exploit the redundancy between neighboring micro-images using a high order prediction model, where the model parameters are either explicitly transmitted or inferred at the decoder, respectively. In both cases, the proposed solutions are able to outperform low order prediction solutions. To encode sub-aperture-images, an HEVC-based solution that exploits their inherent intra and inter redundancies is proposed. In this case, the light field image is encoded as a pseudo video sequence, where the scanning order is signaled, allowing the encoder and decoder to optimize the reference picture lists to improve coding efficiency. A novel hybrid light field representation coding approach is also proposed, by exploiting the combined use of both micro-image and sub-aperture-image representation types, instead of using each representation individually. In order to aid the fast deployment of the light field technology, this Thesis also proposes scalable coding and representation approaches that enable adequate compatibility with legacy displays (e.g., 2D, stereoscopic or multiview) and with future light field displays, while maintaining high coding efficiency. Additionally, viewpoint random access, allowing to improve the light field navigation and to reduce the decoding delay, is also enabled with a flexible trade-off between coding efficiency and viewpoint random access.Esta Tese tem como objetivo avançar o estado da arte em representação e codificação de campos de luz. Neste contexto, são também apresentadas propostas para melhorar funcionalidades como o acesso aleatório ao campo de luz e a escalabilidade. Como a representação do campo de luz limita a abordagem de codificação a ser utilizada, são propostas e estudadas várias técnicas de codificação de campos de luz para explorar as características inerentes aos seus tipos mais populares de representação, que são normalmente baseadas em micro-imagens ou imagens de sub-abertura. Para codificar as micro-imagens, são propostas duas soluções, visando explorar a redundância entre micro-imagens vizinhas utilizando um modelo de predição de alta ordem, onde os parâmetros do modelo são explicitamente transmitidos ou inferidos no decodificador, respetivamente. Em ambos os casos, as soluções propostas são capazes de superar as soluções de predição de baixa ordem. Para codificar imagens de sub-abertura, é proposta uma solução baseada em HEVC que explora a inerente redundância intra e inter deste tipo de imagens. Neste caso, a imagem do campo de luz é codificada como uma pseudo-sequência de vídeo, onde a ordem de varrimento é sinalizada, permitindo ao codificador e decodificador otimizar as listas de imagens de referência para melhorar a eficiência da codificação. Também é proposta uma nova abordagem de codificação baseada na representação híbrida do campo de luz, explorando o uso combinado dos tipos de representação de micro-imagem e sub-imagem, em vez de usar cada representação individualmente. A fim de facilitar a rápida implantação da tecnologia de campo de luz, esta Tese também propõe abordagens escaláveis de codificação e representação que permitem uma compatibilidade adequada com monitores tradicionais (e.g., 2D, estereoscópicos ou multivista) e com futuros monitores de campo de luz, mantendo ao mesmo tempo uma alta eficiência de codificação. Além disso, o acesso aleatório de pontos de vista, permitindo melhorar a navegação no campo de luz e reduzir o atraso na descodificação, também é permitido com um equilíbrio flexível entre eficiência de codificação e acesso aleatório de pontos de vista