End-to-end Projector Photometric Compensation

Projector photometric compensation aims to modify a projector input image such that it can compensate for disturbance from the appearance of projection surface. In this paper, for the first time, we formulate the compensation problem as an end-to-end learning problem and propose a convolutional neural network, named CompenNet, to implicitly learn the complex compensation function. CompenNet consists of a UNet-like backbone network and an autoencoder subnet. Such architecture encourages rich multi-level interactions between the camera-captured projection surface image and the input image, and thus captures both photometric and environment information of the projection surface. In addition, the visual details and interaction information are carried to deeper layers along the multi-level skip convolution layers. The architecture is of particular importance for the projector compensation task, for which only a small training dataset is allowed in practice. Another contribution we make is a novel evaluation benchmark, which is independent of system setup and thus quantitatively verifiable. Such benchmark is not previously available, to our best knowledge, due to the fact that conventional evaluation requests the hardware system to actually project the final results. Our key idea, motivated from our end-to-end problem formulation, is to use a reasonable surrogate to avoid such projection process so as to be setup-independent. Our method is evaluated carefully on the benchmark, and the results show that our end-to-end learning solution outperforms state-of-the-arts both qualitatively and quantitatively by a significant margin.Comment: To appear in the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Source code and dataset are available at https://github.com/BingyaoHuang/compenne

Projector Compensation for Unconventional Projection Surface

Projecting onto irregular textured surfaces found on buildings, automobiles and theatre stages calls for the development of radiometric and geometric compensation algorithms that require no user intervention and compensate for the patterning and colourization of the background surface. This process needs a projector-camera setup where the feedback from the camera is used to learn the background's geometric and radiometric properties. In this thesis, radiometric compensation, which is used to correct for the background texture distortion, is discussed in detail. Existing compensation frameworks assume no inter--pixel coupling and develop an independent compensation model for each projector pixel. This assumption is valid on background with uniform texture variation but fails at sharp contrast differences leading to visible edge artifacts in the compensated image. To overcome the edge artifacts, a novel radiometric compensation approach is presented that directly learns the compensation model, rather than inverting a learned forward model. That is, the proposed method uses spatially uniform camera images to learn the projector images that successfully hide the background. The proposed approach can be used with any existing radiometric compensation algorithm to improve its performance. Comparisons with classical and state-of-the-art methods show the superiority of the proposed method in terms of the perceived image quality and computational complexity. The modified target image from the radiometric compensation algorithm can exceed the limited dynamic range of the projector resulting in saturation artifacts in the compensated image. Since the achievable range of luminance on the background surface with the given projector is limited, the projector compensation should also consider the contents of the target image along with the background properties while calculating the projector image. A novel spatially optimized luminance modification approach is proposed using human visual system properties to reduce the saturation artifacts. Here, the tolerance of the human visual system is exploited to make perceptually less sensitive modifications to the target image that in turn reduces the luminance demands from the projector. The proposed spatial modification approach can be combined with any radiometric compensation models to improve its performance. The simulated results of the proposed luminance modification are evaluated to show the improvement in perceptual performance. The inverse approach combined with the spatial luminance modification concludes the proposed projector compensation, which enables the optimum compensated projection on an arbitrary background surface

Inverse tone mapping

The introduction of High Dynamic Range Imaging in computer graphics has produced a novelty in Imaging that can be compared to the introduction of colour photography or even more. Light can now be captured, stored, processed, and finally visualised without losing information. Moreover, new applications that can exploit physical values of the light have been introduced such as re-lighting of synthetic/real objects, or enhanced visualisation of scenes. However, these new processing and visualisation techniques cannot be applied to movies and pictures that have been produced by photography and cinematography in more than one hundred years. This thesis introduces a general framework for expanding legacy content into High Dynamic Range content. The expansion is achieved avoiding artefacts, producing images suitable for visualisation and re-lighting of synthetic/real objects. Moreover, it is presented a methodology based on psychophysical experiments and computational metrics to measure performances of expansion algorithms. Finally, a compression scheme, inspired by the framework, for High Dynamic Range Textures, is proposed and evaluated

Inverse tone mapping

The introduction of High Dynamic Range Imaging in computer graphics has produced a novelty in Imaging that can be compared to the introduction of colour photography or even more. Light can now be captured, stored, processed, and finally visualised without losing information. Moreover, new applications that can exploit physical values of the light have been introduced such as re-lighting of synthetic/real objects, or enhanced visualisation of scenes. However, these new processing and visualisation techniques cannot be applied to movies and pictures that have been produced by photography and cinematography in more than one hundred years. This thesis introduces a general framework for expanding legacy content into High Dynamic Range content. The expansion is achieved avoiding artefacts, producing images suitable for visualisation and re-lighting of synthetic/real objects. Moreover, it is presented a methodology based on psychophysical experiments and computational metrics to measure performances of expansion algorithms. Finally, a compression scheme, inspired by the framework, for High Dynamic Range Textures, is proposed and evaluated.EThOS - Electronic Theses Online ServiceEngineering and Physical Sciences Research Council (EPSRC) (EP/D032148)GBUnited Kingdo

Quality of Experience in Immersive Video Technologies

Over the last decades, several technological revolutions have impacted the television industry, such as the shifts from black & white to color and from standard to high-definition. Nevertheless, further considerable improvements can still be achieved to provide a better multimedia experience, for example with ultra-high-definition, high dynamic range & wide color gamut, or 3D. These so-called immersive technologies aim at providing better, more realistic, and emotionally stronger experiences. To measure quality of experience (QoE), subjective evaluation is the ultimate means since it relies on a pool of human subjects. However, reliable and meaningful results can only be obtained if experiments are properly designed and conducted following a strict methodology. In this thesis, we build a rigorous framework for subjective evaluation of new types of image and video content. We propose different procedures and analysis tools for measuring QoE in immersive technologies. As immersive technologies capture more information than conventional technologies, they have the ability to provide more details, enhanced depth perception, as well as better color, contrast, and brightness. To measure the impact of immersive technologies on the viewersâ QoE, we apply the proposed framework for designing experiments and analyzing collected subjectsâ ratings. We also analyze eye movements to study human visual attention during immersive content playback. Since immersive content carries more information than conventional content, efficient compression algorithms are needed for storage and transmission using existing infrastructures. To determine the required bandwidth for high-quality transmission of immersive content, we use the proposed framework to conduct meticulous evaluations of recent image and video codecs in the context of immersive technologies. Subjective evaluation is time consuming, expensive, and is not always feasible. Consequently, researchers have developed objective metrics to automatically predict quality. To measure the performance of objective metrics in assessing immersive content quality, we perform several in-depth benchmarks of state-of-the-art and commonly used objective metrics. For this aim, we use ground truth quality scores, which are collected under our subjective evaluation framework. To improve QoE, we propose different systems for stereoscopic and autostereoscopic 3D displays in particular. The proposed systems can help reducing the artifacts generated at the visualization stage, which impact picture quality, depth quality, and visual comfort. To demonstrate the effectiveness of these systems, we use the proposed framework to measure viewersâ preference between these systems and standard 2D & 3D modes. In summary, this thesis tackles the problems of measuring, predicting, and improving QoE in immersive technologies. To address these problems, we build a rigorous framework and we apply it through several in-depth investigations. We put essential concepts of multimedia QoE under this framework. These concepts not only are of fundamental nature, but also have shown their impact in very practical applications. In particular, the JPEG, MPEG, and VCEG standardization bodies have adopted these concepts to select technologies that were proposed for standardization and to validate the resulting standards in terms of compression efficiency

Eighth International Symposium “Monitoring of Mediterranean Coastal Areas. Problems and Measurement Techniques”

The 8th International Symposium "Monitoring of Mediterranean Coastal Areas. Problems and Measurements Techniques" was organized by CNR-IBE in collaboration with FCS Foundation, and Natural History Museum of the Mediterranean and under the patronage of University of Florence, Accademia dei Geogofili, Tuscany Region and Livorno Province. It is the occasion in which scholars can illustrate and exchange their activities and innovative proposals, with common aims to promote actions to preserve coastal marine environment. Considering Symposium interdisciplinary nature, the Scientific Committee, underlining this holistic view of Nature, decided to celebrate Alexander von Humboldt; a nature scholar that proposed the organic and inorganic nature’s aspects as a single system. It represents a sign of continuity considering that in-presence Symposium could not be carried out due to the COVID-19 pandemic restrictions. Subjects are related to coastal topics: morphology; flora and fauna; energy production; management and integrated protection; geography and landscape, cultural heritage and environmental assets, legal and economic aspects

Proceedings of Eighth International Symposium “Monitoring of Mediterranean Coastal Areas. Problems and Measurement Techniques”

The 8th International Symposium "Monitoring of Mediterranean Coastal Areas. Problems and Measurements Techniques" was organized by CNR-IBE in collaboration with FCS Foundation, and Natural History Museum of the Mediterranean and under the patronage of University of Florence, Accademia dei Geogofili, Tuscany Region and Livorno Province. It is the occasion in which scholars can illustrate and exchange their activities and innovative proposals, with common aims to promote actions to preserve coastal marine environment. Considering Symposium interdisciplinary nature, the Scientific Committee, underlining this holistic view of Nature, decided to celebrate Alexander von Humboldt; a nature scholar that proposed the organic and inorganic nature’s aspects as a single system. It represents a sign of continuity considering that in-presence Symposium could not be carried out due to the COVID-19 pandemic restrictions. Subjects are related to coastal topics: morphology; flora and fauna; energy production; management and integrated protection; geography and landscape, cultural heritage and environmental assets, legal and economic aspects

Neolithic land-use in the Dutch wetlands: estimating the land-use implications of resource exploitation strategies in the Middle Swifterbant Culture (4600-3900 BCE)

The Dutch wetlands witness the gradual adoption of Neolithic novelties by foraging societies during the Swifterbant period. Recent analyses provide new insights into the subsistence palette of Middle Swifterbant societies. Small-scale livestock herding and cultivation are in evidence at this time, but their importance if unclear. Within the framework of PAGES Land-use at 6000BP project, we aim to translate the information on resource exploitation into information on land-use that can be incorporated into global climate modelling efforts, with attention for the importance of agriculture. A reconstruction of patterns of resource exploitation and their land-use dimensions is complicated by methodological issues in comparing the results of varied recent investigations. Analyses of organic residues in ceramics have attested to the cooking of aquatic foods, ruminant meat, porcine meat, as well as rare cases of dairy. In terms of vegetative matter, some ceramics exclusively yielded evidence of wild plants, while others preserve cereal remains. Elevated δ15N values of human were interpreted as demonstrating an important aquatic component of the diet well into the 4th millennium BC. Yet recent assays on livestock remains suggest grazing on salt marshes partly accounts for the human values. Finally, renewed archaeozoological investigations have shown the early presence of domestic animals to be more limited than previously thought. We discuss the relative importance of exploited resources to produce a best-fit interpretation of changing patterns of land-use during the Middle Swifterbant phase. Our review combines recent archaeological data with wider data on anthropogenic influence on the landscape. Combining the results of plant macroremains, information from pollen cores about vegetation development, the structure of faunal assemblages, and finds of arable fields and dairy residue, we suggest the most parsimonious interpretation is one of a limited land-use footprint of cultivation and livestock keeping in Dutch wetlands between 4600 and 3900 BCE.NWOVidi 276-60-004Human Origin

Taphonomy, environment or human plant exploitation strategies?: Deciphering changes in Pleistocene-Holocene plant representation at Umhlatuzana rockshelter, South Africa

The period between ~40 and 20 ka BP encompassing the Middle Stone Age (MSA) and Later Stone Age (LSA) transition has long been of interest because of the associated technological change. Understanding this transition in southern Africa is complicated by the paucity of archaeological sites that span this period. With its occupation sequence spanning the last ~70,000 years, Umhlatuzana Rock Shelter is one of the few sites that record this transition. Umhlatuzana thus offers a great opportunity to study past environmental dynamics from the Late Pleistocene (MIS 4) to the Late Holocene, and past human subsistence strategies, their social organisation, technological and symbolic innovations. Although organic preservation is poor (bones, seeds, and charcoal) at the site, silica phytoliths preserve generally well throughout the sequence. These microscopic silica particles can identify different plant types that are no longer visible at the site because of decomposition or burning to a reliable taxonomical level. Thus, to trace site occupation, plant resource use, and in turn reconstruct past vegetation, we applied phytolith analyses to sediment samples of the newly excavated Umhlatuzana sequence. We present results of the phytolith assemblage variability to determine change in plant use from the Pleistocene to the Holocene and discuss them in relation to taphonomical processes and human plant gathering strategies and activities. This study ultimately seeks to provide a palaeoenvironmental context for modes of occupation and will shed light on past human-environmental interactions in eastern South Africa.NWOVidi 276-60-004Human Origin