Survey of image-based representations and compression techniques

In this paper, we survey the techniques for image-based rendering (IBR) and for compressing image-based representations. Unlike traditional three-dimensional (3-D) computer graphics, in which 3-D geometry of the scene is known, IBR techniques render novel views directly from input images. IBR techniques can be classified into three categories according to how much geometric information is used: rendering without geometry, rendering with implicit geometry (i.e., correspondence), and rendering with explicit geometry (either with approximate or accurate geometry). We discuss the characteristics of these categories and their representative techniques. IBR techniques demonstrate a surprising diverse range in their extent of use of images and geometry in representing 3-D scenes. We explore the issues in trading off the use of images and geometry by revisiting plenoptic-sampling analysis and the notions of view dependency and geometric proxies. Finally, we highlight compression techniques specifically designed for image-based representations. Such compression techniques are important in making IBR techniques practical.published_or_final_versio

Compressing the illumination-adjustable images with principal component analysis.

Pun-Mo Ho.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 90-95).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Background --- p.1Chapter 1.2 --- Existing Approaches --- p.2Chapter 1.3 --- Our Approach --- p.3Chapter 1.4 --- Structure of the Thesis --- p.4Chapter 2 --- Related Work --- p.5Chapter 2.1 --- Compression for Navigation --- p.5Chapter 2.1.1 --- Light Field/Lumigraph --- p.5Chapter 2.1.2 --- Surface Light Field --- p.6Chapter 2.1.3 --- Concentric Mosaics --- p.6Chapter 2.1.4 --- On the Compression --- p.7Chapter 2.2 --- Compression for Relighting --- p.7Chapter 2.2.1 --- Previous Approaches --- p.7Chapter 2.2.2 --- Our Approach --- p.8Chapter 3 --- Image-Based Relighting --- p.9Chapter 3.1 --- Plenoptic Illumination Function --- p.9Chapter 3.2 --- Sampling and Relighting --- p.11Chapter 3.3 --- Overview --- p.13Chapter 3.3.1 --- Codec Overview --- p.13Chapter 3.3.2 --- Image Acquisition --- p.15Chapter 3.3.3 --- Experiment Data Sets --- p.16Chapter 4 --- Data Preparation --- p.18Chapter 4.1 --- Block Division --- p.18Chapter 4.2 --- Color Model --- p.23Chapter 4.3 --- Mean Extraction --- p.24Chapter 5 --- Principal Component Analysis --- p.29Chapter 5.1 --- Overview --- p.29Chapter 5.2 --- Singular Value Decomposition --- p.30Chapter 5.3 --- Dimensionality Reduction --- p.34Chapter 5.4 --- Evaluation --- p.37Chapter 6 --- Eigenimage Coding --- p.39Chapter 6.1 --- Transform Coding --- p.39Chapter 6.1.1 --- Discrete Cosine Transform --- p.40Chapter 6.1.2 --- Discrete Wavelet Transform --- p.47Chapter 6.2 --- Evaluation --- p.49Chapter 6.2.1 --- Statistical Evaluation --- p.49Chapter 6.2.2 --- Visual Evaluation --- p.52Chapter 7 --- Relighting Coefficient Coding --- p.57Chapter 7.1 --- Quantization and Bit Allocation --- p.57Chapter 7.2 --- Evaluation --- p.62Chapter 7.2.1 --- Statistical Evaluation --- p.62Chapter 7.2.2 --- Visual Evaluation --- p.62Chapter 8 --- Relighting --- p.65Chapter 8.1 --- Overview --- p.66Chapter 8.2 --- First-Phase Decoding --- p.66Chapter 8.3 --- Second-Phase Decoding --- p.68Chapter 8.3.1 --- Software Relighting --- p.68Chapter 8.3.2 --- Hardware-Assisted Relighting --- p.71Chapter 9 --- Overall Evaluation --- p.81Chapter 9.1 --- Compression of IAIs --- p.81Chapter 9.1.1 --- Statistical Evaluation --- p.81Chapter 9.1.2 --- Visual Evaluation --- p.86Chapter 9.2 --- Hardware-Assisted Relighting --- p.86Chapter 10 --- Conclusion --- p.89Bibliography --- p.9

Efficient acquisition, representation and rendering of light fields

In this thesis we discuss the representation of three-dimensional scenes using image data (image-based rendering), and more precisely the so-called light field approach. We start with an up-to-date survey on previous work in this young field of research. Then we propose a light field representation based on image data and additional per-pixel depth values. This enables us to reconstruct arbitrary views of the scene in an efficient way and with high quality. Furtermore, we can use the same representation to determine optimal reference views during the acquisition of a light field. We further present the so-called free form parameterization, which allows for a relatively free placement of reference views. Finally, we demonstrate a prototype of the Lumi-Shelf system, which acquires, transmits, and renders the light field of a dynamic scene at multiple frames per second.Diese Doktorarbeit beschäftigt sich mit der Repräsentierung dreidimensionaler Szenen durch Bilddaten (engl. image-based rendering, deutsch bildbasierte Bildsynthese), speziell mit dem Ansatz des sog. Lichtfelds. Nach einem aktuellen Überblick über bisherige Arbeiten in diesem jungen Forschungsgebiet stellen wir eine Datenrepräsentation vor, die auf Bilddaten mit zusätzlichen Tiefenwerten basiert. Damit sind wir in der Lage, beliebige Ansichten der Szene effizient und in hoher Qualität zu rekonstruieren sowie die optimalen Referenz-Ansichten bei der Akquisition eines Lichtfelds zu bestimmen. Weiterhin präsentieren wir die sog. Freiform-Parametrisierung, die eine relativ freie Anordnung der Referenz-Ansichten erlaubt. Abschließend demonstrieren wir einen Prototyp des Lumishelf-Systems, welches die Aufnahme, Übertragung und Darstellung des Lichtfeldes einer dynamischen Szene mit mehreren Bildern pro Sekunde ermöglicht

Efficient acquisition, representation and rendering of light fields

In this thesis we discuss the representation of three-dimensional scenes using image data (image-based rendering), and more precisely the so-called light field approach. We start with an up-to-date survey on previous work in this young field of research. Then we propose a light field representation based on image data and additional per-pixel depth values. This enables us to reconstruct arbitrary views of the scene in an efficient way and with high quality. Furtermore, we can use the same representation to determine optimal reference views during the acquisition of a light field. We further present the so-called free form parameterization, which allows for a relatively free placement of reference views. Finally, we demonstrate a prototype of the Lumi-Shelf system, which acquires, transmits, and renders the light field of a dynamic scene at multiple frames per second.Diese Doktorarbeit beschäftigt sich mit der Repräsentierung dreidimensionaler Szenen durch Bilddaten (engl. image-based rendering, deutsch bildbasierte Bildsynthese), speziell mit dem Ansatz des sog. Lichtfelds. Nach einem aktuellen Überblick über bisherige Arbeiten in diesem jungen Forschungsgebiet stellen wir eine Datenrepräsentation vor, die auf Bilddaten mit zusätzlichen Tiefenwerten basiert. Damit sind wir in der Lage, beliebige Ansichten der Szene effizient und in hoher Qualität zu rekonstruieren sowie die optimalen Referenz-Ansichten bei der Akquisition eines Lichtfelds zu bestimmen. Weiterhin präsentieren wir die sog. Freiform-Parametrisierung, die eine relativ freie Anordnung der Referenz-Ansichten erlaubt. Abschließend demonstrieren wir einen Prototyp des Lumishelf-Systems, welches die Aufnahme, Übertragung und Darstellung des Lichtfeldes einer dynamischen Szene mit mehreren Bildern pro Sekunde ermöglicht

Smart-rebinning for Compression of Concentric Mosaics

The concentric mosaics offer a quick solution to construct a virtual copy of a real environment, and navigate in the virtual environment. However, the huge data amount associated with the concentric mosaics is a heavy burden for its application. A 3D wavelet transform based compression has been proposed in a previous work to compress the concentric mosaics. In this paper, we greatly improve the performance of the 3D wavelet concentric mosaic coder with a data rearrangement mechanism called smartrebinning. The proposed scheme first aligns the concentric mosaic image shots along the horizontal direction and then rebins the shots into multi-perspective panoramas. Smart rebinning greatly improves the cross shot correlation and enables the coder to better explore the redundancy among shots. Experimental results show that the performance of the 3D wavelet coder improves on average 4.3dB with the use of smart rebinning. The proposed coder outperforms MPEG-2 coding of concentric mosaics by an average of 3.7dB. Keywords Image based rendering, concentric mosaics, compression, rebinning, multi-perspective panorama, 3D wavelet. 1

Annual Meeting of the Lunar Exploration Analysis Group : October 20 - 22, 2015, Columbia, Maryland, USRA Headquarters

The meeting goals are three-fold: 1. Integrate the perspectives and interests of the different stakeholders (science, engineering, government, and private sector) to explore common goals of lunar exploration. This meeting is focused around the identification, evaluation, extraction, and use of lunar resources. 2. Use the results of recent and ongoing missions to examine the dynamic nature of the Moon and how this could influence future science and exploration missions. 3. Provide a forum for community updates and input into the issues that affect lunar science and exploration.NASA Lunar Exploration Analysis Group (LEAG), Universities Space Research Association (USRA), Lunar and Planetary Institute (LPI), NASA Solar System Exploration Research Virtual Institute (SSERVI), National Aeronautics and Space Administration (NASA)Science Organizing Committee, James Carpenter, ESA-ESTE