Interactive illumination and navigation control in an image-based environment.

Fu Chi-wing.Thesis (M.Phil.)--Chinese University of Hong Kong, 1999.Includes bibliographical references (leaves 141-149).Abstract --- p.iAcknowledgments --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Introduction to Image-based Rendering --- p.1Chapter 1.2 --- Scene Complexity Independent Property --- p.2Chapter 1.3 --- Application of this Research Work --- p.3Chapter 1.4 --- Organization of this Thesis --- p.4Chapter 2 --- Illumination Control --- p.7Chapter 2.1 --- Introduction --- p.7Chapter 2.2 --- Apparent BRDF of Pixel --- p.8Chapter 2.3 --- Sampling Illumination Information --- p.11Chapter 2.4 --- Re-rendering --- p.13Chapter 2.4.1 --- Light Direction --- p.15Chapter 2.4.2 --- Light Intensity --- p.15Chapter 2.4.3 --- Multiple Light Sources --- p.15Chapter 2.4.4 --- Type of Light Sources --- p.18Chapter 2.5 --- Data Compression --- p.22Chapter 2.5.1 --- Intra-pixel coherence --- p.22Chapter 2.5.2 --- Inter-pixel coherence --- p.22Chapter 2.6 --- Implementation and Result --- p.22Chapter 2.6.1 --- An Interactive Viewer --- p.22Chapter 2.6.2 --- Lazy Re-rendering --- p.24Chapter 2.7 --- Conclusion --- p.24Chapter 3 --- Navigation Control - Triangle-based Warping Rule --- p.29Chapter 3.1 --- Introduction to Navigation Control --- p.29Chapter 3.2 --- Related Works --- p.30Chapter 3.3 --- Epipolar Geometry (Perspective Projection Manifold) --- p.31Chapter 3.4 --- Drawing Order for Pixel-Sized Entities --- p.35Chapter 3.5 --- Triangle-based Image Warping --- p.36Chapter 3.5.1 --- Image-based Triangulation --- p.36Chapter 3.5.2 --- Image-based Visibility Sorting --- p.40Chapter 3.5.3 --- Topological Sorting --- p.44Chapter 3.6 --- Results --- p.46Chapter 3.7 --- Conclusion --- p.48Chapter 4 --- Panoramic Projection Manifold --- p.52Chapter 4.1 --- Epipolar Geometry (Spherical Projection Manifold) --- p.53Chapter 4.2 --- Image Triangulation --- p.56Chapter 4.2.1 --- Optical Flow --- p.56Chapter 4.2.2 --- Image Gradient and Potential Function --- p.57Chapter 4.2.3 --- Triangulation --- p.58Chapter 4.3 --- Image-based Visibility Sorting --- p.58Chapter 4.3.1 --- Mapping Criteria --- p.58Chapter 4.3.2 --- Ordering of Two Triangles --- p.59Chapter 4.3.3 --- Graph Construction and Topological Sort --- p.63Chapter 4.4 --- Results --- p.63Chapter 4.5 --- Conclusion --- p.65Chapter 5 --- Panoramic-based Navigation using Real Photos --- p.69Chapter 5.1 --- Introduction --- p.69Chapter 5.2 --- System Overview --- p.71Chapter 5.3 --- Correspondence Matching --- p.72Chapter 5.3.1 --- Basic Model of Epipolar Geometry --- p.72Chapter 5.3.2 --- Epipolar Geometry between two Neighbor Panoramic Nodes --- p.73Chapter 5.3.3 --- Line and Patch Correspondence Matching --- p.74Chapter 5.4 --- Triangle-based Warping --- p.75Chapter 5.4.1 --- Why Warp Triangle --- p.75Chapter 5.4.2 --- Patch and Layer Construction --- p.76Chapter 5.4.3 --- Triangulation and Mesh Subdivision --- p.76Chapter 5.4.4 --- Layered Triangle-based Warping --- p.77Chapter 5.5 --- Implementation --- p.78Chapter 5.5.1 --- Image Sampler and Panoramic Stitcher --- p.78Chapter 5.5.2 --- Interactive Correspondence Matcher and Triangulation --- p.79Chapter 5.5.3 --- Basic Panoramic Viewer --- p.79Chapter 5.5.4 --- Formulating Drag Vector and vn --- p.80Chapter 5.5.5 --- Controlling Walkthrough Parameter --- p.82Chapter 5.5.6 --- Interactive Web-based Panoramic Viewer --- p.83Chapter 5.6 --- Results --- p.84Chapter 5.7 --- Conclusion and Possible Enhancements --- p.84Chapter 6 --- Compositing Warped Images for Object-based Viewing --- p.89Chapter 6.1 --- Modeling Object-based Viewing --- p.89Chapter 6.2 --- Triangulation and Convex Hull Criteria --- p.92Chapter 6.3 --- Warping Multiple Views --- p.94Chapter 6.3.1 --- Method I --- p.95Chapter 6.3.2 --- Method II --- p.95Chapter 6.3.3 --- Method III --- p.95Chapter 6.4 --- Results --- p.97Chapter 6.5 --- Conclusion --- p.100Chapter 7 --- Complete Rendering Pipeline --- p.107Chapter 7.1 --- Reviews on Illumination and Navigation --- p.107Chapter 7.1.1 --- Illumination Rendering Pipeline --- p.107Chapter 7.1.2 --- Navigation Rendering Pipeline --- p.108Chapter 7.2 --- Analysis of the Two Rendering Pipelines --- p.109Chapter 7.2.1 --- Combination on the Architectural Level --- p.109Chapter 7.2.2 --- Ensuring Physical Correctness --- p.111Chapter 7.3 --- Generalizing Apparent BRDF --- p.112Chapter 7.3.1 --- Difficulties to Encode BRDF with Spherical Harmonics --- p.112Chapter 7.3.2 --- Generalize Apparent BRDF --- p.112Chapter 7.3.3 --- Related works for Encoding the generalized apparent BRDF --- p.113Chapter 7.4 --- Conclusion --- p.116Chapter 8 --- Conclusion --- p.117Chapter A --- Spherical Harmonics --- p.120Chapter B --- It is Rare for Cycles to Exist in the Drawing Order Graph --- p.123Chapter B.1 --- Theorem 3 --- p.123Chapter B.2 --- Inside and Outside-directed Triangles in a Triangular Cycle --- p.125Chapter B.2.1 --- Assumption --- p.126Chapter B.2.2 --- Inside-directed and Outside-directed triangles --- p.126Chapter B.3 --- Four Possible Cases to Form a Cycle --- p.127Chapter B.3.1 --- Case(l) Triangular Fan --- p.128Chapter B.3.2 --- Case(2) Two Outside-directed Triangles --- p.129Chapter B.3.3 --- Case(3) Three Outside-directed Triangles --- p.130Chapter B.3.4 --- Case(4) More than Three Outside-directed Triangles --- p.131Chapter B.4 --- Experiment --- p.132Chapter C --- Deriving the Epipolar Line Formula on Cylindrical Projection Manifold --- p.133Chapter C.1 --- Notations --- p.133Chapter C.2 --- General Formula --- p.134Chapter C.3 --- Simplify the General Formula to a Sine Curve --- p.137Chapter C.4 --- Show that the Epipolar Line is a Sine Curve Segment --- p.139Chapter D --- Publications Related to this Research Work --- p.141Bibliography --- p.14

Expressive rendering of mountainous terrain

technical reportPainters and cartographers have developed artistic landscape rendering techniques for centuries. Such renderings can visualize complex three-dimensional landscapes in a pleasing and understandable way. In this work we examine a particular type of artistic depiction, panorama maps, in terms of function and style, and we develop methods to automatically generate panorama map reminiscent renderings from GIS data. In particular, we develop image-based procedural surface textures for mountainous terrain. Our methods use the structural information present in the terrain and are developed with perceptual metrics and artistic considerations in mind

Surface Deformation Potentials on Meshes for Computer Graphics and Visualization

Shape deformation models have been used in computer graphics primarily to describe the dynamics of physical deformations like cloth draping, collisions of elastic bodies, fracture, or animation of hair. Less frequent is their application to problems not directly related to a physical process. In this thesis we apply deformations to three problems in computer graphics that do not correspond to physical deformations. To this end, we generalize the physical model by modifying the energy potential. Originally, the energy potential amounts to the physical work needed to deform a body from its rest state into a given configuration and relates material strain to internal restoring forces that act to restore the original shape. For each of the three problems considered, this potential is adapted to reflect an application specific notion of shape. Under the influence of further constraints, our generalized deformation results in shapes that balance preservation of certain shape properties and application specific objectives similar to physical equilibrium states. The applications discussed in this thesis are surface parameterization, interactive shape editing and automatic design of panorama maps. For surface parameterization, we interpret parameterizations over a planar domain as deformations from a flat initial configuration onto a given surface. In this setting, we review existing parameterization methods by analyzing properties of their potential functions and derive potentials accounting for distortion of geometric properties. Interactive shape editing allows an untrained user to modify complex surfaces, be simply grabbing and moving parts of interest. A deformation model interactively extrapolates the transformation from those parts to the rest of the surface. This thesis proposes a differential shape representation for triangle meshes leading to a potential that can be optimized interactively with a simple, tailored algorithm. Although the potential is not physically accurate, it results in intuitive deformation behavior and can be parameterized to account for different material properties. Panorama maps are blends between landscape illustrations and geographic maps that are traditionally painted by an artist to convey geographic surveyknowledge on public places like ski resorts or national parks. While panorama maps are not drawn to scale, the shown landscape remains recognizable and the observer can easily recover details necessary for self location and orientation. At the same time, important features as trails or ski slopes appear not occluded and well visible. This thesis proposes the first automatic panorama generation method. Its basis is again a surface deformation, that establishes the necessary compromise between shape preservation and feature visibility.Potentiale zur Flächendeformation auf Dreiecksnetzen für Anwendungen in der Computergrafik und Visualisierung Deformationsmodelle werden in der Computergrafik bislang hauptsächlich eingesetzt, um die Dynamik physikalischer Deformationsprozesse zu modellieren. Gängige Beispiele sind Bekleidungssimulationen, Kollisionen elastischer Körper oder Animation von Haaren und Frisuren. Deutlich seltener ist ihre Anwendung auf Probleme, die nicht direkt physikalischen Prozessen entsprechen. In der vorliegenden Arbeit werden Deformationsmodelle auf drei Probleme der Computergrafik angewandt, die nicht unmittelbar einem physikalischen Deformationsprozess entsprechen. Zu diesem Zweck wird das physikalische Modell durch eine passende Änderung der potentiellen Energie verallgemeinert. Die potentielle Energie entspricht normalerweise der physikalischen Arbeit, die aufgewendet werden muss, um einen Körper aus dem Ruhezustand in eine bestimmte Konfiguration zu verformen. Darüber hinaus setzt sie die aktuelle Verformung in Beziehung zu internen Spannungskräften, die wirken um die ursprüngliche Form wiederherzustellen. In dieser Arbeit passen wir für jedes der drei betrachteten Problemfelder die potentielle Energie jeweils so an, dass sie eine anwendungsspezifische Definition von Form widerspiegelt. Unter dem Einfluss weiterer Randbedingungen führt die so verallgemeinerte Deformation zu einer Fläche, die eine Balance zwischen der Erhaltung gewisser Formeigenschaften und Zielvorgaben der Anwendung findet. Diese Balance entspricht dem Equilibrium einer physikalischen Deformation. Die drei in dieser Arbeit diskutierten Anwendungen sind Oberflächenparameterisierung, interaktives Bearbeiten von Flächen und das vollautomatische Erzeugen von Panoramakarten im Stile von Heinrich Berann. Zur Oberflächenparameterisierung interpretieren wir Parameterisierungen über einem flachen Parametergebiet als Deformationen, die ein ursprünglich ebenes Flächenstück in eine gegebene Oberfläche verformen. Innerhalb dieses Szenarios vergleichen wir dann existierende Methoden zur planaren Parameterisierung, indem wir die resultierenden potentiellen Energien analysieren, und leiten weitere Potentiale her, die die Störung geometrischer Eigenschaften wie Fläche und Winkel erfassen. Verfahren zur interaktiven Flächenbearbeitung ermöglichen schnelle und intuitive Änderungen an einer komplexen Oberfläche. Dazu wählt der Benutzer Teile der Fläche und bewegt diese durch den Raum. Ein Deformationsmodell extrapoliert interaktiv die Transformation der gewählten Teile auf die restliche Fläche. Diese Arbeit stellt eine neue differentielle Flächenrepräsentation für diskrete Flächen vor, die zu einem einfach und interaktiv zu optimierendem Potential führt. Obwohl das vorgeschlagene Potential nicht physikalisch korrekt ist, sind die resultierenden Deformationen intuitiv. Mittels eines Parameters lassen sich außerdem bestimmte Materialeigenschaften einstellen. Panoramakarten im Stile von Heinrich Berann sind eine Verschmelzung von Landschaftsillustration und geographischer Karte. Traditionell werden sie so von Hand gezeichnet, dass bestimmt Merkmale wie beispielsweise Skipisten oder Wanderwege in einem Gebiet unverdeckt und gut sichtbar bleiben, was große Kunstfertigkeit verlangt. Obwohl diese Art der Darstellung nicht maßstabsgetreu ist, sind Abweichungen auf den ersten Blick meistens nicht zu erkennen. Dadurch kann der Betrachter markante Details schnell wiederfinden und sich so innerhalb des Gebietes orientieren. Diese Arbeit stellt das erste, vollautomatische Verfahren zur Erzeugung von Panoramakarten vor. Grundlage ist wiederum eine verallgemeinerte Oberflächendeformation, die sowohl auf Formerhaltung als auch auf die Sichtbarkeit vorgegebener geographischer Merkmale abzielt

Interactive Manipulation of 3D Scene Projections

Linear perspective is a good approximation to the format in which the human visual system conveys 3D scene information to the brain. Artists expressing 3D scenes, however, create nonlinear projections that balance their linear perspective view of a scene with elements of aesthetic style, layout and relative importance of scene objects. Manipulating the many parameters of a linear perspective camera to achieve a desired view is not easy. Controlling and combining mul-tiple such cameras to specify a nonlinear projection is an even more cumbersome task. This paper presents a direct interface, where an artist manipulates in 2D the desired projection of a few features of the 3D scene. The features represent a rich set of constraints which define the overall projection of the 3D scene. Desirable properties of local linear perspective and global scene coherence drive a heuristic algorithm that attempts to interactively satisfy the sketched constraints as a weight-averaged projection of a minimal set of linear perspective cameras. This paper shows that 2D fea-ture constraints are a direct and effective approach to control both the 2D layout of scene objects and the conceptually complex, high dimensional parameter space of nonlinear scene projection. The simplicity of our interface also makes it an appealing alternative to standard through-the-lens and widget based techniques to control a single linear perspective camera

Design of Immersive Online Hotel Walkthrough System Using Image-Based (Concentric Mosaics) Rendering

Conventional hotel booking websites only represents their services in 2D photos to show their facilities. 2D photos are just static photos that cannot be move and rotate. Imagebased virtual walkthrough for the hospitality industry is a potential technology to attract more customers. In this project, a research will be carried out to create an Image-based rendering (IBR) virtual walkthrough and panoramic-based walkthrough by using only Macromedia Flash Professional 8, Photovista Panorama 3.0 and Reality Studio for the interaction of the images. The web-based of the image-based are using the Macromedia Dreamweaver Professional 8. The images will be displayed in Adobe Flash Player 8 or higher. In making image-based walkthrough, a concentric mosaic technique is used while image mosaicing technique is applied in panoramic-based walkthrough. A comparison of the both walkthrough is compared. The study is also focus on the comparison between number of pictures and smoothness of the walkthrough. There are advantages of using different techniques such as image-based walkthrough is a real time walkthrough since the user can walk around right, left, forward and backward whereas the panoramic-based cannot experience real time walkthrough because the user can only view 360 degrees from a fixed spot

Localisation and tracking of stationary users for extended reality

In this thesis, we investigate the topics of localisation and tracking in the context of Extended Reality. In many on-site or outdoor Augmented Reality (AR) applications, users are standing or sitting in one place and performing mostly rotational movements, i.e. stationary. This type of stationary motion also occurs in Virtual Reality (VR) applications such as panorama capture by moving a camera in a circle. Both applications require us to track the motion of a camera in potentially very large and open environments. State-of-the-art methods such as Structure-from-Motion (SfM), and Simultaneous Localisation and Mapping (SLAM), tend to rely on scene reconstruction from significant translational motion in order to compute camera positions. This can often lead to failure in application scenarios such as tracking for seated sport spectators, or stereo panorama capture where the translational movement is small compared to the scale of the environment. To begin with, we investigate the topic of localisation as it is key to providing global context for many stationary applications. To achieve this, we capture our own datasets in a variety of large open spaces including two sports stadia. We then develop and investigate these techniques in the context of these sports stadia using a variety of state-of-the-art localisation approaches. We cover geometry-based methods to handle dynamic aspects of a stadium environment, as well as appearance-based methods, and compare them to a state-of-the-art SfM system to identify the most applicable methods for server-based and on-device localisation. Recent work in SfM has shown that the type of stationary motion that we target can be reliably estimated by applying spherical constraints to the pose estimation. In this thesis, we extend these concepts into a real-time keyframe-based SLAM system for the purposes of AR, and develop a unique data structure for simplifying keyframe selection. We show that our constrained approach can track more robustly in these challenging stationary scenarios compared to state-of-the-art SLAM through both synthetic and real-data tests. In the application of capturing stereo panoramas for VR, this thesis demonstrates the unsuitability of standard SfM techniques for reconstructing these circular videos. We apply and extend recent research in spherically constrained SfM to creating stereo panoramas and compare this with state-of-the-art general SfM in a technical evaluation. With a user study, we show that the motion requirements of our SfM approach are similar to the natural motion of users, and that a constrained SfM approach is sufficient for providing stereoscopic effects when viewing the panoramas in VR

Creating 3D models of cultural heritage sites with terrestrial laser scanning and 3D imaging

Includes abstract.Includes bibliographical references.The advent of terrestrial laser-scanners made the digital preservation of cultural heritage sites an affordable technique to produce accurate and detailed 3D-computermodel representations for any kind of 3D-objects, such as buildings, infrastructure, and even entire landscapes. However, one of the key issues with this technique is the large amount of recorded points; a problem which was even more intensified by the recent advances in laser-scanning technology, which increased the data acquisition rate from 25 thousand to 1 million points per second. The following research presents a workflow for the processing of large-volume laser-scanning data, with a special focus on the needs of the Zamani initiative. The research project, based at the University of Cape Town, spatially documents African Cultural Heritage sites and Landscapes and produces meshed 3D models, of various, historically important objects, such as fortresses, mosques, churches, castles, palaces, rock art shelters, statues, stelae and even landscapes

Le cinéma omnistéréo ou l'art d'avoir des yeux tout le tour de la tête

Cette thèse s'intéresse à des aspects du tournage, de la projection et de la perception du cinéma stéréo panoramique, appelé aussi cinéma omnistéréo. Elle s'inscrit en grande partie dans le domaine de la vision par ordinateur, mais elle touche aussi aux domaines de l'infographie et de la perception visuelle humaine. Le cinéma omnistéréo projette sur des écrans immersifs des vidéos qui fournissent de l'information sur la profondeur de la scène tout autour des spectateurs. Ce type de cinéma comporte des défis liés notamment au tournage de vidéos omnistéréo de scènes dynamiques, à la projection polarisée sur écrans très réfléchissants rendant difficile l'estimation de leur forme par reconstruction active, aux distorsions introduites par l'omnistéréo pouvant fausser la perception des profondeurs de la scène. Notre thèse a tenté de relever ces défis en apportant trois contributions majeures. Premièrement, nous avons développé la toute première méthode de création de vidéos omnistéréo par assemblage d'images pour des mouvements stochastiques et localisés. Nous avons mis au point une expérience psychophysique qui montre l'efficacité de la méthode pour des scènes sans structure isolée, comme des courants d'eau. Nous proposons aussi une méthode de tournage qui ajoute à ces vidéos des mouvements moins contraints, comme ceux d'acteurs. Deuxièmement, nous avons introduit de nouveaux motifs lumineux qui permettent à une caméra et un projecteur de retrouver la forme d'objets susceptibles de produire des interréflexions. Ces motifs sont assez généraux pour reconstruire non seulement les écrans omnistéréo, mais aussi des objets très complexes qui comportent des discontinuités de profondeur du point de vue de la caméra. Troisièmement, nous avons montré que les distorsions omnistéréo sont négligeables pour un spectateur placé au centre d'un écran cylindrique, puisqu'elles se situent à la périphérie du champ visuel où l'acuité devient moins précise.This thesis deals with aspects of shooting, projection and perception of stereo panoramic cinema, also called omnistereo cinema. It falls largely in the field of computer vision, but it also in the areas of computer graphics and human visual perception. Omnistereo cinema uses immersive screens to project videos that provide depth information of a scene all around the spectators. Many challenges remain in omnistereo cinema, in particular shooting omnistereo videos for dynamic scenes, polarized projection on highly reflective screens making difficult the process to recover their shape by active reconstruction, and perception of depth distortions introduced by omnistereo images. Our thesis addressed these challenges by making three major contributions. First, we developed the first mosaicing method of omnistereo videos for stochastic and localized motions. We developed a psychophysical experiment that shows the effectiveness of the method for scenes without isolated structure, such as water flows. We also propose a shooting method that adds to these videos foreground motions that are not as constrained, like a moving actor. Second, we introduced new light patterns that allow a camera and a projector to recover the shape of objects likely to produce interreflections. These patterns are general enough to not only recover the shape of omnistereo screens, but also very complex objects that have depth discontinuities from the viewpoint of the camera. Third, we showed that omnistereo distortions are negligible for a viewer located at the center of a cylindrical screen, as they are in the periphery of the visual field where the human visual system becomes less accurate