An Outdoor Stereo Camera System for the Generation of Real-World Benchmark Datasets with Ground Truth

In this report we describe a high-performance stereo camera system to capture image sequences with high temporal and spatial resolution for the evaluation of various image processing tasks. The system was primarily designed for complex outdoor and traffic scenes which frequently occur in the automotive industry, but is also suited for other applications. For this task the system is equipped with a very accurate inertial measurement unit and global positioning system, which provides exact camera movement and position data. The system is already in active use and has produced several terabyte of challenging image sequences which are available for download

Surface Reflectance Estimation and Natural Illumination Statistics

Humans recognize optical reflectance properties of surfaces such as metal, plastic, or paper from a single image without knowledge of illumination. We develop a machine vision system to perform similar recognition tasks automatically. Reflectance estimation under unknown, arbitrary illumination proves highly underconstrained due to the variety of potential illumination distributions and surface reflectance properties. We have found that the spatial structure of real-world illumination possesses some of the statistical regularities observed in the natural image statistics literature. A human or computer vision system may be able to exploit this prior information to determine the most likely surface reflectance given an observed image. We develop an algorithm for reflectance classification under unknown real-world illumination, which learns relationships between surface reflectance and certain features (statistics) computed from a single observed image. We also develop an automatic feature selection method

Real time city visualization

The visualization of cities in real time has a lot of potential applications, from urban and emergency planning, to driving simulators and entertainment. The massive amount of data and the computational requirements needed to render an entire city in detail are the reason why a lot of techniques have been proposed in this eld. Procedural city generation, building simpli cation and visibility processing are some of the approaches used to solve a small subset of the problems that these applications need to face. Our work proposes a new city rendering algorithm that is a radically di erent approach to what has been done before in this eld. The proposed technique is based on a structuration of the city data in a regular grid which is traversed, at runtime, by a ray tracing algorithm that keeps track of visible parts of the scene. As a preprocess, a set of quads de ning the buildings of a city is transformed to the regular grid used by our algorithm. The rendering algorithm uses this data to generate a real time representation of the city minimizing the overdraw, a common problem in other techniques. This is done by means of a geometry shader to generate only the minimum number of fragments needed to render the city from a given position

３Ｄ計測による自然景観及び都市造形物の点群データ処理とその応用

岩手県立大

Sensor planning for novel view generation by camera networks

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2006.Includes bibliographical references (p. 75-78).This document describes a system for generating novel views of an indoor visual scene by gathering successive 2D images from a set of independent networked robotic cameras. Specifically, the robotic cameras work to seek out texture and geometric information needed to generate the specified synthetic view or views, aiming to-with each successive camera move-increase confidence in the estimates of the pixel intensities in the novel view(s). This system lays the groundwork for future explorations in multi-camera video recording for electroholography and image-based rendering.by James Barabas.S.M

Analyse de visibilité et géolocalisation en milieu urbain avec un modèle numérique 3D

L'analyse de visibilité est une question importante de la recherche qui peut trouver des applications dans de nombreux domaines: sécurité, conception de réseau sans fil, gestion du paysage, mise en oeuvre d'accès piétonniers .... La prise en compte de la troisième dimension dans le calcul de visibilité est un réel défi. Seules quelques solutions peuvent détecter les obstacles 3D qui limitent l'isovist. Dans cette communication, nous présentons un nouvel algorithme qui peut détecter tous les objets qui bloquent la vue dans un environnement 30 reconstitué numériquement intégrant le relief. Une démonstration avec des données SIG est également effectuée.La reconnaissance automatique des bâtiments est une étape essentielle pour la réalité augmentée et un outil possible pour la géolocalisation d'une prise de vue. Les recherches dans ce domaine n'utilisent pas la localisation par contenu de l'image. Cet article présente une méthodologie pour l'enrichissement d'une base de données urbaine SIG grâce à un descripteur de texture de façade calculé sur des images de référence. Cet indicateur est ensuite utilisé pour retrouver ce bâtiment dans une nouvelle image et le localiser dans une base de données SIG 3D afin d'estimer sa position et son orientation dans le repère de l'appareil photographique qui a pris le cliché. La qualité des résultats obtenus fait l'objet d'une discussion.The isovist or vision field is an interesting topic with many applications in different fields: security, wireless networkdesign, landscape management and analysis, pedestrian access .... Taking in account 3D environment is a verychallenging task. Only a few solutions can detect 3D obstacles that limit the vision field. We present in this paper a newalgorithm that can detect all the objects which block the sight in a 3D environment including the ground surface. A demonstration with GIS data is also given.Building recognition is the first step for augmented reality and the geolocation of the camera. Research in this field usually does not use the content of the image to locate it. This paper presents a methodology for enhancing and complementing a GIS database of buildings with a texture descriptor of the facades by using information extracted from reference images. This descriptor is used to locate any other image by searching similar texture in the image and locate it through the 3D GIS data base. The absolute position and orientation of the camera of the new image can then be computed if camera parameters (i.e. focal length) are known. The quality of the results is presented and discussed.ST ETIENNE-Bib. électronique (422189901) / SudocSudocFranceF

Detail Enhancing Denoising of Digitized 3D Models from a Mobile Scanning System

The acquisition process of digitizing a large-scale environment produces an enormous amount of raw geometry data. This data is corrupted by system noise, which leads to 3D surfaces that are not smooth and details that are distorted. Any scanning system has noise associate with the scanning hardware, both digital quantization errors and measurement inaccuracies, but a mobile scanning system has additional system noise introduced by the pose estimation of the hardware during data acquisition. The combined system noise generates data that is not handled well by existing noise reduction and smoothing techniques. This research is focused on enhancing the 3D models acquired by mobile scanning systems used to digitize large-scale environments. These digitization systems combine a variety of sensors – including laser range scanners, video cameras, and pose estimation hardware – on a mobile platform for the quick acquisition of 3D models of real world environments. The data acquired by such systems are extremely noisy, often with significant details being on the same order of magnitude as the system noise. By utilizing a unique 3D signal analysis tool, a denoising algorithm was developed that identifies regions of detail and enhances their geometry, while removing the effects of noise on the overall model. The developed algorithm can be useful for a variety of digitized 3D models, not just those involving mobile scanning systems. The challenges faced in this study were the automatic processing needs of the enhancement algorithm, and the need to fill a hole in the area of 3D model analysis in order to reduce the effect of system noise on the 3D models. In this context, our main contributions are the automation and integration of a data enhancement method not well known to the computer vision community, and the development of a novel 3D signal decomposition and analysis tool. The new technologies featured in this document are intuitive extensions of existing methods to new dimensionality and applications. The totality of the research has been applied towards detail enhancing denoising of scanned data from a mobile range scanning system, and results from both synthetic and real models are presented

Methods for Real-time Visualization and Interaction with Landforms

This thesis presents methods to enrich data modeling and analysis in the geoscience domain with a particular focus on geomorphological applications. First, a short overview of the relevant characteristics of the used remote sensing data and basics of its processing and visualization are provided. Then, two new methods for the visualization of vector-based maps on digital elevation models (DEMs) are presented. The first method uses a texture-based approach that generates a texture from the input maps at runtime taking into account the current viewpoint. In contrast to that, the second method utilizes the stencil buffer to create a mask in image space that is then used to render the map on top of the DEM. A particular challenge in this context is posed by the view-dependent level-of-detail representation of the terrain geometry. After suitable visualization methods for vector-based maps have been investigated, two landform mapping tools for the interactive generation of such maps are presented. The user can carry out the mapping directly on the textured digital elevation model and thus benefit from the 3D visualization of the relief. Additionally, semi-automatic image segmentation techniques are applied in order to reduce the amount of user interaction required and thus make the mapping process more efficient and convenient. The challenge in the adaption of the methods lies in the transfer of the algorithms to the quadtree representation of the data and in the application of out-of-core and hierarchical methods to ensure interactive performance. Although high-resolution remote sensing data are often available today, their effective resolution at steep slopes is rather low due to the oblique acquisition angle. For this reason, remote sensing data are suitable to only a limited extent for visualization as well as landform mapping purposes. To provide an easy way to supply additional imagery, an algorithm for registering uncalibrated photos to a textured digital elevation model is presented. A particular challenge in registering the images is posed by large variations in the photos concerning resolution, lighting conditions, seasonal changes, etc. The registered photos can be used to increase the visual quality of the textured DEM, in particular at steep slopes. To this end, a method is presented that combines several georegistered photos to textures for the DEM. The difficulty in this compositing process is to create a consistent appearance and avoid visible seams between the photos. In addition to that, the photos also provide valuable means to improve landform mapping. To this end, an extension of the landform mapping methods is presented that allows the utilization of the registered photos during mapping. This way, a detailed and exact mapping becomes feasible even at steep slopes