A Fast and Robust Extrinsic Calibration for RGB-D Camera Networks

From object tracking to 3D reconstruction, RGB-Depth (RGB-D) camera networks play an increasingly important role in many vision and graphics applications. Practical applications often use sparsely-placed cameras to maximize visibility, while using as few cameras as possible to minimize cost. In general, it is challenging to calibrate sparse camera networks due to the lack of shared scene features across different camera views. In this paper, we propose a novel algorithm that can accurately and rapidly calibrate the geometric relationships across an arbitrary number of RGB-D cameras on a network. Our work has a number of novel features. First, to cope with the wide separation between different cameras, we establish view correspondences by using a spherical calibration object. We show that this approach outperforms other techniques based on planar calibration objects. Second, instead of modeling camera extrinsic calibration using rigid transformation, which is optimal only for pinhole cameras, we systematically test different view transformation functions including rigid transformation, polynomial transformation and manifold regression to determine the most robust mapping that generalizes well to unseen data. Third, we reformulate the celebrated bundle adjustment procedure to minimize the global 3D reprojection error so as to fine-tune the initial estimates. Finally, our scalable client-server architecture is computationally efficient: the calibration of a five-camera system, including data capture, can be done in minutes using only commodity PCs. Our proposed framework is compared with other state-of-the-arts systems using both quantitative measurements and visual alignment results of the merged point clouds

The use of low cost virtual reality and digital technology to aid forensic scene interpretation and recording

© Cranfield University 2005. All rights reserved. No part of this publication may be reproduced without the written permission of the copyright owner.Crime scenes are often short lived and the opportunities must not be lost in acquiring sufficient information before the scene is disturbed. With the growth in information technology (IT) in many other scientific fields, there are also substantial opportunities for IT in the area of forensic science. The thesis sought to explore means by which IT can assist and benefit the ways that forensic information can be illustrated and elucidated in a logical manner. The central research hypothesis considers that through the utilisation of low cost IT, the visual presentation of information will be of significant benefit to forensic science in particular for the recoding of crime scenes and its presentation in court. The research hypothesis was addressed by first exploring the current crime scene documentation techniques; their strengths and weaknesses, giving indication to the possible niche that technology could occupy within forensic science. The underlying principles of panoramic technology were examined, highlighting its ability to express spatial information efficiently. Through literature review and case studies, the current status of the technology within the forensic community and courtrooms was also explored to gauge its possible acceptance as a forensic tool. This led to the construction of a low cost semi-automated imaging system capable of capturing the necessary images for the formation of a panorama. This provides the ability to pan around; effectively placing the viewer at the crime scene. Evaluation and analysis involving forensic personnel was performed to assess the capabilities and effectiveness of the imaging system as a forensic tool. The imaging system was found to enhance the repertoire of techniques available for crime scene documentation; possessing sufficient capabilities and benefits to warrant its use within the area of forensics, thereby supporting the central hypothesis

VR-LAB: A Distributed Multi-User Environment for Educational Purposes and Presentations

In the last three years our research was focused on a new distributed multi-user environment. Finally, all components were integrated in a system called the VR-Lab, which will be described on the following pages. The VR-Lab provides Hard- and Software for a distributed presentation system. Elements which are often used in environments called Computer Supported Cooperative Work (CSCW). In contrast to other projects the VR-Lab integrates a distributed system in a common environment of a lecture room and does not generate a virtual conference room in a computer system. Thus, allowing inexperienced persons to use the VR-LAB and benefit from the multimedia tools in their common environment. To build the VR-LAB we developed a lot of hard- and software and integrated it into a lecture room to perform distributed presentations, conferences or teaching. Additionally other software components were developed to be connected to the VR-LAB, control its components, or distribute content between VR-LAB installations. Beside standard software for video and audio transmission, we developed and integrated a distributed 3D-VRML-Browser to present three dimensional content to a distributed audience. One of the interesting features of this browser is the object oriented distributed scene graph. By coupling a high-speed rendering system with a database we could distribute objects to other participants. So the semantic properties of any geometrical or control object can be kept and used by the remote participant. Because of the high compression achieved by the transport of objects instead of triangles a lot of bandwidth could be saved. Also each participant could select a display quality appropriate to its hardware.Diese Arbeit beschreibt ein integriertes Virtual-Reality System, das VR Lab. Das System besteht aus verschiedenen Hard- und Softwarekomponenten die eine verteiltevirtuelle Multi-User Umgebung darstellen die vor allem im Bereich verteilter Präsentationen verwendet werden kann. Im Gegensatz zu anderen Systemen dieser Art, die oft im Bereich des Computer Supported Cooperative Work (CSCW) eingesetzt werden dient unser System nicht dazu eine Präsentationsumgebung im Computer nachzubilden sondern eine reele Umgebung zu schaffen in der verteilte Präsentationen durchgeführt werden können. Dies soll vor allem ungeübten Personen die Arbeit mit verteilten Umgebungen erleichtern. Dazu wurden verschiedene Hard- und Softwarekomponenten entwickelt. Darunter der verteilte 3D Browser MRT-VR, der es ermöglicht 3D Daten an verschiedenen Stellen gleichzeitig zu visualisieren. MRT-VR zeichnet sich insbesondere dadurch aus, daß die 3D Objekte nicht als Polygondaten transportiert werden, sonderen als Objekte und so deren Objekteigenschaften beibehalten werden. Dies spart nicht nur sehr viel Bandbreite bei der Übertragung sondern ermöglicht auch Darstellungen in unterschiedlichen Qualitätsstufen auf den unterschiedlichen Zielrechnern der Teilnehmer. Ein weiterer Teil der Arbeit beschreibt die Entwicklung einer preiswerten imersiven 3D Umgebung um die 3D Daten in ansprechender Qualität zu visualisieren. Alle Komponenten wurden in einer gemeinsamen Umgebung, dem VR-Lab, integriert und mt Steuerungskomponenten versehen

Personal imaging

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts & Sciences, 1997.Includes bibliographical references (p. 217-223).In this thesis, I propose a new synergy between humans and computers, called "Humanistic Intelligence" (HI), and provide a precise definition of this new form of human-computer interaction. I then present a means and apparatus for reducing this principle to practice. The bulk of this thesis concentrates on a specific embodiment of this invention, called Personal Imaging, most notably, a system which I show attains new levels of creativity in photography, defines a new genre of documentary video, and goes beyond digital photography/video to define a new renaissance in imaging, based on simple principles of projective geometry combined with linearity and superposition properties of light. I first present a mathematical theory of imaging which allows the apparatus to measure, to within a single unknown constant, the quantity of light arriving from each direction, to a fixed point in space, using a collection of images taken from a sensor array having a possibly unknown nonlinearity. Within the context of personal imaging, this theory is a contribution in and of itself (in the sense that it was an unsolved problem previously), but when also combined with the proposed apparatus, it allows one to construct environment maps by simply looking around. I then present a new form of connected humanistic intelligence in which individuals can communicate, across boundaries of time and space, using shared environment maps, and the resulting computer-mediated reality that arises out of long-term adaptation in a personal imaging environment. Finally, I present a new philosophical framework for cultural criticism which arises out of a new concept called 'humanistic property'. This new philosophical framework has two axes, a 'reflectionist' axis and a 'diffusionist' axis. In particular, I apply the new framework to personal imaging, thus completing a body of work that lies at the intersection of art, science, and technology.by Steve Mann.Ph.D

The use of low cost virtual reality and digital technology to aid forensic scene interpretation and recording

Crime scenes are often short lived and the opportunities must not be lost in acquiring sufficient information before the scene is disturbed. With the growth in information technology (IT) in many other scientific fields, there are also substantial opportunities for IT in the area of forensic science. The thesis sought to explore means by which IT can assist and benefit the ways that forensic information can be illustrated and elucidated in a logical manner. The central research hypothesis considers that through the utilisation of low cost IT, the visual presentation of information will be of significant benefit to forensic science in particular for the recoding of crime scenes and its presentation in court. The research hypothesis was addressed by first exploring the current crime scene documentation techniques; their strengths and weaknesses, giving indication to the possible niche that technology could occupy within forensic science. The underlying principles of panoramic technology were examined, highlighting its ability to express spatial information efficiently. Through literature review and case studies, the current status of the technology within the forensic community and courtrooms was also explored to gauge its possible acceptance as a forensic tool. This led to the construction of a low cost semi-automated imaging system capable of capturing the necessary images for the formation of a panorama. This provides the ability to pan around; effectively placing the viewer at the crime scene. Evaluation and analysis involving forensic personnel was performed to assess the capabilities and effectiveness of the imaging system as a forensic tool. The imaging system was found to enhance the repertoire of techniques available for crime scene documentation; possessing sufficient capabilities and benefits to warrant its use within the area of forensics, thereby supporting the central hypothesis.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Fusing spatial and temporal components for real-time depth data enhancement of dynamic scenes

The depth images from consumer depth cameras (e.g., structured-light/ToF devices) exhibit a substantial amount of artifacts (e.g., holes, flickering, ghosting) that needs to be removed for real-world applications. Existing methods cannot entirely remove them and perform slow. This thesis proposes a new real-time spatio-temporal depth image enhancement filter that completely removes flickering and ghosting, and significantly reduces holes. This thesis also presents a novel depth-data capture setup and two data reduction methods to optimize the performance of the proposed enhancement method

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance