Surface camera (SCAM) Light Field Rendering

In this article we present a new variant of the light field representation that supports improved image reconstruction by accommodating sparse correspondence information. This places our representation somewhere between a pure, two-plane parameterized, light field and a lumigraph representation, with its continuous geometric proxy. Our approach factors the rays of a light field into one of two separate classes. All rays consistent with a given correspondence are implicitly represented using a new auxiliary data structure, which we call a surface camera, or scam. The remaining rays of the light field are represented using a standard two-plane parameterized light field. We present an efficient rendering algorithm that combines ray samples from scams with those from the light field. The resulting image reconstructions are noticeably improved over that of a pure light field.Engineering and Applied Science

Estimation of signal distortion using effective sampling density for light field-based free viewpoint video

In a light field-based free viewpoint video (LF-based FVV) system, effective sampling density (ESD) is defined as the number of rays per unit area of the scene that has been acquired and is selected in the rendering process for reconstructing an unknown ray. This paper extends the concept of ESD and shows that ESD is a tractable metric that quantifies the joint impact of the imperfections of LF acquisition and rendering. By deriving and analyzing ESD for the commonly used LF acquisition and rendering methods, it is shown that ESD is an effective indicator determined by system parameters and can be used to directly estimate output video distortion without access to the ground truth. This claim is verified by extensive numerical simulations and comparison to PSNR. Furthermore, an empirical relationship between the output distortion (in PSNR) and the calculated ESD is established to allow direct assessment of the overall video distortion without an actual implementation of the system. A small scale subjective user study is also conducted which indicates a correlation of 0.91 between ESD and perceived quality

sCAM: An Untethered Insertable Laparoscopic Surgical Camera Robot

Fully insertable robotic imaging devices represent a promising future of minimally invasive laparoscopic vision. Emerging research efforts in this field have resulted in several proof-of-concept prototypes. One common drawback of these designs derives from their clumsy tethering wires which not only cause operational interference but also reduce camera mobility. Meanwhile, these insertable laparoscopic cameras are manipulated without any pose information or haptic feedback, which results in open loop motion control and raises concerns about surgical safety caused by inappropriate use of force.This dissertation proposes, implements, and validates an untethered insertable laparoscopic surgical camera (sCAM) robot. Contributions presented in this work include: (1) feasibility of an untethered fully insertable laparoscopic surgical camera, (2) camera-tissue interaction characterization and force sensing, (3) pose estimation, visualization, and feedback with sCAM, and (4) robotic-assisted closed-loop laparoscopic camera control. Borrowing the principle of spherical motors, camera anchoring and actuation are achieved through transabdominal magnetic coupling in a stator-rotor manner. To avoid the tethering wires, laparoscopic vision and control communication are realized with dedicated wireless links based on onboard power. A non-invasive indirect approach is proposed to provide real-time camera-tissue interaction force measurement, which, assisted by camera-tissue interaction modeling, predicts stress distribution over the tissue surface. Meanwhile, the camera pose is remotely estimated and visualized using complementary filtering based on onboard motion sensing. Facilitated by the force measurement and pose estimation, robotic-assisted closed-loop control has been realized in a double-loop control scheme with shared autonomy between surgeons and the robotic controller.The sCAM has brought robotic laparoscopic imaging one step further toward less invasiveness and more dexterity. Initial ex vivo test results have verified functions of the implemented sCAM design and the proposed force measurement and pose estimation approaches, demonstrating the technical feasibility of a tetherless insertable laparoscopic camera. Robotic-assisted control has shown its potential to free surgeons from low-level intricate camera manipulation workload and improve precision and intuitiveness in laparoscopic imaging

Innovative 3D Depth Map Generation From A Holoscopic 3D Image Based on Graph Cut Technique

Holoscopic 3D imaging is a promising technique for capturing full-colour spatial 3D images using a single aperture holoscopic 3D camera. It mimics fly’s eye technique with a microlens array, which views the scene at a slightly different angle to its adjacent lens that records three-dimensional information onto a two-dimensional surface. This paper proposes a method of depth map generation from a holoscopic 3D image based on graph cut technique. The principal objective of this study is to estimate the depth information presented in a holoscopic 3D image with high precision. As such, depth map extraction is measured from a single still holoscopic 3D image which consists of multiple viewpoint images. The viewpoints are extracted and utilised for disparity calculation via disparity space image technique and pixels displacement is measured with sub-pixel accuracy to overcome the issue of the narrow baseline between the viewpoint images for stereo matching. In addition, cost aggregation is used to correlate the matching costs within a particular neighbouring region using sum of absolute difference (SAD) combined with gradient-based metric and “winner takes all” algorithm is employed to select the minimum elements in the array as optimal disparity value. Finally, the optimal depth map is obtained using graph cut technique. The proposed method extends the utilisation of holoscopic 3D imaging system and enables the expansion of the technology for various applications of autonomous robotics, medical, inspection, AR/VR, security and entertainment where 3D depth sensing and measurement are a concern

Survey of image-based representations and compression techniques

In this paper, we survey the techniques for image-based rendering (IBR) and for compressing image-based representations. Unlike traditional three-dimensional (3-D) computer graphics, in which 3-D geometry of the scene is known, IBR techniques render novel views directly from input images. IBR techniques can be classified into three categories according to how much geometric information is used: rendering without geometry, rendering with implicit geometry (i.e., correspondence), and rendering with explicit geometry (either with approximate or accurate geometry). We discuss the characteristics of these categories and their representative techniques. IBR techniques demonstrate a surprising diverse range in their extent of use of images and geometry in representing 3-D scenes. We explore the issues in trading off the use of images and geometry by revisiting plenoptic-sampling analysis and the notions of view dependency and geometric proxies. Finally, we highlight compression techniques specifically designed for image-based representations. Such compression techniques are important in making IBR techniques practical.published_or_final_versio

Editing architecture: architect as mediumistic being

The creation of architecture is based on the relationship the designer has with media. Making media requires both a technical proficiency and a capacity to understand how the medium itself informs the architect and the creative work. I will explore through this thesis how the creation of media is not only a metaphor for the process of architecture, but the act of architecture itself. In addition to the making of media, this work will analyze editing media as a provocative interface of design. Finally, this work will focus on digital media-specifically digital video and the opportunities it may have to inform architecture and the education of architects. Based on the assumption that architecture itself is in fact media, the building itself, is merely a by-product of the process undergone through the manipulation of media. Given this perspective, the product can become different based on the media used in its creation. Architectural media can be created in many formats and for the sake of focus this work will concentrate on video, the manipulation of video, and the conceptual link between video editing and the making of architecture. Of the various forms of media that exist in contemporary culture, video and the manipulation of video stand out as an untapped resource for architecture. Architecture has an opportunity to benefit from moving images-how these images inform spatial perception, how the series of images may represent time and how relationships between spaces and ideas can be articulated through this medium. While the visual benefits of video may seem obvious, the way video is constructed offers another way to access an understanding of idea generation

Creating 3D models of cultural heritage sites with terrestrial laser scanning and 3D imaging

Includes abstract.Includes bibliographical references.The advent of terrestrial laser-scanners made the digital preservation of cultural heritage sites an affordable technique to produce accurate and detailed 3D-computermodel representations for any kind of 3D-objects, such as buildings, infrastructure, and even entire landscapes. However, one of the key issues with this technique is the large amount of recorded points; a problem which was even more intensified by the recent advances in laser-scanning technology, which increased the data acquisition rate from 25 thousand to 1 million points per second. The following research presents a workflow for the processing of large-volume laser-scanning data, with a special focus on the needs of the Zamani initiative. The research project, based at the University of Cape Town, spatially documents African Cultural Heritage sites and Landscapes and produces meshed 3D models, of various, historically important objects, such as fortresses, mosques, churches, castles, palaces, rock art shelters, statues, stelae and even landscapes

Development of Correspondence Field and Its Application to Effective Depth Estimation in Stereo Camera Systems

Stereo camera systems are still the most widely used apparatus for estimating 3D or depth information of a scene due to their low-cost. Estimation of depth using a stereo camera requires first estimating the disparity map using stereo matching algorithms and calculating depth via triangulation based on the camera arrangement (their locations and orientations with respect to the scene). In almost all cases, the arrangement is determined based on human experience since there lacks an effective theoretical tool to guide the design of the camera arrangement. This thesis presents the development of a novel tool, called correspondence field (CF), and its application to optimize the stereo camera arrangement for depth estimation

Investigations of collaborative design environments: A framework for real-time collaborative 3D CAD

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This research investigates computer-based collaborative design environments, in particular issues of real-time collaborative 3D CAD. The thesis first presents a broad perspective of collaborative design environments with a preliminary case study of team design activities in a conventional and a computer mediated setting. This study identifies the impact and the feasibility of computer support for collaborative design and suggests four kinds of essential technologies for a successful collaborative design environment: information-sharing systems, synchronous and asynchronous co- working tools, project management systems, and communication systems. A new conceptual framework for a real-time collaborative 3D design tool, Shared Stage, is proposed based upon the preliminary study. The Shared Stage is defined as a shared 3D design workspace aiming to smoothly incorporate shared 3D workspaces into existing individual 3D workspaces. The addition of a Shared Stage allows collaborating designers to interact in real-time and to have a dynamic and interactive exchange of intermediate 3D design data. The acceptability of collaborative features is maximised by maintaining consistency of the user interface between 3D CAD systems. The framework is subsequently implemented as a software prototype using a new software development environment, customised by integrating related real-time and 3D graphic software development tools. Two main components of the Shared Stage module in the prototype, the Synchronised Stage View (SSV) and the Data Structure Diagram (DSD), provide essential collaborative features for real-time collaborative 3D CAD. These features include synchronised shared 3D representation, dynamic data exchange and awareness support in 3D workspaces. The software prototype is subsequently evaluated to examine the usefulness and usability. A range of quantitative and qualitative methods is used to evaluate the impact of the Shared Stage. The results, including the analysis of collaborative interactions and user perception, illustrate that the Shared Stage is a feasible and valuable addition for real-time collaborative 3D CAD. This research identifies the issues to be addressed for collaborative design environments and also provides a new framework and development strategy of a novel real-time collaborative 3D CAD system. The framework is successfully demonstrated through prototype implementation and an analytical usability evaluation.Financial support from the Department and from the UK government through the Overseas Research Studentship Awards