Surface reconstruction of a blast plate using stereo vision

Includes bibliographical references.This thesis presents method for reconstructing and measuring the profile of a blast metal plate. Among the many methods in computer vision, stereo vision using two cameras is chosen as the range finding method in this thesis. This is because it is a non-contact method and hence eliminates the need to calibrate moving parts. A stereo-rig consists of two calibrated cameras and hence gives two view geometry. Stereoscopic reconstruction relies on epipolar geometry to constrain the relationship between the views. The 3-D point is then estimated using triangulation of the corresponding points from the two views. The blast plates that are reconstructed have highly reflective surfaces. This causes a problem due to specular reflection. This thesis further studies the reflective properties of the metal plate surface. Different methods of scanning the plate using the stereo-rig are investigated. The reconstructions obtained from these methods are analyzed for accuracy and consistency. Since low cost cameras are used in constructing the stereo-rig, the point cloud data obtained is further investigated for consistency by aligning different instances of the reconstruction. This is done using the Iterative Closest Programme (ICP) algorithm which tries to align two sets of data iteratively

Automatic detection of specular reflectance in colour images using the MS diagram

In this paper we present a new method for the identification of specular reflectance in colour images. We have developed a bi-dimensional histogram which allows the exploitation of the relations between the signals of intensity and saturation of a colour image. Once the diagram has been constructed, it is possible to verify that the pixels of the specular reflectance are located in a well-defined region. The brightness is automatically identified by means of the extraction of pixels present in this region of the diagram, independently of their hue values. The effectiveness of the method in a variety of real chromatic images has been proven

Coherent multi-dimensional segmentation of multiview images using a variational framework and applications to image based rendering

Image Based Rendering (IBR) and in particular light field rendering has attracted a lot of attention for interpolating new viewpoints from a set of multiview images. New images of a scene are interpolated directly from nearby available ones, thus enabling a photorealistic rendering. Sampling theory for light fields has shown that exact geometric information in the scene is often unnecessary for rendering new views. Indeed, the band of the function is approximately limited and new views can be rendered using classical interpolation methods. However, IBR using undersampled light fields suffers from aliasing effects and is difficult particularly when the scene has large depth variations and occlusions. In order to deal with these cases, we study two approaches: New sampling schemes have recently emerged that are able to perfectly reconstruct certain classes of parametric signals that are not bandlimited but characterized by a finite number of parameters. In this context, we derive novel sampling schemes for piecewise sinusoidal and polynomial signals. In particular, we show that a piecewise sinusoidal signal with arbitrarily high frequencies can be exactly recovered given certain conditions. These results are applied to parametric multiview data that are not bandlimited. We also focus on the problem of extracting regions (or layers) in multiview images that can be individually rendered free of aliasing. The problem is posed in a multidimensional variational framework using region competition. In extension to previous methods, layers are considered as multi-dimensional hypervolumes. Therefore the segmentation is done jointly over all the images and coherence is imposed throughout the data. However, instead of propagating active hypersurfaces, we derive a semi-parametric methodology that takes into account the constraints imposed by the camera setup and the occlusion ordering. The resulting framework is a global multi-dimensional region competition that is consistent in all the images and efficiently handles occlusions. We show the validity of the approach with captured light fields. Other special effects such as augmented reality and disocclusion of hidden objects are also demonstrated

Image Retrieval in the Plenoptic Space

In this report we study the ways to exploit the vast amount of information inherent in the plenoptic space and constraints of the plenoptic function to improve the efficiency of image retrieval, recognition and matching techniques. The plenoptic space is formed by extending the notion of traditional two-dimensional by adding more dimensions for viewing direction, time and wavelength. Using current hand-held devices' built-in cameras, one can easily capture a large sequence of pictures from a single static scene by moving the camera in one direction, which form a three dimensional plenoptic function

Image Based View Synthesis

This dissertation deals with the image-based approach to synthesize a virtual scene using sparse images or a video sequence without the use of 3D models. In our scenario, a real dynamic or static scene is captured by a set of un-calibrated images from different viewpoints. After automatically recovering the geometric transformations between these images, a series of photo-realistic virtual views can be rendered and a virtual environment covered by these several static cameras can be synthesized. This image-based approach has applications in object recognition, object transfer, video synthesis and video compression. In this dissertation, I have contributed to several sub-problems related to image based view synthesis. Before image-based view synthesis can be performed, images need to be segmented into individual objects. Assuming that a scene can approximately be described by multiple planar regions, I have developed a robust and novel approach to automatically extract a set of affine or projective transformations induced by these regions, correctly detect the occlusion pixels over multiple consecutive frames, and accurately segment the scene into several motion layers. First, a number of seed regions using correspondences in two frames are determined, and the seed regions are expanded and outliers are rejected employing the graph cuts method integrated with level set representation. Next, these initial regions are merged into several initial layers according to the motion similarity. Third, the occlusion order constraints on multiple frames are explored, which guarantee that the occlusion area increases with the temporal order in a short period and effectively maintains segmentation consistency over multiple consecutive frames. Then the correct layer segmentation is obtained by using a graph cuts algorithm, and the occlusions between the overlapping layers are explicitly determined. Several experimental results are demonstrated to show that our approach is effective and robust. Recovering the geometrical transformations among images of a scene is a prerequisite step for image-based view synthesis. I have developed a wide baseline matching algorithm to identify the correspondences between two un-calibrated images, and to further determine the geometric relationship between images, such as epipolar geometry or projective transformation. In our approach, a set of salient features, edge-corners, are detected to provide robust and consistent matching primitives. Then, based on the Singular Value Decomposition (SVD) of an affine matrix, we effectively quantize the search space into two independent subspaces for rotation angle and scaling factor, and then we use a two-stage affine matching algorithm to obtain robust matches between these two frames. The experimental results on a number of wide baseline images strongly demonstrate that our matching method outperforms the state-of-art algorithms even under the significant camera motion, illumination variation, occlusion, and self-similarity. Given the wide baseline matches among images I have developed a novel method for Dynamic view morphing. Dynamic view morphing deals with the scenes containing moving objects in presence of camera motion. The objects can be rigid or non-rigid, each of them can move in any orientation or direction. The proposed method can generate a series of continuous and physically accurate intermediate views from only two reference images without any knowledge about 3D. The procedure consists of three steps: segmentation, morphing and post-warping. Given a boundary connection constraint, the source and target scenes are segmented into several layers for morphing. Based on the decomposition of affine transformation between corresponding points, we uniquely determine a physically correct path for post-warping by the least distortion method. I have successfully generalized the dynamic scene synthesis problem from the simple scene with only rotation to the dynamic scene containing non-rigid objects. My method can handle dynamic rigid or non-rigid objects, including complicated objects such as humans. Finally, I have also developed a novel algorithm for tri-view morphing. This is an efficient image-based method to navigate a scene based on only three wide-baseline un-calibrated images without the explicit use of a 3D model. After automatically recovering corresponding points between each pair of images using our wide baseline matching method, an accurate trifocal plane is extracted from the trifocal tensor implied in these three images. Next, employing a trinocular-stereo algorithm and barycentric blending technique, we generate an arbitrary novel view to navigate the scene in a 2D space. Furthermore, after self-calibration of the cameras, a 3D model can also be correctly augmented into this virtual environment synthesized by the tri-view morphing algorithm. We have applied our view morphing framework to several interesting applications: 4D video synthesis, automatic target recognition, multi-view morphing

Image-Based Localization Using the Plenoptic Function

In this report we study the ways to exploit the vast amount of information inherent in the plenoptic space and constraints of the plenoptic function to improve the efficiency of image retrieval, recognition and matching techniques. The specific application we are concerned with is image-based location recognition on mobile devices. The plenoptic space is formed by extending the notion of traditional two-dimensional by adding more dimensions for viewing direction, time and wavelength. Using current mobile devices' built-in cameras, one can easily capture a large sequence of pictures from a single static scene by moving the camera in one direction, which form a three dimensional plenoptic function

Activity profiling for minimally invasive surgery

Imperial Users onl

Light Fields Reconstructing Geometry and Reflectance Properties

Computer vision plays an important role in the progress of automation and digitalization of our society. One of the key challenges is the creation of accurate 3D representations of our environment. The rich information in light fields can enable highly accurate depth estimates, but requires the development of new algorithms. Especially specular reflections pose a challenge for many reconstruction algorithms. This is due to the violation of the brightness consistency assumption, which only holds for Lambertian surfaces. Most surfaces are to some extent specular and an appropriate handling is central to avoid erroneous depth maps. In this thesis we explore the potential of using specular highlights to determine the orientation of surfaces. To this end, we examine epipolar images in light field set ups. In light field data, reflectance properties can be characterized by intensity variations in the epipolar plane space. This space is analysed and compared to the expected reflectance, which is modelled using the render equation with different bidirectional reflection distribution functions. This approach allows us to infer highly accurate surface normals and depth estimates. Furthermore, it reveals material properties encoded in the reflectance by inspecting the intensity profile. Our results demonstrate the potential to increase the accuracy of the depth maps. Multiple cameras in a light field set up let us retrieve additional material properties encoded in the reflectance