The perceptual consequences and neural basis of monocular occlusions

Occluded areas are abundant in natural scenes and play an important role in stereopsis. However, due to the treatment of occlusions as noise by early researchers of stereopsis, this field of study has not seen much development until the last two decades. Consequently, many aspects of depth perception from occlusions are not well understood. The goal of this thesis was to study several such aspects in order to advance the current understanding of monocular occlusions and their neural underpinnings. The psychophysical and computational studies described in this thesis have demonstrated that: 1) occlusions play an important role in defining the shape and depth of occluding surfaces, 2) depth signals from monocular occlusions and disparity interact in complex ways, 3) there is a single mechanism underlying depth perception from monocular occlusions and 4) this mechanism is likely to rely on monocular occlusion geometry. A unified theory of depth computation from monocular occlusions and disparity was proposed based on these findings. A biologically-plausible computational model based on this theory produced results close to observer percepts for a variety of monocular occlusion phenomena

Dense Vision in Image-guided Surgery

Image-guided surgery needs an efficient and effective camera tracking system in order to perform augmented reality for overlaying preoperative models or label cancerous tissues on the 2D video images of the surgical scene. Tracking in endoscopic/laparoscopic scenes however is an extremely difficult task primarily due to tissue deformation, instrument invasion into the surgical scene and the presence of specular highlights. State of the art feature-based SLAM systems such as PTAM fail in tracking such scenes since the number of good features to track is very limited. When the scene is smoky and when there are instrument motions, it will cause feature-based tracking to fail immediately. The work of this thesis provides a systematic approach to this problem using dense vision. We initially attempted to register a 3D preoperative model with multiple 2D endoscopic/laparoscopic images using a dense method but this approach did not perform well. We subsequently proposed stereo reconstruction to directly obtain the 3D structure of the scene. By using the dense reconstructed model together with robust estimation, we demonstrate that dense stereo tracking can be incredibly robust even within extremely challenging endoscopic/laparoscopic scenes. Several validation experiments have been conducted in this thesis. The proposed stereo reconstruction algorithm has turned out to be the state of the art method for several publicly available ground truth datasets. Furthermore, the proposed robust dense stereo tracking algorithm has been proved highly accurate in synthetic environment (< 0.1 mm RMSE) and qualitatively extremely robust when being applied to real scenes in RALP prostatectomy surgery. This is an important step toward achieving accurate image-guided laparoscopic surgery.Open Acces

A real-time low-cost vision sensor for robotic bin picking

This thesis presents an integrated approach of a vision sensor for bin picking. The vision system that has been devised consists of three major components. The first addresses the implementation of a bifocal range sensor which estimates the depth by measuring the relative blurring between two images captured with different focal settings. A key element in the success of this approach is that it overcomes some of the limitations that were associated with other related implementations and the experimental results indicate that the precision offered by the sensor discussed in this thesis is precise enough for a large variety of industrial applications. The second component deals with the implementation of an edge-based segmentation technique which is applied in order to detect the boundaries of the objects that define the scene. An important issue related to this segmentation technique consists of minimising the errors in the edge detected output, an operation that is carried out by analysing the information associated with the singular edge points. The last component addresses the object recognition and pose estimation using the information resulting from the application of the segmentation algorithm. The recognition stage consists of matching the primitives derived from the scene regions, while the pose estimation is addressed using an appearance-based approach augmented with a range data analysis. The developed system is suitable for real-time operation and in order to demonstrate the validity of the proposed approach it has been examined under varying real-world scenes

Value of remote sensing in forest surveys

Available from British Library Document Supply Centre- DSC:D42883/82 / BLDSC - British Library Document Supply CentreSIGLEGBUnited Kingdo

Matching of repeat remote sensing images for precise analysis of mass movements

Photogrammetry, together with radar interferometry, is the most popular of the remote sensing techniques used to monitor stability of high mountain slopes. By using two images of an area taken from different view angles, photogrammetry produces digital terrain models (DTM) and orthoprojected images. Repeat digital terrain models (DTM) are differenced to compute elevation changes. Repeat orthoimages are matched to compute the horizontal displacement and deformation of the masses. The success of the photogrammetric approach in the computation of horizontal displacement (and also the generation of DTM through parallax matching, although not covered in this work) greatly relies on the success of image matching techniques. The area-based image matching technique with the normalized cross-correlation (NCC) as its similarity measure is widely used in mass movement analysis. This method has some limitations that reduce its precision and reliability compared to its theoretical potential. The precision with which the matching position is located is limited to the pixel size unless some sub-pixel precision procedures are applied. The NCC is only reliable in cases where there is no significant deformation except shift in position. Identification of a matching entity that contains optimum signal-to-noise ratio (SNR) and minimum geometric distortion at each location has always been challenging. Deformation parameters such as strains can only be computed from the inter-template displacement gradient in a post-matching process. To find appropriate solutions for the mentioned limitations, the following investigations were made on three different types of mass movements; namely, glacier flow, rockglacier creep and land sliding. The effects of ground pixel size on the accuracy of the computed mass movement parameters such as displacement were investigated. Different sub-pixel precision algorithms were implemented and evaluated to identify the most precise and reliable algorithm. In one approach images are interpolated to higher spatial resolution prior to matching. In another approach the NCC correlation surface is interpolated to higher resolution so that the location of the correlation peak is more precise. In yet another approach the position of the NCC peak is computed by fitting 2D Gaussian and parabolic curves to the correlation peak turn by turn. The results show that the mean error in metric unit increases linearly with the ground pixel size being about half a pixel at each resolution. The proportion of undetected moving masse increases with ground pixel size depending on the displacement magnitudes. Proportion of mismatching templates increases with increasing ground pixel size depending on the noise content, i.e. temporal difference, of the image pairs. Of the sub-pixel precision algorithms, interpolating the image to higher resolution using bi-cubic convolution prior to matching performs best. For example, by increasing the spatial resolution (i.e. reducing the ground pixel size) of the matched images by 2 to 16 times using intensity interpolation, 40% to 80% of the performances of the same resolution original image can be achieved. A new spatially adaptive algorithm that defines the template sizes by optimizing the SNR, minimizing the geometric distortion and optimizing the similarity measure was also devised, implemented and evaluated on aerial and satellite images of mass movements. The algorithm can also exclude ambiguous and occluded entities from the matching. The evaluation of the algorithm was conducted on simulated deformation images and in relation to the image-wide fixed template sizes ranging from 11 to 101 pixels. The evaluation of the algorithm on the real mass movements is conducted by a novel technique of reconstructing the reference image from the deformed image and computing the global correlation coefficient and the corresponding SNR between the reference and the reconstructed image. The results show that the algorithm could reduce the error of displacement estimation by up to over 90% (in the simulated case) and improve the SNR of the matching by up to over 4 times compared to the globally fixed template sizes. The algorithm pushes terrain displacement measurement from repeat images one step forward towards full automation. The least squares image matching (LSM) matches images precisely by modeling both the geometric and radiometric deformation. The potential of the LSM is not fully utilized for mass movement analysis. Here, the procedures with which horizontal surface displacement, rotation and strain rates of glacier flow, rockglacier creep and land sliding are computed from the spatial transformation parameters of LSM automatically during the matching are implemented and evaluated. The results show that the approach computes longitudinal strain rates, transverse strain rates and shear strain rates reliably with mean absolute deviation in the order of 10-4 as evaluated on stable grounds. The LSM also improves the accuracy of displacement estimation of the NCC by about 90% in ideal (simulated) case and the SNR of the matching by about 25% in real multi-temporal images of mass movements. Additionally, advanced spatial transformation models such as projective and second degree polynomial are used for the first time for mass movement analysis in addition to the affine. They are also adapted spatially based on the minimization of the sum of square deviation between the matching templates. The spatially adaptive approach produces the best matching, closely followed by the second-order polynomial. Affine and projective models show similar results closely following the two approaches. In the case of the spatially adaptive approach, over 60% of the entities matched for the rockglacier and the landslide are best fit by the second-order polynomial model. In general, the NCC alone may be sufficient for low resolution images of moving masses with limited or no deformation. To gain better precision and reliability in such cases, the template sizes can be adapted spatially and the images can be interpolated to higher resolution (preferably not more detail than 1/16th of a pixel) prior to the matching. For highly deformed masses where higher resolution images are used, the LSM is recommended as it results in more accurate matching and deformation parameters. Improved accuracy and precision are obtained by selecting matchable areas using the spatially adaptive algorithm, identifying approximate matches using the NCC and optimizing the matches and measuring the deformation parameters using the LSM algorithm

Holoscopic 3D image depth estimation and segmentation techniques

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonToday’s 3D imaging techniques offer significant benefits over conventional 2D imaging techniques. The presence of natural depth information in the scene affords the observer an overall improved sense of reality and naturalness. A variety of systems attempting to reach this goal have been designed by many independent research groups, such as stereoscopic and auto-stereoscopic systems. Though the images displayed by such systems tend to cause eye strain, fatigue and headaches after prolonged viewing as users are required to focus on the screen plane/accommodation to converge their eyes to a point in space in a different plane/convergence. Holoscopy is a 3D technology that targets overcoming the above limitations of current 3D technology and was recently developed at Brunel University. This work is part W4.1 of the 3D VIVANT project that is funded by the EU under the ICT program and coordinated by Dr. Aman Aggoun at Brunel University, West London, UK. The objective of the work described in this thesis is to develop estimation and segmentation techniques that are capable of estimating precise 3D depth, and are applicable for holoscopic 3D imaging system. Particular emphasis is given to the task of automatic techniques i.e. favours algorithms with broad generalisation abilities, as no constraints are placed on the setting. Algorithms that provide invariance to most appearance based variation of objects in the scene (e.g. viewpoint changes, deformable objects, presence of noise and changes in lighting). Moreover, have the ability to estimate depth information from both types of holoscopic 3D images i.e. Unidirectional and Omni-directional which gives horizontal parallax and full parallax (vertical and horizontal), respectively. The main aim of this research is to develop 3D depth estimation and 3D image segmentation techniques with great precision. In particular, emphasis on automation of thresholding techniques and cues identifications for development of robust algorithms. A method for depth-through-disparity feature analysis has been built based on the existing correlation between the pixels at a one micro-lens pitch which has been exploited to extract the viewpoint images (VPIs). The corresponding displacement among the VPIs has been exploited to estimate the depth information map via setting and extracting reliable sets of local features. ii Feature-based-point and feature-based-edge are two novel automatic thresholding techniques for detecting and extracting features that have been used in this approach. These techniques offer a solution to the problem of setting and extracting reliable features automatically to improve the performance of the depth estimation related to the generalizations, speed and quality. Due to the resolution limitation of the extracted VPIs, obtaining an accurate 3D depth map is challenging. Therefore, sub-pixel shift and integration is a novel interpolation technique that has been used in this approach to generate super-resolution VPIs. By shift and integration of a set of up-sampled low resolution VPIs, the new information contained in each viewpoint is exploited to obtain a super resolution VPI. This produces a high resolution perspective VPI with wide Field Of View (FOV). This means that the holoscopic 3D image system can be converted into a multi-view 3D image pixel format. Both depth accuracy and a fast execution time have been achieved that improved the 3D depth map. For a 3D object to be recognized the related foreground regions and depth information map needs to be identified. Two novel unsupervised segmentation methods that generate interactive depth maps from single viewpoint segmentation were developed. Both techniques offer new improvements over the existing methods due to their simple use and being fully automatic; therefore, producing the 3D depth interactive map without human interaction. The final contribution is a performance evaluation, to provide an equitable measurement for the extent of the success of the proposed techniques for foreground object segmentation, 3D depth interactive map creation and the generation of 2D super-resolution viewpoint techniques. The no-reference image quality assessment metrics and their correlation with the human perception of quality are used with the help of human participants in a subjective manner

An Analysis of Astronaut Performance Capability in the Lunar Environment. Volume 2 - Performance Capability Support Data

Astronaut performance capability in lunar environmen

On Martian Surface Exploration: Development of Automated 3D Reconstruction and Super-Resolution Restoration Techniques for Mars Orbital Images

Very high spatial resolution imaging and topographic (3D) data play an important role in modern Mars science research and engineering applications. This work describes a set of image processing and machine learning methods to produce the “best possible” high-resolution and high-quality 3D and imaging products from existing Mars orbital imaging datasets. The research work is described in nine chapters of which seven are based on separate published journal papers. These include a) a hybrid photogrammetric processing chain that combines the advantages of different stereo matching algorithms to compute stereo disparity with optimal completeness, fine-scale details, and minimised matching artefacts; b) image and 3D co-registration methods that correct a target image and/or 3D data to a reference image and/or 3D data to achieve robust cross-instrument multi-resolution 3D and image co-alignment; c) a deep learning network and processing chain to estimate pixel-scale surface topography from single-view imagery that outperforms traditional photogrammetric methods in terms of product quality and processing speed; d) a deep learning-based single-image super-resolution restoration (SRR) method to enhance the quality and effective resolution of Mars orbital imagery; e) a subpixel-scale 3D processing system using a combination of photogrammetric 3D reconstruction, SRR, and photoclinometric 3D refinement; and f) an optimised subpixel-scale 3D processing system using coupled deep learning based single-view SRR and deep learning based 3D estimation to derive the best possible (in terms of visual quality, effective resolution, and accuracy) 3D products out of present epoch Mars orbital images. The resultant 3D imaging products from the above listed new developments are qualitatively and quantitatively evaluated either in comparison with products from the official NASA planetary data system (PDS) and/or ESA planetary science archive (PSA) releases, and/or in comparison with products generated with different open-source systems. Examples of the scientific application of these novel 3D imaging products are discussed