Automated interpretation of benthic stereo imagery

Autonomous benthic imaging, reduces human risk and increases the amount of collected data. However, manually interpreting these high volumes of data is onerous, time consuming and in many cases, infeasible. The objective of this thesis is to improve the scientific utility of the large image datasets. Fine-scale terrain complexity is typically quantified by rugosity and measured by divers using chains and tape measures. This thesis proposes a new technique for measuring terrain complexity from 3D stereo image reconstructions, which is non-contact and can be calculated at multiple scales over large spatial extents. Using robots, terrain complexity can be measured without endangering humans, beyond scuba depths. Results show that this approach is more robust, flexible and easily repeatable than traditional methods. These proposed terrain complexity features are combined with visual colour and texture descriptors and applied to classifying imagery. New multi-dataset feature selection methods are proposed for performing feature selection across multiple datasets, and are shown to improve the overall classification performance. The results show that the most informative predictors of benthic habitat types are the new terrain complexity measurements. This thesis presents a method that aims to reduce human labelling effort, while maximising classification performance by combining pre-clustering with active learning. The results support that utilising the structure of the unlabelled data in conjunction with uncertainty sampling can significantly reduce the number of labels required for a given level of accuracy. Typically 0.00001–0.00007% of image data is annotated and processed for science purposes (20–50 points in 1–2% of the images). This thesis proposes a framework that uses existing human-annotated point labels to train a superpixel-based automated classification system, which can extrapolate the classified results to every pixel across all the images of an entire survey

Automated interpretation of benthic stereo imagery

Autonomous benthic imaging, reduces human risk and increases the amount of collected data. However, manually interpreting these high volumes of data is onerous, time consuming and in many cases, infeasible. The objective of this thesis is to improve the scientific utility of the large image datasets. Fine-scale terrain complexity is typically quantified by rugosity and measured by divers using chains and tape measures. This thesis proposes a new technique for measuring terrain complexity from 3D stereo image reconstructions, which is non-contact and can be calculated at multiple scales over large spatial extents. Using robots, terrain complexity can be measured without endangering humans, beyond scuba depths. Results show that this approach is more robust, flexible and easily repeatable than traditional methods. These proposed terrain complexity features are combined with visual colour and texture descriptors and applied to classifying imagery. New multi-dataset feature selection methods are proposed for performing feature selection across multiple datasets, and are shown to improve the overall classification performance. The results show that the most informative predictors of benthic habitat types are the new terrain complexity measurements. This thesis presents a method that aims to reduce human labelling effort, while maximising classification performance by combining pre-clustering with active learning. The results support that utilising the structure of the unlabelled data in conjunction with uncertainty sampling can significantly reduce the number of labels required for a given level of accuracy. Typically 0.00001–0.00007% of image data is annotated and processed for science purposes (20–50 points in 1–2% of the images). This thesis proposes a framework that uses existing human-annotated point labels to train a superpixel-based automated classification system, which can extrapolate the classified results to every pixel across all the images of an entire survey

ACCURATE AND FAST STEREO VISION

Stereo vision from short-baseline image pairs is one of the most active research fields in computer vision. The estimation of dense disparity maps from stereo image pairs is still a challenging task and there is further space for improving accuracy, minimizing the computational cost and handling more efficiently outliers, low-textured areas, repeated textures, disparity discontinuities and light variations. This PhD thesis presents two novel methodologies relating to stereo vision from short-baseline image pairs: I. The first methodology combines three different cost metrics, defined using colour, the CENSUS transform and SIFT (Scale Invariant Feature Transform) coefficients. The selected cost metrics are aggregated based on an adaptive weights approach, in order to calculate their corresponding cost volumes. The resulting cost volumes are merged into a combined one, following a novel two-phase strategy, which is further refined by exploiting semi-global optimization. A mean-shift segmentation-driven approach is exploited to deal with outliers in the disparity maps. Additionally, low-textured areas are handled using disparity histogram analysis, which allows for reliable disparity plane fitting on these areas. II. The second methodology relies on content-based guided image filtering and weighted semi-global optimization. Initially, the approach uses a pixel-based cost term that combines gradient, Gabor-Feature and colour information. The pixel-based matching costs are filtered by applying guided image filtering, which relies on support windows of two different sizes. In this way, two filtered costs are estimated for each pixel. Among the two filtered costs, the one that will be finally assigned to each pixel, depends on the local image content around this pixel. The filtered cost volume is further refined by exploiting weighted semi-global optimization, which improves the disparity accuracy. The handling of the occluded areas is enhanced by incorporating a straightforward and time efficient scheme. The evaluation results show that both methodologies are very accurate, since they handle efficiently low-textured/occluded areas and disparity discontinuities. Additionally, the second approach has very low computational complexity. Except for the aforementioned two methodologies that use as input short-baseline image pairs, this PhD thesis presents a novel methodology for generating 3D point clouds of good accuracy from wide-baseline stereo pairs

Similarity Measure Based on Entropy and Census and Multi-Resolution Disparity Estimation Technique for Stereo Matching

Stereo matching is one of the most active research areas in the field of computer vision. Stereo matching aims to obtain 3D information by extracting correct correspondence between two images captured from different point of views. There are two research parts in stereo matching: similarity measure between correspondence points and optimization technique for dence disparity estimation. The crux of stereo matching problem in similarity measure perspective is how to deal with the inferent points ambiguity that results from the ambiguous local appearances of image points. Similarity measures in stereo matching are classified as feature-based, intensity-based or non-parametric measure. And most similarity measures in the literatures are based on pixel intensity comparison. When images are taken at different illumination conditions or different sensors used, it is very unlikely that the corresponding pixels would have the same intensity creating false correspondences if it is only based on intensity matching functions alone. Especially illumination variations between input images can cause serious degrade in the performance of stereo matching algorithms. In this situation, mutual information-based method is powerful. However, it is still ambiguous or erroneous in considering local illumination variations between images. Therefore, similarity measure to these radiometric variations are demanded and become inevitable for stereo matching. Optimization method in stereo matching can be classified into two categories: local and global optimization methods, and most state-of-the-art algorithms fall into global optimization method. Global optimization methods can greatly suppress the matching ambiguities caused by various factors such as occluded and textureless regions. However, They are usually computationally expensive due to the slow-converging optimization process. In this paper, it was proposed that a stereo matching similarity measure based on entropy and census transform and an optimization technique using dynamic programming to estimate disparity efficiently based on multi-resolution method. Proposed similarity measure is composed of entropy, Haar wavelet feature vector, and modified Census transform. In general, mutual information similarity measure based on entropy about stereo images and disparity map is a popular and powerful similarity measure which is robust to complex intensity transformation. However, it is still ambiguous or erroneous with local radiometric variations, since it only accounts for global variation between images, and does not contain spatial information. Haar wavelet response can express frequency properties of image regions and is robust to various intensity changes and bias. Therefore, entropy was utilized with Haar wavelet feature vector as geometric measure. Modified Census transform was used as another spatial similarity measure. Census transform is a well-known non-parametric measure. And it is powerful to textureless and disparity discontinuity region and robust to noisy environment. A combination of entropy with Haar wavelet feature vector and modified Census transform as similarity measure was proposed to find correspondence. It is invariant to local radiometric variations and global illumination changes, so it can be applied to find correspondence for images which undergo local as well as global radiometric variations. Proposed optimization method is a new disparity estimation technique based on dynamic programming. A method using dynamic programming with 8-direction energy aggregation to estimate accurate disparity map was applied. Using 8-direction energy aggregation, accurate disparities can be found at disparity discontinuous region and suppress a streaking phenomenon in disparity map. Finally, the multi-resolution scheme was proposed to increase efficiency while processing and disparity estimation method. A Gaussian pyramid which prevent the ailasing at low-resolution image pyramid levels was used. And the multi-resolution scheme was proposed to perform matching at every levels to find accurate disparity. This method can perform matching efficiently and make accurate disparity map. And proposed method was validated with experimental results on stereo images.제 1 장 서론...........................................1 1.1 연구 목적 및 배경...............................1 1.2관련 연구........................................3 1.3연구 내용........................................6 1.4논문의 구성......................................7 제 2 장 스테레오 시각과 스테레오 정합..................8 2.1 스테레오 시각...................................8 2.2스테레오정합....................................10 2.2.1 유사도 척도.................................10 2.2.2 최적화 방법.................................15 2.3 환경 변화에 강인한 유사도 척도.................18 2.3.1 특징 기반 유사도 척도.......................18 2.3.2 명암도 기반 유사도 척도.....................19 2.3.3 비모수 유사도 척도..........................22 제 3 장 엔트로피 및 Census 기반의 유사도 척도.........24 3.1 엔트로피 기반의 유사도 척도....................26 3.1.1 엔트로피....................................26 3.1.2 엔트로피를 이용한 MI 유사도 척도............27 3.2 제안한 Haar 웨이블렛 특징을 결합한 엔트로피 유사도척도.................................................28 3.2.1 화소 단위 엔트로피.........................28 3.2.2 Haar웨이블렛 특징을 결합한 엔트로피........35 3.3 제안한Census 변환 기반의 유사도 척도..........46 3.3.1 Census 변환................................46 3.3.2 제안한 Census 변환을 이용한 유사도 척도....49 제 4 장 8방향 동적 계획법을 이용한 변위추정..........53 4.1 동적 계획법...................................53 4.2 제안한 8방향 동적 계획법......................57 제 5 장 다해상도 기반의 스테레오 정합................67 5.1 가우시안 영상 피라미드........................67 5.2 제안한 다해상도 기반 스테레오 정합............71 제 6 장 실험 및 고찰.................................77 6.1 정합 성능 평가 방법...........................77 6.2 스테레오 정합 실험............................79 6.2.1 RDS 영상 실험..............................79 6.2.2 환경 변화가 없는 표준 영상 실험............84 6.2.3 환경 변화가 발생한 표준 영상 실험..........92 6.2.4 실제 영상 실험............................110 6.3 계산 속도....................................118 6.4 제안한 방법의 정합 성능에 대한 고찰..........121 제 7 장 결 론.....................................123 참고 문헌...........................................12