자율주행을 위한 카메라 기반 거리 측정 및 측위

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·컴퓨터공학부, 2017. 8. 서승우.Automated driving vehicles or advanced driver assistance systems (ADAS) have continued to be an important research topic in transportation area. They can promise to reduce road accidents and eliminate traffic congestions. Automated driving vehicles are composed of two parts. On-board sensors are used to observe the environments and then, the captured sensor data are processed to interpret the environments and to make appropriate driving decisions. Some sensors have already been widely used in existing driver-assistance systems, e.g., camera systems are used in lane-keeping systems to recognize lanes on roadsradars (Radio Detection And Ranging) are used in adaptive cruise systems to measure the distance to a vehicle ahead such that a safe distance can be guaranteedLIDAR (Light Detection And Ranging) sensors are used in the autonomous emergency braking system to detect other vehicles or pedestrians in the vehicle path to avoid collisionaccelerometers are used to measure vehicle speed changes, which are especially useful for air-bagswheel encoder sensors are used to measure wheel rotations in a vehicle anti-lock brake system and GPS sensors are embedded on vehicles to provide the global positions of the vehicle for path navigation. In this dissertation, we cover three important application for automated driving vehicles by using camera sensors in vehicular environments. Firstly, precise and robust distance measurement is one of the most important requirements for driving assistance systems and automated driving systems. We propose a new method for providing accurate distance measurements through a frequency-domain analysis based on a stereo camera by exploiting key information obtained from the analysis of captured images. Secondly, precise and robust localization is another important requirement for safe automated driving. We propose a method for robust localization in diverse driving situations that measures the vehicle positions using a camera with respect to a given map for vision based navigation. The proposed method includes technology for removing dynamic objects and preserving features in vehicular environments using a background model accumulated from previous frames and we improve image quality using illuminant invariance characteristics of the log-chromaticity. We also propose a vehicle localization method using structure tensor and mutual information theory. Finally, we propose a novel algorithm for estimating the drivable collision-free space for autonomous navigation of on-road vehicles. In contrast to previous approaches that use stereo cameras or LIDAR, we solve this problem using a sensor fusion of cameras and LIDAR.1 Introduction 1 1.1 Background and Motivations 1 1.2 Contributions and Outline of the Dissertation 3 1.2.1 Accurate Object Distance Estimation based on Frequency-Domain Analysis with a Stereo Camera 3 1.2.2 Visual Map Matching based on Structural Tensor and Mutual Information using 3D High Resolution Digital Map 3 1.2.3 Free Space Computation using a Sensor Fusion of LIDAR and RGB camera in Vehicular Environment 4 2 Accurate Object Distance Estimation based on Frequency-Domain Analysis with a Stereo Camera 5 2.1 Introduction 5 2.2 Related Works 7 2.3 Algrorithm Description 10 2.3.1 Overall Procedure 10 2.3.2 Preliminaries 12 2.3.3 Pre-processing 12 2.4 Frequency-domain Analysis 15 2.4.1 Procedure 15 2.4.2 Contour-based Cost Computation 20 2.5 Cost Optimization and Distance Estimation 21 2.5.1 Disparity Optimization 21 2.5.2 Post-processing and Distance Estimation 23 2.6 Experimental Results 24 2.6.1 Test Environment 24 2.6.2 Experiment on KITTI Dataset 25 2.6.3 Performance Evaluation and Analysis 28 2.7 Conclusion 32 3 Visual Map Matching Based on Structural Tensor and Mutual Information using 3D High Resolution Digital Map 33 3.1 Introduction 33 3.2 Related Work 35 3.3 Methodology 37 3.3.1 Sensor Calibration 37 3.3.2 Digital Map Generation and Synthetic View Conversion 39 3.3.3 Dynamic Object Removal 41 3.3.4 Illuminant Invariance 43 3.3.5 Visual Map Matching using Structure Tensor and Mutual Information 43 3.4 Experiments and Result 49 3.4.1 Methodology 49 3.4.2 Quantitative Results 53 3.5 Conclusions and Future Works 54 4 Free Space Computation using a Sensor Fusion of LIDAR and RGB Camera in Vehicular Environments 55 4.1 Introduction 55 4.2 Methodology 57 4.2.1 Dense Depth Map Generation 57 4.2.2 Color Distribution Entropy 58 4.2.3 Edge Extraction 60 4.2.4 Temporal Smoothness 61 4.2.5 Spatial Smoothness 62 4.3 Experiment and Evaluation 63 4.3.1 Evaluated Methods 63 4.3.2 Experiment on KITTI Dataset 64 4.4 Conclusion 68 5 Conclusion 70 Abstract (In Korean) 87Docto

Similarity Measure Based on Entropy and Census and Multi-Resolution Disparity Estimation Technique for Stereo Matching

Stereo matching is one of the most active research areas in the field of computer vision. Stereo matching aims to obtain 3D information by extracting correct correspondence between two images captured from different point of views. There are two research parts in stereo matching: similarity measure between correspondence points and optimization technique for dence disparity estimation. The crux of stereo matching problem in similarity measure perspective is how to deal with the inferent points ambiguity that results from the ambiguous local appearances of image points. Similarity measures in stereo matching are classified as feature-based, intensity-based or non-parametric measure. And most similarity measures in the literatures are based on pixel intensity comparison. When images are taken at different illumination conditions or different sensors used, it is very unlikely that the corresponding pixels would have the same intensity creating false correspondences if it is only based on intensity matching functions alone. Especially illumination variations between input images can cause serious degrade in the performance of stereo matching algorithms. In this situation, mutual information-based method is powerful. However, it is still ambiguous or erroneous in considering local illumination variations between images. Therefore, similarity measure to these radiometric variations are demanded and become inevitable for stereo matching. Optimization method in stereo matching can be classified into two categories: local and global optimization methods, and most state-of-the-art algorithms fall into global optimization method. Global optimization methods can greatly suppress the matching ambiguities caused by various factors such as occluded and textureless regions. However, They are usually computationally expensive due to the slow-converging optimization process. In this paper, it was proposed that a stereo matching similarity measure based on entropy and census transform and an optimization technique using dynamic programming to estimate disparity efficiently based on multi-resolution method. Proposed similarity measure is composed of entropy, Haar wavelet feature vector, and modified Census transform. In general, mutual information similarity measure based on entropy about stereo images and disparity map is a popular and powerful similarity measure which is robust to complex intensity transformation. However, it is still ambiguous or erroneous with local radiometric variations, since it only accounts for global variation between images, and does not contain spatial information. Haar wavelet response can express frequency properties of image regions and is robust to various intensity changes and bias. Therefore, entropy was utilized with Haar wavelet feature vector as geometric measure. Modified Census transform was used as another spatial similarity measure. Census transform is a well-known non-parametric measure. And it is powerful to textureless and disparity discontinuity region and robust to noisy environment. A combination of entropy with Haar wavelet feature vector and modified Census transform as similarity measure was proposed to find correspondence. It is invariant to local radiometric variations and global illumination changes, so it can be applied to find correspondence for images which undergo local as well as global radiometric variations. Proposed optimization method is a new disparity estimation technique based on dynamic programming. A method using dynamic programming with 8-direction energy aggregation to estimate accurate disparity map was applied. Using 8-direction energy aggregation, accurate disparities can be found at disparity discontinuous region and suppress a streaking phenomenon in disparity map. Finally, the multi-resolution scheme was proposed to increase efficiency while processing and disparity estimation method. A Gaussian pyramid which prevent the ailasing at low-resolution image pyramid levels was used. And the multi-resolution scheme was proposed to perform matching at every levels to find accurate disparity. This method can perform matching efficiently and make accurate disparity map. And proposed method was validated with experimental results on stereo images.제 1 장 서론...........................................1 1.1 연구 목적 및 배경...............................1 1.2관련 연구........................................3 1.3연구 내용........................................6 1.4논문의 구성......................................7 제 2 장 스테레오 시각과 스테레오 정합..................8 2.1 스테레오 시각...................................8 2.2스테레오정합....................................10 2.2.1 유사도 척도.................................10 2.2.2 최적화 방법.................................15 2.3 환경 변화에 강인한 유사도 척도.................18 2.3.1 특징 기반 유사도 척도.......................18 2.3.2 명암도 기반 유사도 척도.....................19 2.3.3 비모수 유사도 척도..........................22 제 3 장 엔트로피 및 Census 기반의 유사도 척도.........24 3.1 엔트로피 기반의 유사도 척도....................26 3.1.1 엔트로피....................................26 3.1.2 엔트로피를 이용한 MI 유사도 척도............27 3.2 제안한 Haar 웨이블렛 특징을 결합한 엔트로피 유사도척도.................................................28 3.2.1 화소 단위 엔트로피.........................28 3.2.2 Haar웨이블렛 특징을 결합한 엔트로피........35 3.3 제안한Census 변환 기반의 유사도 척도..........46 3.3.1 Census 변환................................46 3.3.2 제안한 Census 변환을 이용한 유사도 척도....49 제 4 장 8방향 동적 계획법을 이용한 변위추정..........53 4.1 동적 계획법...................................53 4.2 제안한 8방향 동적 계획법......................57 제 5 장 다해상도 기반의 스테레오 정합................67 5.1 가우시안 영상 피라미드........................67 5.2 제안한 다해상도 기반 스테레오 정합............71 제 6 장 실험 및 고찰.................................77 6.1 정합 성능 평가 방법...........................77 6.2 스테레오 정합 실험............................79 6.2.1 RDS 영상 실험..............................79 6.2.2 환경 변화가 없는 표준 영상 실험............84 6.2.3 환경 변화가 발생한 표준 영상 실험..........92 6.2.4 실제 영상 실험............................110 6.3 계산 속도....................................118 6.4 제안한 방법의 정합 성능에 대한 고찰..........121 제 7 장 결 론.....................................123 참고 문헌...........................................12