Robust Feature Matching Method for SAR and Optical Images by Using Gaussian-Gamma-Shaped Bi-Windows-Based Descriptor and Geometric Constraint

Improving the matching reliability of multi-sensor imagery is one of the most challenging issues in recent years, particularly for synthetic aperture radar (SAR) and optical images. It is difficult to deal with the noise influence, geometric distortions, and nonlinear radiometric difference between SAR and optical images. In this paper, a method for SAR and optical images matching is proposed. First, interest points that are robust to speckle noise in SAR images are detected by improving the original phase-congruency-based detector. Second, feature descriptors are constructed for all interest points by combining a new Gaussian-Gamma-shaped bi-windows-based gradient operator and the histogram of oriented gradient pattern. Third, descriptor similarity and geometrical relationship are combined to constrain the matching processing. Finally, an approach based on global and local constraints is proposed to eliminate outliers. In the experiments, SAR images including COSMO-Skymed, RADARSAT-2, TerraSAR-X and HJ-1C images, and optical images including ZY-3 and Google Earth images are used to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed method provides significant improvements in the number of correct matches and matching precision compared with the state-of-the-art SIFT-like methods. Near 1 pixel registration accuracy is obtained based on the matching results of the proposed method

FEATURE MATCHING ENHANCEMENT OF UAV IMAGES USING GEOMETRIC CONSTRAINTS

Preliminary matching of image features is based on the distance between their descriptors. Matches are further filtered using RANSAC, or a similar method that fits the matches to a model; usually the fundamental matrix and rejects matches not belonging to that model. There are a few issues with this scheme. First, mismatches are no longer considered after RANSAC rejection. Second, RANSAC might fail to detect an accurate model if the number of outliers is significant. Third, a fundamental matrix model could be degenerate even if the matches are all inliers. To address these issues, a new method is proposed that relies on the prior knowledge of the images’ geometry, which can be obtained from the orientation sensors or a set of initial matches. Using a set of initial matches, a fundamental matrix and a global homography can be estimated. These two entities are then used with a detect-and-match strategy to gain more accurate matches. Features are detected in one image, then the locations of their correspondences in the other image are predicted using the epipolar constraints and the global homography. The feature correspondences are then corrected with template matching. Since global homography is only valid with a plane-to-plane mapping, discrepancy vectors are introduced to represent an alternative to local homographies. The method was tested on Unmanned Aerial Vehicle (UAV) images, where the images are usually taken successively, and differences in scale and orientation are not an issue. The method promises to find a well-distributed set of matches over the scene structure, especially with scenes of multiple depths. Furthermore; the number of outliers is reduced, encouraging to use a least square adjustment instead of RANSAC, to fit a non-degenerate model

Interferometric Synthetic Aperture RADAR and Radargrammetry towards the Categorization of Building Changes

The purpose of this work is the investigation of SAR techniques relying on multi image acquisition for fully automatic and rapid change detection analysis at building level. In particular, the benefits and limitations of a complementary use of two specific SAR techniques, InSAR and radargrammetry, in an emergency context are examined in term of quickness, globality and accuracy. The analysis is performed using spaceborne SAR data

A Study on Image Registration between High Resolution Optical and SAR Images Using SAR-SIFT and DLSS

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 건설환경공학부, 2018. 8. 김용일.최근 위성센서 기술의 발달로 다양한 센서를 탑재한 지구관측위성이 발사되면서, 다중센서 위성영상을 융합 분석하는 연구가 활발히 진행되고 있다. 특히, 광학영상과 SAR영상은 취급하는 파장대가 달라 동시에 활용할 경우 지표면에 대해 보다 구체적인 정보를 취득할 수 있으며, 더 나아가, 객체 추출, 변화탐지, 재난재해 모니터링 등 원격탐사 분야에 폭넓게 적용이 가능하다. 이를 위해서는 전처리 작업으로 두 영상 간 정합이 필수적으로 이루어져야 한다. 그러나, 광학영상과 SAR영상은 영상 취득시 위성센서 자세 및 취급하는 파장대의 상이함으로 기하 및 분광 정보 차이를 유발하여 영상 정합에 있어 유독 어려움이 존재한다. 이러한 차이는 건물이 밀집된 도심지역에서 부각되며, 중·저해상도 영상보다 고해상도 영상에서 두드러진다. 따라서, 본 연구에서는 도심지역에 대한 고해상도 광학영상과 SAR영상 간 정합에 효과적인 방법론을 제안하였다. 기존 광학영상과 SAR영상 간 정합 관련 연구는 크게 특징기반 정합기법과 강도기반 정합기법으로 진행되었다. 강도기반 정합기법은 분광 특성이 다른 영상 간 정합에 효과적이나, 영상 간 왜곡이 존재하지 않거나 기하학적 위치 차이가 적을 때에만 적용 가능하다. 고해상도 광학영상과 SAR영상은 지역적 왜곡이 존재하며, 두 영상 간 수십m 이상의 기하학적 위치 차이가 발생할 수 있다. 따라서, 고해상도 광학영상과 SAR영상 간 정합 연구는 강도기반 정합기법 보다 특징기반 정합기법이 중점적으로 진행되고 있다. 그러나, 특징기반 정합기법은 분광 특성이 다른 광학영상과 SAR영상에서 오정합쌍을 다수 추출하여 정합 성능이 떨어진다. 이를 해결하기 위해, 강도기반 정합기법과 특징기반 정합기법을 결합한 기법들이 제안되었으나, 도심지역에서 원 형상이 존재하는 지역이나 건물밀집지역을 제외한 지역 등과 같이 제한된 지역에서만 적용 가능하다는 한계점을 보였다. 이를 개선하기 위해, 본 연구에서는 특징기반 정합기법인 SAR-SIFT 기법과 강도기반 정합기법인 DLSS 기법을 결합한 정합기법을 제안하였다. 또한, 정합쌍을 추출하기 위해, 전처리 단계, 후보 정합쌍 추출 단계, 정밀 정합쌍 추출 단계인 총 세 단계를 추가하였다. 고해상도 광학영상과 SAR영상 간 정합을 위해서, SAR-SIFT 기법을 이용하여 특징점을 추출하고, 추출된 특징점에서 DLSS 기법을 이용하여 정합쌍을 추출하였다. 그러나, 추출된 정합쌍에 다수의 오정합쌍이 포함되는 문제점이 존재하였다. 이를 해결하기 위해, 추출된 정합쌍에서 임계치와 특징점 간 변위량을 이용한 전처리 단계와 후보 정합쌍 추출 단계를 통해 후보 정합쌍을 추출하고, 후보 정합쌍에 RANSAC 기법을 적용하여 정밀 정합쌍을 추출하는 방법을 제안하였다. 최종적으로 추출된 정밀 정합쌍을 이용하여 어핀 변환식(affine transformation)을 구성하고, 이를 적용하여 광학영상에 정합된 SAR영상을 생성하였다. 본 연구의 정확도를 검증하기 위하여, 대표적인 고해상도 광학영상인 KOMPSAT-2영상과 고해상도 SAR영상인 TerraSAR-X, Cosmo-SkyMed영상을 사용하였고, 시각적, 정량적 평가를 진행하였다. 시각적 평가를 위해서 모자이크 영상을 생성하였으며, 두 영상 간 경계에서 객체의 형상이 유지됨을 통해 정합이 우수하게 수행됨을 확인하였다. 정량적 평가를 위해서 수동 검사점을 통한 RMSE Ⅰ과 교차검증을 통한 RMSE Ⅱ를 사용하였으며, 모든 실험지역에 대해 RMSE Ⅰ은 1.51m에서 2.04m, RMSE Ⅱ는 1.34m에서 1.69m로 정확도가 도출되었다. 이는, 선행연구결과와 비교하였을 때 우수한 수준의 정확도로 확인되었다. 이를 통해, 제안 기법이 고해상도 광학영상과 SAR영상 간 정합에 효과적이며, 두 영상 간 융합 분석을 위해 효과적인 정합 기술로 활용될 것으로 사료된다.1. 서 론 1 1.1 연구배경 1 1.2 연구동향 4 1.3 연구의 목적 및 범위 7 2. 특징점 추출 10 2.1 영상 전처리 10 2.2 SAR-SIFT 기법을 통한 특징점 추출 11 2.2.1. SIFT 기법의 문제점 11 2.2.2. SAR-SIFT 기법 15 3. 정합쌍 추출 18 3.1 DLSS 기법을 통한 정합쌍 추출 18 3.1.1. 형상 서술자 LSS 19 3.1.2. 형상 서술자 벡터 DLSS 21 3.1.3. DLSS 기법의 문제점 22 3.2 제안된 정합쌍 추출 방법 24 3.2.1. 전처리 단계 24 3.2.2. 후보 정합쌍 추출 단계 26 3.2.3. 정밀 정합쌍 추출 단계 28 3.3 정합 및 정확도 평가 방법 29 3.3.1. 어핀 변환식 29 3.3.2. 정확도 평가 방법 31 4. 실험의 적용 및 평가 32 4.1 실험지역 및 자료 32 4.2 특징점 추출 결과 35 4.2.1. SIFT 기법을 통한 특징점 추출 결과 35 4.2.2. SAR-SIFT 기법을 통한 특징점 추출 결과 37 4.3 정합쌍 추출 결과 40 4.3.1. 기존 기법을 통한 정합쌍 추출 결과 40 4.3.2. 제안 기법을 통한 정합쌍 추출 결과 44 4.4 정합 결과 및 평가 49 5. 결론 55 Abstract 67Maste

Synthetic Aperture Radar (SAR) Meets Deep Learning

This reprint focuses on the application of the combination of synthetic aperture radars and depth learning technology. It aims to further promote the development of SAR image intelligent interpretation technology. A synthetic aperture radar (SAR) is an important active microwave imaging sensor, whose all-day and all-weather working capacity give it an important place in the remote sensing community. Since the United States launched the first SAR satellite, SAR has received much attention in the remote sensing community, e.g., in geological exploration, topographic mapping, disaster forecast, and traffic monitoring. It is valuable and meaningful, therefore, to study SAR-based remote sensing applications. In recent years, deep learning represented by convolution neural networks has promoted significant progress in the computer vision community, e.g., in face recognition, the driverless field and Internet of things (IoT). Deep learning can enable computational models with multiple processing layers to learn data representations with multiple-level abstractions. This can greatly improve the performance of various applications. This reprint provides a platform for researchers to handle the above significant challenges and present their innovative and cutting-edge research results when applying deep learning to SAR in various manuscript types, e.g., articles, letters, reviews and technical reports

Proceedings of the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

This book is a collection of 15 reviewed technical reports summarizing the presentations at the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory. The covered topics include image processing, optical signal processing, visual inspection, pattern recognition and classification, human-machine interaction, world and situation modeling, autonomous system localization and mapping, information fusion, and trust propagation in sensor networks

On-the-fly dense 3D surface reconstruction for geometry-aware augmented reality.

Augmented Reality (AR) is an emerging technology that makes seamless connections between virtual space and the real world by superimposing computer-generated information onto the real-world environment. AR can provide additional information in a more intuitive and natural way than any other information-delivery method that a human has ever in- vented. Camera tracking is the enabling technology for AR and has been well studied for the last few decades. Apart from the tracking problems, sensing and perception of the surrounding environment are also very important and challenging problems. Although there are existing hardware solutions such as Microsoft Kinect and HoloLens that can sense and build the environmental structure, they are either too bulky or too expensive for AR. In this thesis, the challenging real-time dense 3D surface reconstruction technologies are studied and reformulated for the reinvention of basic position-aware AR towards geometry-aware and the outlook of context- aware AR. We initially propose to reconstruct the dense environmental surface using the sparse point from Simultaneous Localisation and Map- ping (SLAM), but this approach is prone to fail in challenging Minimally Invasive Surgery (MIS) scenes such as the presence of deformation and surgical smoke. We subsequently adopt stereo vision with SLAM for more accurate and robust results. With the success of deep learning technology in recent years, we present learning based single image re- construction and achieve the state-of-the-art results. Moreover, we pro- posed context-aware AR, one step further from purely geometry-aware AR towards the high-level conceptual interaction modelling in complex AR environment for enhanced user experience. Finally, a learning-based smoke removal method is proposed to ensure an accurate and robust reconstruction under extreme conditions such as the presence of surgical smoke

Object Recognition

Vision-based object recognition tasks are very familiar in our everyday activities, such as driving our car in the correct lane. We do these tasks effortlessly in real-time. In the last decades, with the advancement of computer technology, researchers and application developers are trying to mimic the human's capability of visually recognising. Such capability will allow machine to free human from boring or dangerous jobs

Multi-scale texture segmentation of synthetic aperture radar images

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Pattern Recognition

A wealth of advanced pattern recognition algorithms are emerging from the interdiscipline between technologies of effective visual features and the human-brain cognition process. Effective visual features are made possible through the rapid developments in appropriate sensor equipments, novel filter designs, and viable information processing architectures. While the understanding of human-brain cognition process broadens the way in which the computer can perform pattern recognition tasks. The present book is intended to collect representative researches around the globe focusing on low-level vision, filter design, features and image descriptors, data mining and analysis, and biologically inspired algorithms. The 27 chapters coved in this book disclose recent advances and new ideas in promoting the techniques, technology and applications of pattern recognition