Assessment of Structure from Motion for Reconnaissance Augmentation and Bandwidth Usage Reduction

Modern militaries rely upon remote image sensors for real-time intelligence. A typical remote system consists of an unmanned aerial vehicle, or UAV, with an attached camera. A video stream is sent from the UAV, through a bandwidth-constrained satellite connection, to an intelligence processing unit. In this research, an upgrade to this method of collection is proposed. A set of synthetic images of a scene captured by a UAV in a virtual environment is sent to a pipeline of computer vision algorithms, collectively known as Structure from Motion. The output of Structure from Motion, a three-dimensional model, is then assessed in a 3D virtual world as a possible replacement for the images from which it was created. This study shows Structure from Motion results from a modifiable spiral flight path and compares the geoaccuracy of each result. A flattening of height is observed, and an automated compensation for this flattening is performed. Each reconstruction is also compressed, and the size of the compression is compared with the compressed size of the images from which it was created. A reduction of 49-60% of required space is shown

One-Two-One Network for Compression Artifacts Reduction in Remote Sensing

Compression artifacts reduction (CAR) is a challenging problem in the field of remote sensing. Most recent deep learning based methods have demonstrated superior performance over the previous hand-crafted methods. In this paper, we propose an end-to-end one-two-one (OTO) network, to combine different deep models, i.e., summation and difference models, to solve the CAR problem. Particularly, the difference model motivated by the Laplacian pyramid is designed to obtain the high frequency information, while the summation model aggregates the low frequency information. We provide an in-depth investigation into our OTO architecture based on the Taylor expansion, which shows that these two kinds of information can be fused in a nonlinear scheme to gain more capacity of handling complicated image compression artifacts, especially the blocking effect in compression. Extensive experiments are conducted to demonstrate the superior performance of the OTO networks, as compared to the state-of-the-arts on remote sensing datasets and other benchmark datasets. The source code will be available here: https://github.com/bczhangbczhang/

Using Shadows to Detect Targets in Synthetic Aperture Radar Imagery

Synthetic Aperture Radar (SAR) can generate high resolution imagery of re- mote scenes by combining the phase information of multiple radar pulses along a given path. SAR based Intelligence, Surveillance, and Reconnaissance (ISR) has the advantage over optical ISR that it can provide usable imagery in adverse weather or nighttime conditions. Certain radar frequencies can even result in foliage or limited soil penetration, enabling imagery to be created of objects of interest that would otherwise be hidden from optical surveillance systems. This thesis demonstrates the capability of locating stationary targets of interest based on the locations of their shadows and the characteristics of pixel intensity distributions within the SAR imagery. Shadows, in SAR imagery, represent the absence of a detectable signal reflection due to the physical obstruction of the transmitted radar energy. An object\u27s shadow indicates its true geospatial location. This thesis demonstrates target detection based on shadow location using three types of target vehicles, each located in urban and rural clutter scenes, from the publicly available Moving and Stationary Target Acquisition and Recognition (MSTAR) data set. The proposed distribution characterization method for detecting shadows demonstrates the capability of isolating distinct regions within SAR imagery and using the junctions between shadow and non-shadow regions to locate individual shadow-casting objects. Targets of interest are then located within that collection of objects with an average detection accuracy rate of 93%. The shadow-based target detection algorithm results in a lower false alarm rate compared to previous research conducted with the same data set, with 71% fewer false alarms for the same clutter region. Utilizing the absence of signal, in conjunction with surrounding signal reflections, provides accurate stationary target detection. This capability could greatly assist in track initialization or the location of otherwise obscured targets of interest

CNN기반의 FusionNet 신경망과 농지 경계추출 알고리즘을 이용한 토지피복분류모델 개발

학위논문 (석사) -- 서울대학교 대학원 : 농업생명과학대학 생태조경.지역시스템공학부(지역시스템공학전공), 2021. 2. 송인홍.토지이용이 빠르게 변화함에 따라, 토지 피복에 대한 공간정보를 담고 있는 토지 피복 지도의 신속한 최신화는 필수적이다. 하지만, 현 토지 피복 지도는 많은 시간과 노동력을 요구하는 manual digitizing 방법으로 제작됨에 따라, 토지 피복 지도의 업데이트 및 배포에 긴 시간 간격이 발생하는 실정이다. 본 연구에서는 convolutional neural network (CNN) 기반의 인공신경망을 이용하여 high-resolution remote sensing (HRRS) 영상으로부터 토지 피복을 분류하는 모델을 개발하고, 특히 농지 경계추출 알고리즘을 적용하여 농업지역에서 분류 정확도를 개선하고자 하였다. 개발된 토지 피복 분류모델은 전처리(pre-processing) 모듈, 토지 피복 분류(land cover classification) 모듈, 그리고 후처리(post-processing) 모듈의 세 모듈로 구성된다. 전처리 모듈은 입력된 HRRS 영상을 75%씩 중첩 분할하여 관점을 다양화하는 모듈로, 한 관점에서 토지 피복을 분류할 때 발생할 수 있는 오분류를 줄이고자 하였다. 토지 피복 분류 모듈은 FusionNet model 구조를 바탕으로 개발되었고, 이는 분할된 HRRS 이미지의 픽셀별로 최적 토지 피복을 부여하도록 설계되었다. 후처리 모듈은 픽셀별 최종 토지 피복을 결정하는 모듈로, 분할된 HRRS 이미지의 분류결과를 취합하여 최빈값을 최종 토지 피복으로 결정한다. 추가로 농지에서는 농지경계를 추출하고, 필지별 분류된 토지 피복을 집계하여 한 필지에 같은 토지 피복을 부여하였다. 개발된 토지 피복 분류모델은 전라남도 지역(면적: 547 km2)의 2018년 정사영상과 토지 피복 지도를 이용하여 학습되었다. 토지 피복 분류모델 검증은 학습지역과 시간, 공간적으로 구분된, 2018년 전라남도 수북면과 2016년 충청북도 대소면의 두 검증지역에서 수행되었다. 각 검증지역에서 overall accuracy는 0.81, 0.71로 집계되었고, kappa coefficients는 0.75, 0.64로 산정되어 substantial 수준의 토지 피복 분류 정확도를 확인하였다. 특히, 개발된 모델은 필지 경계를 고려한 농업지역에서 overall accuracy 0.89, kappa coefficient 0.81로 almost perfect 수준의 우수한 분류 정확도를 보였다. 이에 개발된 토지 피복 분류모델은 특히 농업지역에서 현 토지 피복 분류 방법을 지원하여 토지 피복 지도의 빠르고 정확한 최신화에 기여할 수 있을 것으로 생각된다.The rapid update of land cover maps is necessary because spatial information of land cover is widely used in various areas. However, these maps have been released or updated in the interval of several years primarily owing to the manual digitizing method, which is time-consuming and labor-intensive. This study was aimed to develop a land cover classification model using the concept of a convolutional neural network (CNN) that classifies land cover labels from high-resolution remote sensing (HRRS) images and to increase the classification accuracy in agricultural areas using the parcel boundary extraction algorithm. The developed model comprises three modules, namely the pre-processing, land cover classification, and post-processing modules. The pre-processing module diversifies the perspective of the HRRS images by separating images with 75% overlaps to reduce the misclassification that can occur in a single image. The land cover classification module was designed based on the FusionNet model structure, and the optimal land cover type was assigned for each pixel of the separated HRRS images. The post-processing module determines the ultimate land cover types for each pixel unit by summing up the several-perspective classification results and aggregating the pixel-classification result for the parcel-boundary unit in agricultural areas. The developed model was trained with land cover maps and orthographic images (area: 547 km2) from the Jeonnam province in Korea. Model validation was conducted with two spatially and temporally different sites including Subuk-myeon of Jeonnam province in 2018 and Daseo-myeon of Chungbuk province in 2016. In the respective validation sites, the models overall accuracies were 0.81 and 0.71, and kappa coefficients were 0.75 and 0.64, implying substantial model performance. The model performance was particularly better when considering parcel boundaries in agricultural areas, exhibiting an overall accuracy of 0.89 and kappa coefficient 0.81 (almost perfect). It was concluded that the developed model may help perform rapid and accurate land cover updates especially for agricultural areas.Chapter 1. Introduction 1 1.1. Study background 1 1.2. Objective of thesis 4 Chapter 2. Literature review 6 2.1. Development of remote sensing technique 6 2.2. Land cover segmentation 9 2.3. Land boundary extraction 13 Chapter 3. Development of the land cover classification model 15 3.1. Conceptual structure of the land cover classification model 15 3.2. Pre-processing module 16 3.3. CNN based land cover classification module 17 3.4. Post processing module 22 3.4.1 Determination of land cover in a pixel unit 22 3.4.2 Aggregation of land cover to parcel boundary 24 Chapter 4. Verification of the land cover classification model 30 4.1. Study area and data acquisition 31 4.1.1. Training area 31 4.1.2. Verification area 32 4.1.3. Data acquisition 33 4.2. Training the land cover classification model 36 4.3. Verification method 37 4.3.1. The performance measurement methods of land cover classification model 37 4.3.2. Accuracy estimation methods of agricultural parcel boundary 39 4.3.3. Comparison of boundary based classification result with ERDAS Imagine 41 4.4. Verification of land cover classification model 42 4.4.1. Performance of land cover classification at the child subcategory 42 4.4.2. Classification accuracy of the aggregated land cover to main category 46 4.4.3. Classification accuracy of boundary based aggregation in agricultural area 57 Chapter 5. Conclusions 71 Reference 73 국 문 초 록 83Maste

The Effects of Signal and Image Compression of SAR Data on Change Detection Algorithms

With massive amounts of SAR imagery and data being collected, the need for effective compression techniques is growing. One of the most popular applications for remote sensing is change detection, which compares two geo-registered images for changes in the scene. While lossless compression is needed for signal compression, the same is not often required for image compression. In almost every case the compression ratios are much higher in lossy compression making them more appealing when bandwidth and storage becomes an issue. This research analyzes different types of compression techniques that are adapted for SAR imagery, and tests these techniques with three different change detection algorithms. Many algorithms exist that allow large compression ratios, however, the usefulness of the data is always the final concern. It is necessary to identify compression methods that will not degrade the performance of change detection analysis

Clearing the Clouds: Extracting 3D information from amongst the noise

Advancements permitting the rapid extraction of 3D point clouds from a variety of imaging modalities across the global landscape have provided a vast collection of high fidelity digital surface models. This has created a situation with unprecedented overabundance of 3D observations which greatly outstrips our current capacity to manage and infer actionable information. While years of research have removed some of the manual analysis burden for many tasks, human analysis is still a cornerstone of 3D scene exploitation. This is especially true for complex tasks which necessitate comprehension of scale, texture and contextual learning. In order to ameliorate the interpretation burden and enable scientific discovery from this volume of data, new processing paradigms are necessary to keep pace. With this context, this dissertation advances fundamental and applied research in 3D point cloud data pre-processing and deep learning from a variety of platforms. We show that the representation of 3D point data is often not ideal and sacrifices fidelity, context or scalability. First ground scanning terrestrial LIght Detection And Ranging (LiDAR) models are shown to have an inherent statistical bias, and present a state of the art method for correcting this, while preserving data fidelity and maintaining semantic structure. This technique is assessed in the dense canopy of Micronesia, with our technique being the best at retaining high levels of detail under extreme down-sampling (\u3c 1%). Airborne systems are then explored with a method which is presented to pre-process data to preserve a global contrast and semantic content in deep learners. This approach is validated with a building footprint detection task from airborne imagery captured in Eastern TN from the 3D Elevation Program (3DEP), our approach was found to achieve significant accuracy improvements over traditional techniques. Finally, topography data spanning the globe is used to assess past and previous global land cover change. Utilizing Shuttle Radar Topography Mission (SRTM) and Moderate Resolution Imaging Spectroradiometer (MODIS) data, paired with the airborne preprocessing technique described previously, a model for predicting land-cover change from topography observations is described. The culmination of these efforts have the potential to enhance the capabilities of automated 3D geospatial processing, substantially lightening the burden of analysts, with implications improving our responses to global security, disaster response, climate change, structural design and extraplanetary exploration