Ship Wake Detection in SAR Images via Sparse Regularization

In order to analyse synthetic aperture radar (SAR) images of the sea surface, ship wake detection is essential for extracting information on the wake generating vessels. One possibility is to assume a linear model for wakes, in which case detection approaches are based on transforms such as Radon and Hough. These express the bright (dark) lines as peak (trough) points in the transform domain. In this paper, ship wake detection is posed as an inverse problem, which the associated cost function including a sparsity enforcing penalty, i.e. the generalized minimax concave (GMC) function. Despite being a non-convex regularizer, the GMC penalty enforces the overall cost function to be convex. The proposed solution is based on a Bayesian formulation, whereby the point estimates are recovered using maximum a posteriori (MAP) estimation. To quantify the performance of the proposed method, various types of SAR images are used, corresponding to TerraSAR-X, COSMO-SkyMed, Sentinel-1, and ALOS2. The performance of various priors in solving the proposed inverse problem is first studied by investigating the GMC along with the L1, Lp, nuclear and total variation (TV) norms. We show that the GMC achieves the best results and we subsequently study the merits of the corresponding method in comparison to two state-of-the-art approaches for ship wake detection. The results show that our proposed technique offers the best performance by achieving 80% success rate.Comment: 18 page

훈련 자료 자동 추출 알고리즘과 기계 학습을 통한 SAR 영상 기반의 선박 탐지

학위논문 (석사) -- 서울대학교 대학원 : 자연과학대학 지구환경과학부, 2021. 2. 김덕진.Detection and surveillance of vessels are regarded as a crucial application of SAR for their contribution to the preservation of marine resources and the assurance on maritime safety. Introduction of machine learning to vessel detection significantly enhanced the performance and efficiency of the detection, but a substantial majority of studies focused on modifying the object detector algorithm. As the fundamental enhancement of the detection performance would be nearly impossible without accurate training data of vessels, this study implemented AIS information containing real-time information of vessel’s movement in order to propose a robust algorithm which acquires the training data of vessels in an automated manner. As AIS information was irregularly and discretely obtained, the exact target interpolation time for each vessel was precisely determined, followed by the implementation of Kalman filter, which mitigates the measurement error of AIS sensor. In addition, as the velocity of each vessel renders an imprint inside the SAR image named as Doppler frequency shift, it was calibrated by restoring the elliptic satellite orbit from the satellite state vector and estimating the distance between the satellite and the target vessel. From the calibrated position of the AIS sensor inside the corresponding SAR image, training data was directly obtained via internal allocation of the AIS sensor in each vessel. For fishing boats, separate information system named as VPASS was applied for the identical procedure of training data retrieval. Training data of vessels obtained via the automated training data procurement algorithm was evaluated by a conventional object detector, for three detection evaluating parameters: precision, recall and F1 score. All three evaluation parameters from the proposed training data acquisition significantly exceeded that from the manual acquisition. The major difference between two training datasets was demonstrated in the inshore regions and in the vicinity of strong scattering vessels in which land artifacts, ships and the ghost signals derived from them were indiscernible by visual inspection. This study additionally introduced a possibility of resolving the unclassified usage of each vessel by comparing AIS information with the accurate vessel detection results.전천후 지구 관측 위성인 SAR를 통한 선박 탐지는 해양 자원의 확보와 해상 안전 보장에 매우 중요한 역할을 한다. 기계 학습 기법의 도입으로 인해 선박을 비롯한 사물 탐지의 정확도 및 효율성이 향상되었으나, 이와 관련된 다수의 연구는 탐지 알고리즘의 개량에 집중되었다. 그러나, 탐지 정확도의 근본적인 향상은 정밀하게 취득된 대량의 훈련자료 없이는 불가능하기에, 본 연구에서는 선박의 실시간 위치, 속도 정보인 AIS 자료를 이용하여 인공 지능 기반의 선박 탐지 알고리즘에 사용될 훈련자료를 자동적으로 취득하는 알고리즘을 제안하였다. 이를 위해 이산적인 AIS 자료를 SAR 영상의 취득시각에 맞추어 정확하게 보간하고, AIS 센서 자체가 가지는 오차를 최소화하였다. 또한, 이동하는 산란체의 시선 속도로 인해 발생하는 도플러 편이 효과를 보정하기 위해 SAR 위성의 상태 벡터를 이용하여 위성과 산란체 사이의 거리를 정밀하게 계산하였다. 이렇게 계산된 AIS 센서의 영상 내의 위치로부터 선박 내 AIS 센서의 배치를 고려하여 선박 탐지 알고리즘의 훈련자료 형식에 맞추어 훈련자료를 취득하고, 어선에 대한 위치, 속도 정보인 VPASS 자료 역시 유사한 방법으로 가공하여 훈련자료를 취득하였다. AIS 자료로부터 취득한 훈련자료는 기존 방법대로 수동 취득한 훈련자료와 함께 인공 지능 기반 사물 탐지 알고리즘을 통해 정확도를 평가하였다. 그 결과, 제시된 알고리즘으로 취득한 훈련 자료는 수동 취득한 훈련 자료 대비 더 높은 탐지 정확도를 보였으며, 이는 기존의 사물 탐지 알고리즘의 평가 지표인 정밀도, 재현율과 F1 score를 통해 진행되었다. 본 연구에서 제안한 훈련자료 자동 취득 기법으로 얻은 선박에 대한 훈련자료는 특히 기존의 선박 탐지 기법으로는 분별이 어려웠던 항만에 인접한 선박과 산란체 주변의 신호에 대한 정확한 분별 결과를 보였다. 본 연구에서는 이와 함께, 선박 탐지 결과와 해당 지역에 대한 AIS 및 VPASS 자료를 이용하여 선박의 미식별성을 판정할 수 있는 가능성 또한 제시하였다.Chapter 1. Introduction - 1 - 1.1 Research Background - 1 - 1.2 Research Objective - 8 - Chapter 2. Data Acquisition - 10 - 2.1 Acquisition of SAR Image Data - 10 - 2.2 Acquisition of AIS and VPASS Information - 20 - Chapter 3. Methodology on Training Data Procurement - 26 - 3.1 Interpolation of Discrete AIS Data - 29 - 3.1.1 Estimation of Target Interpolation Time for Vessels - 29 - 3.1.2 Application of Kalman Filter to AIS Data - 34 - 3.2 Doppler Frequency Shift Correction - 40 - 3.2.1 Theoretical Basis of Doppler Frequency Shift - 40 - 3.2.2 Mitigation of Doppler Frequency Shift - 48 - 3.3 Retrieval of Training Data of Vessels - 53 - 3.4 Algorithm on Vessel Training Data Acquisition from VPASS Information - 61 - Chapter 4. Methodology on Object Detection Architecture - 66 - Chapter 5. Results - 74 - 5.1 Assessment on Training Data - 74 - 5.2 Assessment on AIS-based Ship Detection - 79 - 5.3 Assessment on VPASS-based Fishing Boat Detection - 91 - Chapter 6. Discussions - 110 - 6.1 Discussion on AIS-Based Ship Detection - 110 - 6.2 Application on Determining Unclassified Vessels - 116 - Chapter 7. Conclusion - 125 - 국문 요약문 - 128 - Bibliography - 130 -Maste

A comparative study of operational vessel detectors for maritime surveillance using satellite-borne synthetic aperture radar

This paper presents a comparative study among four operational detectors that work by automatically post-processing synthetic aperture radar (SAR) images acquired from the satellite platforms RADARSAT-2 and COSMO-SkyMed. Challenging maritime scenarios have been chosen to assess the detectors' performance against features such as ambiguities, significant sea clutter, or irregular shorelines. The SAR images which form the test data are complemented with ground truth to define the reference detection configuration, which permits quantifying the probability of detection, the false alarm rate, and the accuracy of estimating ship dimensions. Although the results show that all the detectors perform well, there is no perfect detector, and a better detection system could be developed that combines the best elements from each of the single detectors. In addition to the comparison exercise, the study has facilitated the improvement of the detectors by highlighting weaknesses and providing means for fixing them.Peer ReviewedPostprint (published version

Space-based Global Maritime Surveillance. Part I: Satellite Technologies

Maritime surveillance (MS) is crucial for search and rescue operations, fishery monitoring, pollution control, law enforcement, migration monitoring, and national security policies. Since the early days of seafaring, MS has been a critical task for providing security in human coexistence. Several generations of sensors providing detailed maritime information have become available for large offshore areas in real time: maritime radar sensors in the 1950s and the automatic identification system (AIS) in the 1990s among them. However, ground-based maritime radars and AIS data do not always provide a comprehensive and seamless coverage of the entire maritime space. Therefore, the exploitation of space-based sensor technologies installed on satellites orbiting around the Earth, such as satellite AIS data, synthetic aperture radar, optical sensors, and global navigation satellite systems reflectometry, becomes crucial for MS and to complement the existing terrestrial technologies. In the first part of this work, we provide an overview of the main available space-based sensors technologies and present the advantages and limitations of each technology in the scope of MS. The second part, related to artificial intelligence, signal processing and data fusion techniques, is provided in a companion paper, titled: "Space-based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques" [1].Comment: This paper has been submitted to IEEE Aerospace and Electronic Systems Magazin

The Costa Concordia last cruise: The first application of high frequency monitoring based on COSMO-SkyMed constellation for wreck removal

AbstractThe Italian vessel Costa Concordia wrecked on January 13th 2012 offshore the Giglio Island (Tuscany, Italy), with the loss of 32 lives. Salvage operation of the vessel started immediately after the wreck. This operation was the largest and most expensive maritime salvage ever attempted on a wrecked ship and it ended in July 2014 when the Costa Concordia was removed from the Giglio Island, and dragged in the port of Genoa where it was dismantled. The refloating and removal phases of the Costa Concordia were monitored, in the period between 14th and 27th of July, exploiting SAR (Synthetic Aperture Radar) images acquired by the X-band COSMO-SkyMed satellite constellation in crisis mode. The main targets of the monitoring system were: (i) the detection of possible spill of pollutant material from the vessel and (ii) to exclude that oil slicks, illegally produced by other vessels, could be improperly linked to the naval convoy during its transit along the route between the Giglio Island and the port of Genoa. Results point out that the adopted monitoring system, through the use of the COSMO-SkyMed constellation, can be profitably employed to monitor emergency phases related to single ship or naval convoy over wide areas and with a suitable temporal coverage. Furthermore, the refloating and removal phases of the Costa Concordia were a success because no pollution was produced during the operations

Maritime vessel classification to monitor fisheries with SAR: demonstration in the North Sea

Integration of methods based on satellite remote sensing into current maritime monitoring strategies could help tackle the problem of global overfishing. Operational software is now available to perform vessel detection on satellite imagery, but research on vessel classification has mainly focused on bulk carriers, container ships, and oil tankers, using high-resolution commercial Synthetic Aperture Radar (SAR) imagery. Here, we present a method based on Random Forest (RF) to distinguish fishing and non-fishing vessels, and apply it to an area in the North Sea. The RF classifier takes as input the vessel’s length, longitude, and latitude, its distance to the nearest shore, and the time of the measurement (am or pm). The classifier is trained and tested on data from the Automatic Identification System (AIS). The overall classification accuracy is 91%, but the precision for the fishing class is only 58% because of specific regions in the study area where activities of fishing and non-fishing vessels overlap. We then apply the classifier to a collection of vessel detections obtained by applying the Search for Unidentified Maritime Objects (SUMO) vessel detector to the 2017 Sentinel-1 SAR images of the North Sea. The trend in our monthly fishing-vessel count agrees with data from Global Fishing Watch on fishing-vessel presence. These initial results suggest that our approach could help monitor intensification or reduction of fishing activity, which is critical in the context of the global overfishing problem

A combined use of multispectral and SAR images for ship detection and characterization through object based image analysis

Marine routes represent a huge portion of commercial and human trades, therefore surveillance, security and environmental protection themes are gaining increasing importance. Being able to overcome the limits imposed by terrestrial means of monitoring, ship detection from satellite has recently prompted a renewed interest for a continuous monitoring of illegal activities. This paper describes an automatic Object Based Image Analysis (OBIA) approach to detect vessels made of different materials in various sea environments. The combined use of multispectral and SAR images allows for a regular observation unrestricted by lighting and atmospheric conditions and complementarity in terms of geographic coverage and geometric detail. The method developed adopts a region growing algorithm to segment the image in homogeneous objects, which are then classified through a decision tree algorithm based on spectral and geometrical properties. Then, a spatial analysis retrieves the vessels' position, length and heading parameters and a speed range is associated. Optimization of the image processing chain is performed by selecting image tiles through a statistical index. Vessel candidates are detected over amplitude SAR images using an adaptive threshold Constant False Alarm Rate (CFAR) algorithm prior the object based analysis. Validation is carried out by comparing the retrieved parameters with the information provided by the Automatic Identification System (AIS), when available, or with manual measurement when AIS data are not available. The estimation of length shows R2=0.85 and estimation of heading R2=0.92, computed as the average of R2values obtained for both optical and radar images