Editorial for the Special Issue "Remote Sensing of Target Detection in Marine Environment"

First paragraph: Remote sensing is a powerful tool used to obtain an unprecedented amount of information about the ocean from a distance, usually from satellites or aircrafts. Measurements collected by active and passive remote sensing instruments can be used for both marine and maritime applications. They allow monitoring of vast areas of the Earth that are difficult to access and sample using traditional methods. Within this context, the observation of targets at sea, e.g.; man-made targets (ships or oil/gas rigs/platforms and wind turbines) and natural targets (icebergs, surfactants, etc.) is nowadays a very hot-topic in the field of global monitoring of environment and security

Deep Learning for Detecting and Classifying Ocean Objects:Application of YoloV3 for Iceberg–Ship Discrimination

Synthetic aperture radar (SAR) plays a remarkable role in ocean surveillance, with capabilities of detecting oil spills, icebergs, and marine traffic both at daytime and at night, regardless of clouds and extreme weather conditions. The detection of ocean objects using SAR relies on well-established methods, mostly adaptive thresholding algorithms. In most waters, the dominant ocean objects are ships, whereas in arctic waters the vast majority of objects are icebergs drifting in the ocean and can be mistaken for ships in terms of navigation and ocean surveillance. Since these objects can look very much alike in SAR images, the determination of what objects actually are still relies on manual detection and human interpretation. With the increasing interest in the arctic regions for marine transportation, it is crucial to develop novel approaches for automatic monitoring of the traffic in these waters with satellite data. Hence, this study aims at proposing a deep learning model based on YoloV3 for discriminating icebergs and ships, which could be used for mapping ocean objects ahead of a journey. Using dual-polarization Sentinel-1 data, we pilot-tested our approach on a case study in Greenland. Our findings reveal that our approach is capable of training a deep learning model with reliable detection accuracy. Our methodical approach along with the choice of data and classifiers can be of great importance to climate change researchers, shipping industries and biodiversity analysts. The main difficulties were faced in the creation of training data in the Arctic waters and we concluded that future work must focus on issues regarding training data

Space-based Global Maritime Surveillance. Part I: Satellite Technologies

Maritime surveillance (MS) is crucial for search and rescue operations, fishery monitoring, pollution control, law enforcement, migration monitoring, and national security policies. Since the early days of seafaring, MS has been a critical task for providing security in human coexistence. Several generations of sensors providing detailed maritime information have become available for large offshore areas in real time: maritime radar sensors in the 1950s and the automatic identification system (AIS) in the 1990s among them. However, ground-based maritime radars and AIS data do not always provide a comprehensive and seamless coverage of the entire maritime space. Therefore, the exploitation of space-based sensor technologies installed on satellites orbiting around the Earth, such as satellite AIS data, synthetic aperture radar, optical sensors, and global navigation satellite systems reflectometry, becomes crucial for MS and to complement the existing terrestrial technologies. In the first part of this work, we provide an overview of the main available space-based sensors technologies and present the advantages and limitations of each technology in the scope of MS. The second part, related to artificial intelligence, signal processing and data fusion techniques, is provided in a companion paper, titled: "Space-based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques" [1].Comment: This paper has been submitted to IEEE Aerospace and Electronic Systems Magazin

훈련 자료 자동 추출 알고리즘과 기계 학습을 통한 SAR 영상 기반의 선박 탐지

학위논문 (석사) -- 서울대학교 대학원 : 자연과학대학 지구환경과학부, 2021. 2. 김덕진.Detection and surveillance of vessels are regarded as a crucial application of SAR for their contribution to the preservation of marine resources and the assurance on maritime safety. Introduction of machine learning to vessel detection significantly enhanced the performance and efficiency of the detection, but a substantial majority of studies focused on modifying the object detector algorithm. As the fundamental enhancement of the detection performance would be nearly impossible without accurate training data of vessels, this study implemented AIS information containing real-time information of vessel’s movement in order to propose a robust algorithm which acquires the training data of vessels in an automated manner. As AIS information was irregularly and discretely obtained, the exact target interpolation time for each vessel was precisely determined, followed by the implementation of Kalman filter, which mitigates the measurement error of AIS sensor. In addition, as the velocity of each vessel renders an imprint inside the SAR image named as Doppler frequency shift, it was calibrated by restoring the elliptic satellite orbit from the satellite state vector and estimating the distance between the satellite and the target vessel. From the calibrated position of the AIS sensor inside the corresponding SAR image, training data was directly obtained via internal allocation of the AIS sensor in each vessel. For fishing boats, separate information system named as VPASS was applied for the identical procedure of training data retrieval. Training data of vessels obtained via the automated training data procurement algorithm was evaluated by a conventional object detector, for three detection evaluating parameters: precision, recall and F1 score. All three evaluation parameters from the proposed training data acquisition significantly exceeded that from the manual acquisition. The major difference between two training datasets was demonstrated in the inshore regions and in the vicinity of strong scattering vessels in which land artifacts, ships and the ghost signals derived from them were indiscernible by visual inspection. This study additionally introduced a possibility of resolving the unclassified usage of each vessel by comparing AIS information with the accurate vessel detection results.전천후 지구 관측 위성인 SAR를 통한 선박 탐지는 해양 자원의 확보와 해상 안전 보장에 매우 중요한 역할을 한다. 기계 학습 기법의 도입으로 인해 선박을 비롯한 사물 탐지의 정확도 및 효율성이 향상되었으나, 이와 관련된 다수의 연구는 탐지 알고리즘의 개량에 집중되었다. 그러나, 탐지 정확도의 근본적인 향상은 정밀하게 취득된 대량의 훈련자료 없이는 불가능하기에, 본 연구에서는 선박의 실시간 위치, 속도 정보인 AIS 자료를 이용하여 인공 지능 기반의 선박 탐지 알고리즘에 사용될 훈련자료를 자동적으로 취득하는 알고리즘을 제안하였다. 이를 위해 이산적인 AIS 자료를 SAR 영상의 취득시각에 맞추어 정확하게 보간하고, AIS 센서 자체가 가지는 오차를 최소화하였다. 또한, 이동하는 산란체의 시선 속도로 인해 발생하는 도플러 편이 효과를 보정하기 위해 SAR 위성의 상태 벡터를 이용하여 위성과 산란체 사이의 거리를 정밀하게 계산하였다. 이렇게 계산된 AIS 센서의 영상 내의 위치로부터 선박 내 AIS 센서의 배치를 고려하여 선박 탐지 알고리즘의 훈련자료 형식에 맞추어 훈련자료를 취득하고, 어선에 대한 위치, 속도 정보인 VPASS 자료 역시 유사한 방법으로 가공하여 훈련자료를 취득하였다. AIS 자료로부터 취득한 훈련자료는 기존 방법대로 수동 취득한 훈련자료와 함께 인공 지능 기반 사물 탐지 알고리즘을 통해 정확도를 평가하였다. 그 결과, 제시된 알고리즘으로 취득한 훈련 자료는 수동 취득한 훈련 자료 대비 더 높은 탐지 정확도를 보였으며, 이는 기존의 사물 탐지 알고리즘의 평가 지표인 정밀도, 재현율과 F1 score를 통해 진행되었다. 본 연구에서 제안한 훈련자료 자동 취득 기법으로 얻은 선박에 대한 훈련자료는 특히 기존의 선박 탐지 기법으로는 분별이 어려웠던 항만에 인접한 선박과 산란체 주변의 신호에 대한 정확한 분별 결과를 보였다. 본 연구에서는 이와 함께, 선박 탐지 결과와 해당 지역에 대한 AIS 및 VPASS 자료를 이용하여 선박의 미식별성을 판정할 수 있는 가능성 또한 제시하였다.Chapter 1. Introduction - 1 - 1.1 Research Background - 1 - 1.2 Research Objective - 8 - Chapter 2. Data Acquisition - 10 - 2.1 Acquisition of SAR Image Data - 10 - 2.2 Acquisition of AIS and VPASS Information - 20 - Chapter 3. Methodology on Training Data Procurement - 26 - 3.1 Interpolation of Discrete AIS Data - 29 - 3.1.1 Estimation of Target Interpolation Time for Vessels - 29 - 3.1.2 Application of Kalman Filter to AIS Data - 34 - 3.2 Doppler Frequency Shift Correction - 40 - 3.2.1 Theoretical Basis of Doppler Frequency Shift - 40 - 3.2.2 Mitigation of Doppler Frequency Shift - 48 - 3.3 Retrieval of Training Data of Vessels - 53 - 3.4 Algorithm on Vessel Training Data Acquisition from VPASS Information - 61 - Chapter 4. Methodology on Object Detection Architecture - 66 - Chapter 5. Results - 74 - 5.1 Assessment on Training Data - 74 - 5.2 Assessment on AIS-based Ship Detection - 79 - 5.3 Assessment on VPASS-based Fishing Boat Detection - 91 - Chapter 6. Discussions - 110 - 6.1 Discussion on AIS-Based Ship Detection - 110 - 6.2 Application on Determining Unclassified Vessels - 116 - Chapter 7. Conclusion - 125 - 국문 요약문 - 128 - Bibliography - 130 -Maste

Oil spill and ship detection using high resolution polarimetric X-band SAR data

Among illegal human activities, marine pollution and target detection are the key concern of Maritime Security and Safety. This thesis deals with oil spill and ship detection using high resolution X-band polarimetric SAR (PolSAR). Polarimetry aims at analysing the polarization state of a wave field, in order to obtain physical information from the observed object. In this dissertation PolSAR techniques are suggested as improvement of the current State-of-the-Art of SAR marine pollution and target detection, by examining in depth Near Real Time suitability

A comparative study of operational vessel detectors for maritime surveillance using satellite-borne synthetic aperture radar

This paper presents a comparative study among four operational detectors that work by automatically post-processing synthetic aperture radar (SAR) images acquired from the satellite platforms RADARSAT-2 and COSMO-SkyMed. Challenging maritime scenarios have been chosen to assess the detectors' performance against features such as ambiguities, significant sea clutter, or irregular shorelines. The SAR images which form the test data are complemented with ground truth to define the reference detection configuration, which permits quantifying the probability of detection, the false alarm rate, and the accuracy of estimating ship dimensions. Although the results show that all the detectors perform well, there is no perfect detector, and a better detection system could be developed that combines the best elements from each of the single detectors. In addition to the comparison exercise, the study has facilitated the improvement of the detectors by highlighting weaknesses and providing means for fixing them.Peer ReviewedPostprint (published version

Selection of the key earth observation sensors and platforms focusing on applications for Polar Regions in the scope of Copernicus system 2020-2030

An optimal payload selection conducted in the frame of the H2020 ONION project (id 687490) is presented based on the ability to cover the observation needs of the Copernicus system in the time period 2020–2030. Payload selection is constrained by the variables that can be measured, the power consumption, and weight of the instrument, and the required accuracy and spatial resolution (horizontal or vertical). It involved 20 measurements with observation gaps according to the user requirements that were detected in the top 10 use cases in the scope of Copernicus space infrastructure, 9 potential applied technologies, and 39 available commercial platforms. Additional Earth Observation (EO) infrastructures are proposed to reduce measurements gaps, based on a weighting system that assigned high relevance for measurements associated to Marine for Weather Forecast over Polar Regions. This study concludes with a rank and mapping of the potential technologies and the suitable commercial platforms to cover most of the requirements of the top ten use cases, analyzing the Marine for Weather Forecast, Sea Ice Monitoring, Fishing Pressure, and Agriculture and Forestry: Hydric stress as the priority use cases.Peer ReviewedPostprint (published version

SHIP DETECTION USING SENTINEL-1 SAR DATA

The Earth’s surface is covered with 72% water. This fact alone emphasizes the importance of proper monitoring and regulation of maritime activities. This monitoring can be useful in an array of applications including illegal transitions, rescue operations, territory regulation among many other applications. In order to achieve the task of “Maritime Surveillance” or simply the marine object detection, we need a structured approach combined with a set of algorithms. The objective of this paper is to study an emerging open source tool- Search for Unidentified Maritime Objects (SUMO) developed for the detection of ships which work regardless of weather conditions and coverage limits. Based on the Synthetic Aperture Radar (SAR) data, this paper aims to process the satellite-borne data provided by the Sentinel-1 satellite. Proposed by the Joint Research Centre, SUMO is a pixel-based algorithm which follows a structured approach in order to identify marine objects and remove false alarms. It is observed that many of the false alarms are caused due to the presence of land. These are reduced by using the buffered coastlines referred to as land masks. A local threshold is calculated using the background clutter for the generation of false alarm rate and the pixels above this threshold are identified and clustered to form targets. A reliability value is computed for the elimination of azimuth ambiguities. Also, various attributes of the detected targets are calculated in order to give an accurate description of ships and its characteristics. With the SAR data being freely available due to the open data policy of the EU’s Copernicus program, it has never been more viable to employ new methods for marine object detection and this paper explores this possibility by analyzing the results obtained. Specifically, the employed data consists of Sentinel-1 fine dual-pol acquisitions over the coastal regions of India

Gaps analysis and requirements specification for the evolution of Copernicus system for polar regions monitoring: addressing the challenges in the horizon 2020-2030

This work was developed as part of the European H2020 ONION (Operational Network of Individual Observation Nodes) project, aiming at identifying the technological opportunity areas to complement the Copernicus space infrastructure in the horizon 2020–2030 for polar region monitoring. The European Earth Observation (EO) infrastructure is assessed through of comprehensive end-user need and data gap analysis. This review was based on the top 10 use cases, identifying 20 measurements with gaps and 13 potential EO technologies to cover the identified gaps. It was found that the top priority is the observation of polar regions to support sustainable and safe commercial activities and the preservation of the environment. Additionally, an analysis of the technological limitations based on measurement requirements was performed. Finally, this analysis was used for the basis of the architecture design of a potential polar mission.Peer ReviewedPostprint (published version