AIS-based Evaluation of Target Detectors and SAR Sensors Characteristics for Maritime Surveillance

International audienceThis paper studies the performances of different ship detectors based on adaptive threshold algorithms. The detec- tion algorithms are based on various clutter distributions and assessed automatically with a systematic methodology. Evaluation using large datasets of medium resolution SAR images and AIS (Automatic Identification System) data as ground truths allows to evaluate the efficiency of each detector. Depending on the datasets used for testing, the detection algorithms offer different advantages and disadvantages. The systematic method used in discriminating real detected targets and false alarms in order to determine the detection rate, allows us to perform an appropriate and consistent comparison of the detectors. The impact of SAR sensors characteristics (incidence angle, polarization, frequency and spatial resolution) is fully assessed, the vessels' length being also considered. Experiments are conducted on Radarsat-2 and CosmoSkymed ScanSAR datasets and AIS data acquired by coastal stations

A comparative study of operational vessel detectors for maritime surveillance using satellite-borne synthetic aperture radar

This paper presents a comparative study among four operational detectors that work by automatically post-processing synthetic aperture radar (SAR) images acquired from the satellite platforms RADARSAT-2 and COSMO-SkyMed. Challenging maritime scenarios have been chosen to assess the detectors' performance against features such as ambiguities, significant sea clutter, or irregular shorelines. The SAR images which form the test data are complemented with ground truth to define the reference detection configuration, which permits quantifying the probability of detection, the false alarm rate, and the accuracy of estimating ship dimensions. Although the results show that all the detectors perform well, there is no perfect detector, and a better detection system could be developed that combines the best elements from each of the single detectors. In addition to the comparison exercise, the study has facilitated the improvement of the detectors by highlighting weaknesses and providing means for fixing them.Peer ReviewedPostprint (published version

Space-based Global Maritime Surveillance. Part I: Satellite Technologies

Maritime surveillance (MS) is crucial for search and rescue operations, fishery monitoring, pollution control, law enforcement, migration monitoring, and national security policies. Since the early days of seafaring, MS has been a critical task for providing security in human coexistence. Several generations of sensors providing detailed maritime information have become available for large offshore areas in real time: maritime radar sensors in the 1950s and the automatic identification system (AIS) in the 1990s among them. However, ground-based maritime radars and AIS data do not always provide a comprehensive and seamless coverage of the entire maritime space. Therefore, the exploitation of space-based sensor technologies installed on satellites orbiting around the Earth, such as satellite AIS data, synthetic aperture radar, optical sensors, and global navigation satellite systems reflectometry, becomes crucial for MS and to complement the existing terrestrial technologies. In the first part of this work, we provide an overview of the main available space-based sensors technologies and present the advantages and limitations of each technology in the scope of MS. The second part, related to artificial intelligence, signal processing and data fusion techniques, is provided in a companion paper, titled: "Space-based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques" [1].Comment: This paper has been submitted to IEEE Aerospace and Electronic Systems Magazin

훈련 자료 자동 추출 알고리즘과 기계 학습을 통한 SAR 영상 기반의 선박 탐지

학위논문 (석사) -- 서울대학교 대학원 : 자연과학대학 지구환경과학부, 2021. 2. 김덕진.Detection and surveillance of vessels are regarded as a crucial application of SAR for their contribution to the preservation of marine resources and the assurance on maritime safety. Introduction of machine learning to vessel detection significantly enhanced the performance and efficiency of the detection, but a substantial majority of studies focused on modifying the object detector algorithm. As the fundamental enhancement of the detection performance would be nearly impossible without accurate training data of vessels, this study implemented AIS information containing real-time information of vessel’s movement in order to propose a robust algorithm which acquires the training data of vessels in an automated manner. As AIS information was irregularly and discretely obtained, the exact target interpolation time for each vessel was precisely determined, followed by the implementation of Kalman filter, which mitigates the measurement error of AIS sensor. In addition, as the velocity of each vessel renders an imprint inside the SAR image named as Doppler frequency shift, it was calibrated by restoring the elliptic satellite orbit from the satellite state vector and estimating the distance between the satellite and the target vessel. From the calibrated position of the AIS sensor inside the corresponding SAR image, training data was directly obtained via internal allocation of the AIS sensor in each vessel. For fishing boats, separate information system named as VPASS was applied for the identical procedure of training data retrieval. Training data of vessels obtained via the automated training data procurement algorithm was evaluated by a conventional object detector, for three detection evaluating parameters: precision, recall and F1 score. All three evaluation parameters from the proposed training data acquisition significantly exceeded that from the manual acquisition. The major difference between two training datasets was demonstrated in the inshore regions and in the vicinity of strong scattering vessels in which land artifacts, ships and the ghost signals derived from them were indiscernible by visual inspection. This study additionally introduced a possibility of resolving the unclassified usage of each vessel by comparing AIS information with the accurate vessel detection results.전천후 지구 관측 위성인 SAR를 통한 선박 탐지는 해양 자원의 확보와 해상 안전 보장에 매우 중요한 역할을 한다. 기계 학습 기법의 도입으로 인해 선박을 비롯한 사물 탐지의 정확도 및 효율성이 향상되었으나, 이와 관련된 다수의 연구는 탐지 알고리즘의 개량에 집중되었다. 그러나, 탐지 정확도의 근본적인 향상은 정밀하게 취득된 대량의 훈련자료 없이는 불가능하기에, 본 연구에서는 선박의 실시간 위치, 속도 정보인 AIS 자료를 이용하여 인공 지능 기반의 선박 탐지 알고리즘에 사용될 훈련자료를 자동적으로 취득하는 알고리즘을 제안하였다. 이를 위해 이산적인 AIS 자료를 SAR 영상의 취득시각에 맞추어 정확하게 보간하고, AIS 센서 자체가 가지는 오차를 최소화하였다. 또한, 이동하는 산란체의 시선 속도로 인해 발생하는 도플러 편이 효과를 보정하기 위해 SAR 위성의 상태 벡터를 이용하여 위성과 산란체 사이의 거리를 정밀하게 계산하였다. 이렇게 계산된 AIS 센서의 영상 내의 위치로부터 선박 내 AIS 센서의 배치를 고려하여 선박 탐지 알고리즘의 훈련자료 형식에 맞추어 훈련자료를 취득하고, 어선에 대한 위치, 속도 정보인 VPASS 자료 역시 유사한 방법으로 가공하여 훈련자료를 취득하였다. AIS 자료로부터 취득한 훈련자료는 기존 방법대로 수동 취득한 훈련자료와 함께 인공 지능 기반 사물 탐지 알고리즘을 통해 정확도를 평가하였다. 그 결과, 제시된 알고리즘으로 취득한 훈련 자료는 수동 취득한 훈련 자료 대비 더 높은 탐지 정확도를 보였으며, 이는 기존의 사물 탐지 알고리즘의 평가 지표인 정밀도, 재현율과 F1 score를 통해 진행되었다. 본 연구에서 제안한 훈련자료 자동 취득 기법으로 얻은 선박에 대한 훈련자료는 특히 기존의 선박 탐지 기법으로는 분별이 어려웠던 항만에 인접한 선박과 산란체 주변의 신호에 대한 정확한 분별 결과를 보였다. 본 연구에서는 이와 함께, 선박 탐지 결과와 해당 지역에 대한 AIS 및 VPASS 자료를 이용하여 선박의 미식별성을 판정할 수 있는 가능성 또한 제시하였다.Chapter 1. Introduction - 1 - 1.1 Research Background - 1 - 1.2 Research Objective - 8 - Chapter 2. Data Acquisition - 10 - 2.1 Acquisition of SAR Image Data - 10 - 2.2 Acquisition of AIS and VPASS Information - 20 - Chapter 3. Methodology on Training Data Procurement - 26 - 3.1 Interpolation of Discrete AIS Data - 29 - 3.1.1 Estimation of Target Interpolation Time for Vessels - 29 - 3.1.2 Application of Kalman Filter to AIS Data - 34 - 3.2 Doppler Frequency Shift Correction - 40 - 3.2.1 Theoretical Basis of Doppler Frequency Shift - 40 - 3.2.2 Mitigation of Doppler Frequency Shift - 48 - 3.3 Retrieval of Training Data of Vessels - 53 - 3.4 Algorithm on Vessel Training Data Acquisition from VPASS Information - 61 - Chapter 4. Methodology on Object Detection Architecture - 66 - Chapter 5. Results - 74 - 5.1 Assessment on Training Data - 74 - 5.2 Assessment on AIS-based Ship Detection - 79 - 5.3 Assessment on VPASS-based Fishing Boat Detection - 91 - Chapter 6. Discussions - 110 - 6.1 Discussion on AIS-Based Ship Detection - 110 - 6.2 Application on Determining Unclassified Vessels - 116 - Chapter 7. Conclusion - 125 - 국문 요약문 - 128 - Bibliography - 130 -Maste

Sea target detection using spaceborne GNSS-R delay-doppler maps: theory and experimental proof of concept using TDS-1 data

© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This study addresses a novel application of global navigation satellite system-reflectometry (GNSS-R) delay-Doppler maps (DDMs), namely sea target detection. In contrast with other competing remote sensing technologies, such as synthetic aperture radar and optical systems, typically exploited in the field of sea target detection, GNSS-R systems could be employed as satellite constellations, so as to fulfill the temporal requirements for near real-time ships and sea ice sheets monitoring. In this study, the revisit time offered by GNSS-R systems is quantitatively evaluated by means of a simulation analysis, in which three different realistic GNSS-R missions are simulated and analyzed. Then, a sea target detection algorithm from spaceborne GNSS-R DDMs is described and assessed. The algorithm is based on a sea clutter compensation step and uses an adaptive threshold to take into account spatial variations in the sea background and/or noise statistics. Finally, the sea target detector algorithm is tested and validated for the first time ever using experimental GNSS-R data from the U.K. TechDemoSat-1 dataset. Performance is assessed by providing the receiver operating characteristic curves, and some preliminary experimental results are presented.Peer ReviewedPostprint (published version

Software Defined Multi-Spectral Imaging for Arctic Sensor Networks

Availability of off-the-shelf infrared sensors combined with high definition visible cameras has made possible the construction of a Software Defined Multi-Spectral Imager (SDMSI) combining long-wave, near-infrared and visible imaging. The SDMSI requires a real-time embedded processor to fuse images and to create real-time depth maps for opportunistic uplink in sensor networks. Researchers at Embry Riddle Aeronautical University working with University of Alaska Anchorage at the Arctic Domain Awareness Center and the University of Colorado Boulder have built several versions of a low-cost drop-in-place SDMSI to test alternatives for power efficient image fusion. The SDMSI is intended for use in field applications including marine security, search and rescue operations and environmental surveys in the Arctic region. Based on Arctic marine sensor network mission goals, the team has designed the SDMSI to include features to rank images based on saliency and to provide on camera fusion and depth mapping. A major challenge has been the design of the camera computing system to operate within a 10 to 20 Watt power budget. This paper presents a power analysis of three options: 1) multi-core, 2) field programmable gate array with multi-core, and 3) graphics processing units with multi-core. For each test, power consumed for common fusion workloads has been measured at a range of frame rates and resolutions. Detailed analyses from our power efficiency comparison for workloads specific to stereo depth mapping and sensor fusion are summarized. Preliminary mission feasibility results from testing with off-the-shelf long-wave infrared and visible cameras in Alaska and Arizona are also summarized to demonstrate the value of the SDMSI for applications such as ice tracking, ocean color, soil moisture, animal and marine vessel detection and tracking. The goal is to select the most power efficient solution for the SDMSI for use on UAVs (Unoccupied Aerial Vehicles) and other drop-in-place installations in the Arctic. The prototype selected will be field tested in Alaska in the summer of 2016

GNSS-based passive radar techniques for maritime surveillance

The improvement of maritime traffic safety and security is a subject of growing interest, since the traffic is constantly increasing. In fact, a large number of human activities take place in maritime domain, varying from cruise and trading ships up to vessels involved in nefarious activities such as piracy, human smuggling or terrorist actions. The systems based on Automatic Identification System (AIS) transponder cannot cope with non-cooperative or non-equipped vessels that instead can be detected, tracked and identified by means of radar system. In particular, passive bistatic radar (PBR) systems can perform these tasks without a dedicated transmitter, since they exploit illuminators of opportunity as transmitters. The lack of a dedicated transmitter makes such systems low cost and suitable to be employed in areas where active sensors cannot be placed such as, for example, marine protected areas. Innovative solutions based on terrestrial transmitters have been considered in order to increase maritime safety and security, but these kinds of sources cannot guarantee a global coverage, such as in open sea. To overcome this problem, the exploitation of global navigation satellites system (GNSS) as transmitters of opportunity is a prospective solution. The global, reliable and persistent nature of these sources makes them potentially able to guarantee the permanent monitoring of both coastal and open sea areas. To this aim, this thesis addresses the exploitation of Global Navigation Satellite Systems (GNSS) as transmitters of opportunity in passive bistatic radar (PBR) systems for maritime surveillance. The main limitation of this technology is the restricted power budget provided by navigation satellites, which makes it necessary to define innovative moving target detection techniques specifically tailored for the system under consideration. For this reason, this thesis puts forward long integration time techniques able to collect the signal energy over long time intervals (tens of seconds), allowing the retrieval of suitable levels of signal-to-disturbance ratios for detection purposes. The feasibility of this novel application is firstly investigated in a bistatic system configuration. A long integration time moving target detection technique working in bistatic range&Doppler plane is proposed and its effectiveness is proved against synthetic and experimental datasets. Subsequently the exploitation of multiple transmitters for the joint detection and localization of vessels at sea is also investigated. A single-stage approach to jointly detect and localize the ship targets by making use of long integration times (tens of seconds) and properly exploiting the spatial diversity offered by such a configuration is proposed. Furthermore, the potential of the system to extract information concerning the detected target characteristics for further target classification is assessed

Deep learning-based vessel detection from very high and medium resolution optical satellite images as component of maritime surveillance systems

This thesis presents an end-to-end multiclass vessel detection method from optical satellite images. The proposed workflow covers the complete processing chain and involves rapid image enhancement techniques, the fusion with automatic identification system (AIS) data, and the detection algorithm based on convolutional neural networks (CNN). The algorithms presented are implemented in the form of independent software processors and integrated in an automated processing chain as part of the Earth Observation Maritime Surveillance System (EO-MARISS).In der vorliegenden Arbeit wird eine Methode zur Detektion von Schiffen unterschiedlicher Klassen in optischen Satellitenbildern vorgestellt. Diese gliedert sich in drei aufeinanderfolgende Funktionen: i) die Bildbearbeitung zur Verbesserung der Bildeigenschaften, ii) die Datenfusion mit den Daten des Automatischen Identifikation Systems (AIS) und iii) dem auf „Convolutional Neural Network“ (CNN) basierenden Detektionsalgorithmus. Die vorgestellten Algorithmen wurden in Form eigenständiger Softwareprozessoren implementiert und als Teil des maritimen Erdbeobachtungssystems integriert

Oil spill and ship detection using high resolution polarimetric X-band SAR data

Among illegal human activities, marine pollution and target detection are the key concern of Maritime Security and Safety. This thesis deals with oil spill and ship detection using high resolution X-band polarimetric SAR (PolSAR). Polarimetry aims at analysing the polarization state of a wave field, in order to obtain physical information from the observed object. In this dissertation PolSAR techniques are suggested as improvement of the current State-of-the-Art of SAR marine pollution and target detection, by examining in depth Near Real Time suitability