DVB-S based passive polarimetric ISAR – methods and experimental validation

In this work, we focus on passive polarimetric ISAR for ship target imaging using DVB-S signals of opportunity. A first goal of the research is to investigate if, within the challenging passive environment, different scattering mechanisms, belonging to distinct parts of the imaged target, can be separated in the polarimetric domain. Furthermore, a second goal is at verifying if polarimetric diversity could enable the formation of ISAR products with enhanced quality with respect to the single channel case, particularly in terms of better reconstruction of the target shape. To this purpose, a dedicated trial has been conducted along the river Rhine in Germany by means of an experimental DVB-S based system developed at Fraunhofer FHR and considering a ferry as cooperative target. To avoid inaccuracies due to data-driven motion compensation procedures and to fairly interpret the polarimetric results, we processed the data by means of a known-motion back-projection algorithm obtaining ISAR images at each polarimetric channel. Then, different approaches in the polarimetric domain have been introduced. The first one is based on the well-known Pauli Decomposition. The others can be divided in two main groups: (i) techniques aimed at separating the different backscattering mechanisms, and (ii) image domain techniques to fuse the polarimetric information in a single ISAR image with enhanced quality. The different considered techniques have been applied to several data sets with distinct bistatic geometries. The obtained results clearly demonstrate the potentialities of polarimetric diversity that could be fruitfully exploited for classification purposes

훈련 자료 자동 추출 알고리즘과 기계 학습을 통한 SAR 영상 기반의 선박 탐지

학위논문 (석사) -- 서울대학교 대학원 : 자연과학대학 지구환경과학부, 2021. 2. 김덕진.Detection and surveillance of vessels are regarded as a crucial application of SAR for their contribution to the preservation of marine resources and the assurance on maritime safety. Introduction of machine learning to vessel detection significantly enhanced the performance and efficiency of the detection, but a substantial majority of studies focused on modifying the object detector algorithm. As the fundamental enhancement of the detection performance would be nearly impossible without accurate training data of vessels, this study implemented AIS information containing real-time information of vessel’s movement in order to propose a robust algorithm which acquires the training data of vessels in an automated manner. As AIS information was irregularly and discretely obtained, the exact target interpolation time for each vessel was precisely determined, followed by the implementation of Kalman filter, which mitigates the measurement error of AIS sensor. In addition, as the velocity of each vessel renders an imprint inside the SAR image named as Doppler frequency shift, it was calibrated by restoring the elliptic satellite orbit from the satellite state vector and estimating the distance between the satellite and the target vessel. From the calibrated position of the AIS sensor inside the corresponding SAR image, training data was directly obtained via internal allocation of the AIS sensor in each vessel. For fishing boats, separate information system named as VPASS was applied for the identical procedure of training data retrieval. Training data of vessels obtained via the automated training data procurement algorithm was evaluated by a conventional object detector, for three detection evaluating parameters: precision, recall and F1 score. All three evaluation parameters from the proposed training data acquisition significantly exceeded that from the manual acquisition. The major difference between two training datasets was demonstrated in the inshore regions and in the vicinity of strong scattering vessels in which land artifacts, ships and the ghost signals derived from them were indiscernible by visual inspection. This study additionally introduced a possibility of resolving the unclassified usage of each vessel by comparing AIS information with the accurate vessel detection results.전천후 지구 관측 위성인 SAR를 통한 선박 탐지는 해양 자원의 확보와 해상 안전 보장에 매우 중요한 역할을 한다. 기계 학습 기법의 도입으로 인해 선박을 비롯한 사물 탐지의 정확도 및 효율성이 향상되었으나, 이와 관련된 다수의 연구는 탐지 알고리즘의 개량에 집중되었다. 그러나, 탐지 정확도의 근본적인 향상은 정밀하게 취득된 대량의 훈련자료 없이는 불가능하기에, 본 연구에서는 선박의 실시간 위치, 속도 정보인 AIS 자료를 이용하여 인공 지능 기반의 선박 탐지 알고리즘에 사용될 훈련자료를 자동적으로 취득하는 알고리즘을 제안하였다. 이를 위해 이산적인 AIS 자료를 SAR 영상의 취득시각에 맞추어 정확하게 보간하고, AIS 센서 자체가 가지는 오차를 최소화하였다. 또한, 이동하는 산란체의 시선 속도로 인해 발생하는 도플러 편이 효과를 보정하기 위해 SAR 위성의 상태 벡터를 이용하여 위성과 산란체 사이의 거리를 정밀하게 계산하였다. 이렇게 계산된 AIS 센서의 영상 내의 위치로부터 선박 내 AIS 센서의 배치를 고려하여 선박 탐지 알고리즘의 훈련자료 형식에 맞추어 훈련자료를 취득하고, 어선에 대한 위치, 속도 정보인 VPASS 자료 역시 유사한 방법으로 가공하여 훈련자료를 취득하였다. AIS 자료로부터 취득한 훈련자료는 기존 방법대로 수동 취득한 훈련자료와 함께 인공 지능 기반 사물 탐지 알고리즘을 통해 정확도를 평가하였다. 그 결과, 제시된 알고리즘으로 취득한 훈련 자료는 수동 취득한 훈련 자료 대비 더 높은 탐지 정확도를 보였으며, 이는 기존의 사물 탐지 알고리즘의 평가 지표인 정밀도, 재현율과 F1 score를 통해 진행되었다. 본 연구에서 제안한 훈련자료 자동 취득 기법으로 얻은 선박에 대한 훈련자료는 특히 기존의 선박 탐지 기법으로는 분별이 어려웠던 항만에 인접한 선박과 산란체 주변의 신호에 대한 정확한 분별 결과를 보였다. 본 연구에서는 이와 함께, 선박 탐지 결과와 해당 지역에 대한 AIS 및 VPASS 자료를 이용하여 선박의 미식별성을 판정할 수 있는 가능성 또한 제시하였다.Chapter 1. Introduction - 1 - 1.1 Research Background - 1 - 1.2 Research Objective - 8 - Chapter 2. Data Acquisition - 10 - 2.1 Acquisition of SAR Image Data - 10 - 2.2 Acquisition of AIS and VPASS Information - 20 - Chapter 3. Methodology on Training Data Procurement - 26 - 3.1 Interpolation of Discrete AIS Data - 29 - 3.1.1 Estimation of Target Interpolation Time for Vessels - 29 - 3.1.2 Application of Kalman Filter to AIS Data - 34 - 3.2 Doppler Frequency Shift Correction - 40 - 3.2.1 Theoretical Basis of Doppler Frequency Shift - 40 - 3.2.2 Mitigation of Doppler Frequency Shift - 48 - 3.3 Retrieval of Training Data of Vessels - 53 - 3.4 Algorithm on Vessel Training Data Acquisition from VPASS Information - 61 - Chapter 4. Methodology on Object Detection Architecture - 66 - Chapter 5. Results - 74 - 5.1 Assessment on Training Data - 74 - 5.2 Assessment on AIS-based Ship Detection - 79 - 5.3 Assessment on VPASS-based Fishing Boat Detection - 91 - Chapter 6. Discussions - 110 - 6.1 Discussion on AIS-Based Ship Detection - 110 - 6.2 Application on Determining Unclassified Vessels - 116 - Chapter 7. Conclusion - 125 - 국문 요약문 - 128 - Bibliography - 130 -Maste

The InflateSAR Campaign: Evaluating SAR Identification Capabilities of Distressed Refugee Boats

Most of the recent research in the field of marine target detection has been concentrating on ships with large metallic parts. The focus of this work is on much more challenging targets represented by small rubber inflatables. They are of importance, since in recent years they have largely been used by migrants to cross the Mediterranean Sea between Libya and Europe. The motivation of this research is to mitigate the ongoing humanitarian crisis at Europe’s southern borders. These boats, packed with up to 200 people, are in no way suitable to cross the Mediterranean Sea or any other big water body and are in distress from the moment of departure. The establishment of a satellite-based surveillance infrastructure could considerably support search and rescue missions in the Mediterranean Sea, reduce the number of such boats being missed and mitigate the ongoing death in the open ocean. In this work we describe and analyze data from the InflateSAR acquisition campaign, wherein we gathered multiple-platform SAR imagery of an original refugee inflatable. The test site for this campaign is a lake which provides background clutter that is more predictable. The analysis considered a sum of experiments, enabling investigations of a broad range of scene settings, such as the vessel’s orientation, superstructures and speed. We assess their impact on the detectability of the chosen target under different sensor parameters, such as polarimetry, resolution and incidence angle. Results show that TerraSAR-X Spotlight and Stripmap modes offer good capabilities to potentially detect those types of boats in distress. Low incidence angles and cross-polarization decrease the chance of a successful identification, whereas a fully occupied inflatable, orthogonally oriented to the line of sight, seems to be better visible than an empty one. The polarimetric analyses prove the vessel’s different polarimetric behavior in comparison with the water surface, especially when it comes to entropy. The analysis considered state-of-the-art methodologies with single polarization and dual polarization channels. Finally, different metrics are used to discuss whether and to which extent the results are applicable to other open ocean datasets. This paper does not introduce any vessel detection or classification algorithm from SAR images. Rather, its results aim at paving the way to the design and the development of a specially tailored detection algorithm for small rubber inflatables

Research on Ship Classification using Faster Region Convolutional Neural Network for Port Security

Huvudsyftet med studien var att se i vilken grad det går att finna samarbeten genom material- och/eller energiutbyten mellan närliggande anläggningar inom skogsindustrin i Sverige. Genom att göra en inventering av vilka anläggningar som finns inom skogsindustrin och sedan kontakta dessa, sammanställdes en lista över de olika anläggningarna och deras olika samarbeten. Inventeringen gjordes med hjälp av olika branschorganisationer samt sökmotorer på Internet. Utöver detta besöktes också fyra intressanta fall för att ge en inblick i hur dessa samarbeten kan se ut. Studien visar på att den här typen av samarbeten existerar inom skogsindustrin och att drygt en tredjedel av de studerade anläggningarna har någon form av samarbeten rörande dessa frågor. Detta pekar på att man inom skogsindustrin är långt framme när det gäller resursutnyttjande och att möjligheten att minimera sin energi- och materialanvändning hela tiden är en relevant fråga. Det finns med stor sannolikhet ännu fler sådana samarbeten som inte framkommit vid undersökningen och en intressant aspekt är att vid de besök som gjordes upptäcktes samarbeten som inte uppmärksammats vid tidigare kontakter. Av de 152 tillfrågade anläggningarna i inventeringen erhölls svar från 117 stycken vilket tyder på att det finns ett stort intresse för dessa frågor inom skogsindustrin. Flera av de anläggningar som inte hade några samarbeten kring dessa frågor svarade också att de hela tiden undersöker möjligheten till att inleda sådana. Många av samarbetena rörande dessa frågor kretsar kring leveranser av el och ånga samt spån och flis men en del andra intressanta samarbeten har också framkommit. Exempelvis används slam från bioreningsdammar till bränsle, jordförbättringsmedel och som täckmaterial vid deponier. Sammanfattningsvis tyder detta på att skogsindustrin ligger långt framme gällande dessa frågor men att det fortfarande finns mer att göra om energi- och materialanvändningen och därigenom den negativa miljöpåverkan ska minimeras.The aim and objective with this study was to investigate to what extent co-operation through material and energy exchange between adjacent industries among the forest industry in Sweden could be found. First, an inventory of the industries in the forest industry was conducted. Secondly, each company was contacted with questions concerning this issue. Complementary field studies of four specific cases were conducted in order to give an insight to how these co-operations may function in reality. The result of this study illustrates that co-operations among the industries exist in the forest industry sector as more than a third of the investigated industries has some kind of co-operation regarding material and energy exchange with adjacent industries. A total number of 152 industries were identified during the inventory phase and 117 of those industries participated in the study with their own answers. This high participation rate enhances the impression that these are important questions to the forest industry sector. Numerous of the co-operations mentioned revolve around electricity, steam, and by products from sawmills, like woodchips and sawdust. Nevertheless, a few other interesting co-operations have also been revealed during the study, for example; sludge from some of the pulp mills are used as fuel, soil fertilizer and as covering material at landfills. An interesting point is that co-operations, which not were discovered during the earlier correspondence with the industries, in fact were revealed during the field studies. Therefore, the probability that there are more existing co-operations between adjacent industries than the findings in the study reveals, are high. To sum up, this shows that the forest industry is well in advance regarding co-operation through material and energy exchange between adjacent industries. However, there is still a lot to be done if the negative effect on the environment from the forest industry should be minimised

Space Remote Sensing and Detecting Systems of Oceangoing Ships

This paper introduces the implementation of space remote sensing and detecting systems of oceangoing ships as an alternative to the Radio – Automatic Identification System (R-AIS), Satellite – Automatic Identification System (S-AIS), Long Range Identification and Tracking (LRIT), and other current vessel tracking systems. In this paper will be not included  a new project known as a Global Ship Tracking (GST) as an autonomous and discrete satellite network designed by the Space Science Centre (SSC) for research and postgraduate studies in Satellite Communication, Navigation and Surveillance (CNS) at Durban University of Technology (DUT). The ship detection from satellite remote sensing imagery system is a crucial application for maritime safety and security, which includes among others ship tracking, detecting and traffic surveillance, oil spill detection service, and discharge control, sea pollution monitoring, sea ice monitoring service, and protection against illegal fisheries activities. The establishment of a modern sea surface and ships monitoring system needs enhancement of the Satellite Synthetic Aperture Radar (SSAR) that is here discussed as a modern observation infrastructure integrated with Ships Surveillance and Detecting via SSAR TerraSAR-X Spacecraft, Ships Surveillance and Detecting via SSAR Radarsat Spacecraft and Vessels Detecting System (VDS) via SSAR

Ellipse Encoding for Arbitrary-Oriented SAR Ship Detection Based on Dynamic Key Points

In recent years, there has been growing interest in developing oriented bounding box (OBB)-based deep learning approaches to detect arbitrary-oriented ship targets in synthetic aperture radar (SAR) images. However, most existing OBB-based detection methods suffer from boundary discontinuity problems for bounding box angle prediction and key point regression challenges. In this article, we present a novel OBB-based detection algorithm that utilizes ellipse encoding to effectively exploit the geometric and scattering properties of ship targets. Specifically, the ship contour is fit by an OBB inscribed ellipse that is encoded as a set of distances between dynamic key points on the bow and target center. By combining the bow angle interval and the decoding process, the negative impact of the boundary discontinuity problem is avoided. In addition, we propose an elliptical Gaussian distribution heatmap and a pooling strategy termed double-peak max-pooling (DPM) to deal with the challenge of separating densely distributed ships in inshore scenes. The former can enhance the heatmap’s ship-side score gap between neighboring ship targets, while the latter can solve the problem of target center responses being suppressed after max-pooling. Simulation experiments conducted on the benchmark Rotating SAR Ship Detection Dataset (RSSDD) and Rotated Ship Detection Dataset in SAR Images (RSDD-SAR) demonstrate the superior performance of our method for ship target detection compared to several state-of-the-art OBB-based algorithms. Ablation experiments show that elliptical Gaussian distribution heatmap and DPM can further improve the inshore detection performance

Ellipse Encoding for Arbitrary-Oriented SAR Ship Detection Based on Dynamic Key Points

In recent years, there has been growing interest in developing oriented bounding box (OBB)-based deep learning approaches to detect arbitrary-oriented ship targets in synthetic aperture radar (SAR) images. However, most existing OBB-based detection methods suffer from boundary discontinuity problems for bounding box angle prediction and key point regression challenges. In this article, we present a novel OBB-based detection algorithm that utilizes ellipse encoding to effectively exploit the geometric and scattering properties of ship targets. Specifically, the ship contour is fit by an OBB inscribed ellipse that is encoded as a set of distances between dynamic key points on the bow and target center. By combining the bow angle interval and the decoding process, the negative impact of the boundary discontinuity problem is avoided. In addition, we propose an elliptical Gaussian distribution heatmap and a pooling strategy termed double-peak max-pooling (DPM) to deal with the challenge of separating densely distributed ships in inshore scenes. The former can enhance the heatmap’s ship-side score gap between neighboring ship targets, while the latter can solve the problem of target center responses being suppressed after max-pooling. Simulation experiments conducted on the benchmark Rotating SAR Ship Detection Dataset (RSSDD) and Rotated Ship Detection Dataset in SAR Images (RSDD-SAR) demonstrate the superior performance of our method for ship target detection compared to several state-of-the-art OBB-based algorithms. Ablation experiments show that elliptical Gaussian distribution heatmap and DPM can further improve the inshore detection performance

The InflateSAR Campaign: Testing SAR Vessel Detection Systems for Refugee Rubber Inflatables

Countless numbers of people lost their lives at Europe’s southern borders in recent years in the attempt to cross to Europe in small rubber inflatables. This work examines satellite-based approaches to build up future systems that can automatically detect those boats. We compare the performance of several automatic vessel detectors using real synthetic aperture radar (SAR) data from X-band and C-band sensors on TerraSAR-X and Sentinel-1. The data was collected in an experimental campaign where an empty boat lies on a lake’s surface to analyse the influence of main sensor parameters (incidence angle, polarization mode, spatial resolution) on the detectability of our inflatable. All detectors are implemented with a moving window and use local clutter statistics from the adjacent water surface. Among tested detectors are well-known intensity-based (CA-CFAR), sublook-based (sublook correlation) and polarimetric-based (PWF, PMF, PNF, entropy, symmetry and iDPolRAD) approaches. Additionally, we introduced a new version of the volume detecting iDPolRAD aimed at detecting surface anomalies and compare two approaches to combine the volume and the surface in one algorithm, producing two new highly performing detectors. The results are compared with receiver operating characteristic (ROC) curves, enabling us to compare detectors independently of threshold selection