A novel marine radar targets extraction approach based on sequential images and Bayesian Network

This research proposes a Bayesian Network-based methodology to extract moving vessels from a plethora of blips captured in frame-by-frame radar images. First, the inter-frame differences or graph characteristics of blips, such as velocity, direction, and shape, are quantified and selected as nodes to construct a Directed Acyclic Graph (DAG), which is used for reasoning the probability of a blip being a moving vessel. Particularly, an unequal-distance discretisation method is proposed to reduce the intervals of a blip’s characteristics for avoiding the combinatorial explosion problem. Then, the undetermined DAG structure and parameters are learned from manually verified data samples. Finally, based on the probabilities reasoned by the DAG, judgments on blips being moving vessels are determined by an appropriate threshold on a Receiver Operating Characteristic (ROC) curve. The unique strength of the proposed methodology includes laying the foundation of targets extraction on original radar images and verified records without making any unrealistic assumptions on objects' states. A real case study has been conducted to validate the effectiveness and accuracy of the proposed methodology

Deep Learning-Based Maritime Environment Segmentation for Unmanned Surface Vehicles Using Superpixel Algorithms

Unmanned surface vehicles (USVs) are receiving increasing attention in recent years from both academia and industry. To make a high-level autonomy for USVs, the environment situational awareness is a key capability. However, due to the richness of the features in marine environments, as well as the complexity of the environment influenced by sun glare and sea fog, the development of a reliable situational awareness system remains a challenging problem that requires further studies. This paper, therefore, proposes a new deep semantic segmentation model together with a Simple Linear Iterative Clustering (SLIC) algorithm, for an accurate perception for various maritime environments. More specifically, powered by the SLIC algorithm, the new segmentation model can achieve refined results around obstacle edges and improved accuracy for water surface obstacle segmentation. The overall structure of the new model employs an encoder–decoder layout, and a superpixel refinement is embedded before final outputs. Three publicly available maritime image datasets are used in this paper to train and validate the segmentation model. The final output demonstrates that the proposed model can provide accurate results for obstacle segmentation

Energy minimization with one dot fuzzy initialization for marine oil spill segmentation

Detecting marine oil spill regions in synthetic aperture radar (SAR) images has always been posed as a segmentation problem in terms of minimizing a certain energy function(al). As most energy minimization problems do not have analytical solutions, minimizing an energy function(al) is usually achieved in an iterative numerical manner. In this scenario, one key factor that affects the segmentation accuracy is the initialization for starting or constraining the numerical iterations. To guarantee accurate segmentation, a proper initialization that characterizes the marine oil spill layouts in a SAR image is required. However, marine oil spill regions are always complicatedly shaped, and it is inefficient to manually devise precise initializations for capturing various marine oil spill shapes. In order to address this problem and render efficient and robust segmentation, we develop a one dot fuzzy initialization strategy. In contrast to the normal practice of manually labeling a large amount of pixels (possibly lines or cycles of pixels subject to strict spatial conditions) as initialization, our strategy just requires one arbitrary pixel within a marine oil spill region as the initial dot. The intuition of our strategy is that the fuzzy connectedness between an arbitrary initial dot and the rest pixels enables the derivation of a physically homogeneous region which is consistent for initializing the energy minimization. In the light of this observation, we develop schemes for exploiting the one dot derived region to initialize both level sets for minimizing continuous energy functionals and graph cuts for minimizing discrete energy functions. Experimental results validate the robustness of our one dot fuzzy initialization strategy

Ship detection in SAR images based on Maxtree representation and graph signal processing

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper discusses an image processing architecture and tools to address the problem of ship detection in synthetic-aperture radar images. The detection strategy relies on a tree-based representation of images, here a Maxtree, and graph signal processing tools. Radiometric as well as geometric attributes are evaluated and associated with the Maxtree nodes. They form graph attribute signals which are processed with graph filters. The goal of this filtering step is to exploit the correlation existing between attribute values on neighboring tree nodes. Considering that trees are specific graphs where the connectivity toward ancestors and descendants may have a different meaning, we analyze several linear, nonlinear, and morphological filtering strategies. Beside graph filters, two new filtering notions emerge from this analysis: tree and branch filters. Finally, we discuss a ship detection architecture that involves graph signal filters and machine learning tools. This architecture demonstrates the interest of applying graph signal processing tools on the tree-based representation of images and of going beyond classical graph filters. The resulting approach significantly outperforms state-of-the-art algorithms. Finally, a MATLAB toolbox allowing users to experiment with the tools discussed in this paper on Maxtree or Mintree has been created and made public.Peer ReviewedPostprint (author's final draft

Target recognition for coastal surveillance based on radar images and generalised Bayesian inference

For coastal surveillance, this study proposes a novel approach to identify moving vessels from radar images with the use of a generalised Bayesian inference technique, namely the evidential reasoning (ER) rule. First of all, the likelihood information about radar blips is obtained in terms of the velocity, direction, and shape attributes of the verified samples. Then, it is transformed to be multiple pieces of evidence, which are formulated as generalised belief distributions representing the probabilistic relationships between the blip's states of authenticity and the values of its attributes. Subsequently, the ER rule is used to combine these pieces of evidence, taking into account their corresponding reliabilities and weights. Furthermore, based on different objectives and verified samples, weight coefficients can be trained with a non-linear optimisation model. Finally, two field tests of identifying moving vessels from radar images have been conducted to validate the effectiveness and flexibility of the proposed approach

Knowledge-Driven Semantic Segmentation for Waterway Scene Perception

Semantic segmentation as one of the most popular scene perception techniques has been studied for autonomous vehicles. However, deep learning-based solutions rely on the volume and quality of data and knowledge from specific scene might not be incorporated. A novel knowledge-driven semantic segmentation method is proposed for waterway scene perception. Based on the knowledge that water is irregular and dynamically changing, a Life Time of Feature (LToF) detector is designed to distinguish water region from surrounding scene. Using a Bayesian framework, the detector as the likelihood function is combined with U-Net based semantic segmentation to achieve an optimized solution. Finally, two public datasets and typical semantic segmentation networks, FlowNet, DeepLab and DVSNet are selected to evaluate the proposed method. Also, the sensitivity of these methods and ours to dataset is discussed

Reliable Navigational Scene Perception for Autonomous Ships in Maritime Environment

Due to significant advances in robotics and transportation, research on autonomous ships has attracted considerable attention. The most critical task is to make the ships capable of accurately, reliably, and intelligently detecting their surroundings to achieve high levels of autonomy. Three deep learning-based models are constructed in this thesis to perform complex perceptual tasks such as identifying ships, analysing encounter situations, and recognising water surface objects. In this thesis, sensors, including the Automatic Identification System (AIS) and cameras, provide critical information for scene perception. Specifically, the AIS enables mid-range and long-range detection, assisting the decision-making system to take suitable and decisive action. A Convolutional Neural Network-Ship Movement Modes Classification (CNN-SMMC) is used to detect ships or objects. Following that, a Semi- Supervised Convolutional Encoder-Decoder Network (SCEDN) is developed to classify ship encounter situations and make a collision avoidance plan for the moving ships or objects. Additionally, cameras are used to detect short-range objects, a supplementary solution to ships or objects not equipped with an AIS. A Water Obstacle Detection Network based on Image Segmentation (WODIS) is developed to find potential threat targets. A series of quantifiable experiments have demonstrated that these models can provide reliable scene perception for autonomous ships

SAR Ship Target Recognition via Selective Feature Discrimination and Multifeature Center Classifier

Maritime surveillance is not only necessary for every country, such as in maritime safeguarding and fishing controls, but also plays an essential role in international fields, such as in rescue support and illegal immigration control. Most of the existing automatic target recognition (ATR) methods directly send the extracted whole features of SAR ships into one classifier. The classifiers of most methods only assign one feature center to each class. However, the characteristics of SAR ship images, large inner-class variance, and small interclass difference lead to the whole features containing useless partial features and a single feature center for each class in the classifier failing with large inner-class variance. We proposes a SAR ship target recognition method via selective feature discrimination and multifeature center classifier. The selective feature discrimination automatically finds the similar partial features from the most similar interclass image pairs and the dissimilar partial features from the most dissimilar inner-class image pairs. It then provides a loss to enhance these partial features with more interclass separability. Motivated by divide and conquer, the multifeature center classifier assigns multiple learnable feature centers for each ship class. In this way, the multifeature centers divide the large inner-class variance into several smaller variances and conquered by combining all feature centers of one ship class. Finally, the probability distribution over all feature centers is considered comprehensively to achieve an accurate recognition of SAR ship images. The ablation experiments and experimental results on OpenSARShip and FUSAR-Ship datasets show that our method has achieved superior recognition performance under decreasing training SAR ship samples

Marine Vessel Inspection as a Novel Field for Service Robotics: A Contribution to Systems, Control Methods and Semantic Perception Algorithms.

This cumulative thesis introduces a novel field for service robotics: the inspection of marine vessels using mobile inspection robots. In this thesis, three scientific contributions are provided and experimentally verified in the field of marine inspection, but are not limited to this type of application. The inspection scenario is merely a golden thread to combine the cumulative scientific results presented in this thesis. The first contribution is an adaptive, proprioceptive control approach for hybrid leg-wheel robots, such as the robot ASGUARD described in this thesis. The robot is able to deal with rough terrain and stairs, due to the control concept introduced in this thesis. The proposed system is a suitable platform to move inside the cargo holds of bulk carriers and to deliver visual data from inside the hold. Additionally, the proposed system also has stair climbing abilities, allowing the system to move between different decks. The robot adapts its gait pattern dynamically based on proprioceptive data received from the joint motors and based on the pitch and tilt angle of the robot's body during locomotion. The second major contribution of the thesis is an independent ship inspection system, consisting of a magnetic wall climbing robot for bulkhead inspection, a particle filter based localization method, and a spatial content management system (SCMS) for spatial inspection data representation and organization. The system described in this work was evaluated in several laboratory experiments and field trials on two different marine vessels in close collaboration with ship surveyors. The third scientific contribution of the thesis is a novel approach to structural classification using semantic perception approaches. By these methods, a structured environment can be semantically annotated, based on the spatial relationships between spatial entities and spatial features. This method was verified in the domain of indoor perception (logistics and household environment), for soil sample classification, and for the classification of the structural parts of a marine vessel. The proposed method allows the description of the structural parts of a cargo hold in order to localize the inspection robot or any detected damage. The algorithms proposed in this thesis are based on unorganized 3D point clouds, generated by a LIDAR within a ship's cargo hold. Two different semantic perception methods are proposed in this thesis. One approach is based on probabilistic constraint networks; the second approach is based on Fuzzy Description Logic and spatial reasoning using a spatial ontology about the environment