Deep Convolutional Neural Network based Ship Images Classification

Ships are an integral part of maritime traffic where they play both militaries as well as non-combatant roles. This vast maritime traffic needs to be managed and monitored by identifying and recognising vessels to ensure the maritime safety and security. As an approach to find an automated and efficient solution, a deep learning model exploiting convolutional neural network (CNN) as a basic building block, has been proposed in this paper. CNN has been predominantly used in image recognition due to its automatic high-level features extraction capabilities and exceptional performance. We have used transfer learning approach using pre-trained CNNs based on VGG16 architecture to develop an algorithm that performs the different ship types classification. This paper adopts data augmentation and fine-tuning to further improve and optimize the baseline VGG16 model. The proposed model attains an average classification accuracy of 97.08% compared to the average classification accuracy of 88.54% obtained from the baseline model

Deep learning-based vessel detection from very high and medium resolution optical satellite images as component of maritime surveillance systems

This thesis presents an end-to-end multiclass vessel detection method from optical satellite images. The proposed workflow covers the complete processing chain and involves rapid image enhancement techniques, the fusion with automatic identification system (AIS) data, and the detection algorithm based on convolutional neural networks (CNN). The algorithms presented are implemented in the form of independent software processors and integrated in an automated processing chain as part of the Earth Observation Maritime Surveillance System (EO-MARISS).In der vorliegenden Arbeit wird eine Methode zur Detektion von Schiffen unterschiedlicher Klassen in optischen Satellitenbildern vorgestellt. Diese gliedert sich in drei aufeinanderfolgende Funktionen: i) die Bildbearbeitung zur Verbesserung der Bildeigenschaften, ii) die Datenfusion mit den Daten des Automatischen Identifikation Systems (AIS) und iii) dem auf „Convolutional Neural Network“ (CNN) basierenden Detektionsalgorithmus. Die vorgestellten Algorithmen wurden in Form eigenständiger Softwareprozessoren implementiert und als Teil des maritimen Erdbeobachtungssystems integriert

Quadruplet Selection Methods for Deep Embedding Learning

Recognition of objects with subtle differences has been used in many practical applications, such as car model recognition and maritime vessel identification. For discrimination of the objects in fine-grained detail, we focus on deep embedding learning by using a multi-task learning framework, in which the hierarchical labels (coarse and fine labels) of the samples are utilized both for classification and a quadruplet-based loss function. In order to improve the recognition strength of the learned features, we present a novel feature selection method specifically designed for four training samples of a quadruplet. By experiments, it is observed that the selection of very hard negative samples with relatively easy positive ones from the same coarse and fine classes significantly increases some performance metrics in a fine-grained dataset when compared to selecting the quadruplet samples randomly. The feature embedding learned by the proposed method achieves favorable performance against its state-of-the-art counterparts.Comment: 6 pages, 2 figures, accepted by IEEE ICIP 201

How a ship´s bridge knows its position - ECDIS assisted accidents from a contemporary human factors perspective

The technological artifacts used in ship navigation have undergone substantial changes during the last decades, and real-time digital navigation is a reality with the introduction of the ECDIS. Despite the obvious merits of this new navigation mode, and the imagined improvement in safety that it theoretically should bring, ECDIS has in recent years been associated with several accidents. The term ECDIS assisted accidents has emerged in official accident investigation reports and is widely used among the applied technology community as well as having led to the term reverberating the RADAR assisted accidents that the maritime industry has used following the introduction of the RADAR. Despite the focus on the causal contribution from the interplay between the ECDIS and the navigator, the conclusions in the official accident investigation reports are predominantly directed towards the abilities of the ECDIS operator to use the equipment properly, and to a lesser extent on the features of the ECDIS. The reports do not at all investigate how the equipment could have helped navigators, by offering better support in reaching their contextual goals, i.e., to remain in control of the ship and to maintain safe navigation. Parallel accounts emanating from the applied community of ship navigation seem to suggest that functioning of the ECDIS is far from perfect, and at times is considered suboptimal by navigators. The ambition driving this thesis work was to explore these second stories about navigation with ECDIS, based on operator experiences, in order to gain leverage for new ways to inform future development and design of ECDIS, which to a higher degree would need to take into account the contextual conditions and demands that operators experience in the field of practice, and thereby to minimize the gap between how designers, and other remote stakeholders, imagine ECDIS operations, and how these actually play out. Naturalistic research was carried out by attending three ships ́ bridges while the ships were operating. Insights were gained into what sometimes make work difficult during navigation by ECDIS. The findings were juxtaposed with information found in three official accident accounts of ECDIS assisted accidents, and finally the results were discussed based on a theoretical framework based on contemporary human factors and systems safety research literature, including Cognitive Systems Engineering. Thus, it was concluded how the methods applied in this thesis work, and its findings, could be useful to future ECDIS design and development

From multiple aspect trajectories to predictive analysis: a case study on fishing vessels in the Northern Adriatic sea

In this paper we model spatio-temporal data describing the fishing activities in the Northern Adriatic Sea over four years. We build, implement and analyze a database based on the fusion of two complementary data sources: trajectories from fishing vessels (obtained from terrestrial Automatic Identification System, or AIS, data feed) and fish catch reports (i.e., the quantity and type of fish caught) of the main fishing market of the area. We present all the phases of the database creation, starting from the raw data and proceeding through data exploration, data cleaning, trajectory reconstruction and semantic enrichment. We implement the database by using MobilityDB, an open source geospatial trajectory data management and analysis platform. Subsequently, we perform various analyses on the resulting spatio-temporal database, with the goal of mapping the fishing activities on some key species, highlighting all the interesting information and inferring new knowledge that will be useful for fishery management. Furthermore, we investigate the use of machine learning methods for predicting the Catch Per Unit Effort (CPUE), an indicator of the fishing resources exploitation in order to drive specific policy design. A variety of prediction methods, taking as input the data in the database and environmental factors such as sea temperature, waves height and Clorophill-a, are put at work in order to assess their prediction ability in this field. To the best of our knowledge, our work represents the first attempt to integrate fishing ships trajectories derived from AIS data, environmental data and catch data for spatio-temporal prediction of CPUE – a challenging task

A geo-database for potentially polluting marine sites and associated risk index

The increasing availability of geospatial marine data provides an opportunity for hydrographic offices to contribute to the identification of Potentially Polluting Marine Sites (PPMS). To adequately manage these sites, a PPMS Geospatial Database (GeoDB) application was developed to collect and store relevant information suitable for site inventory and geo-spatial analysis. The benefits of structuring the data to conform to the Universal Hydrographic Data Model (IHO S-100) and to use the Geographic Mark-Up Language (GML) for encoding are presented. A storage solution is proposed using a GML-enabled spatial relational database management system (RDBMS). In addition, an example of a risk index methodology is provided based on the defined data structure. The implementation of this example was performed using scripts containing SQL statements. These procedures were implemented using a cross-platform C++ application based on open-source libraries and called PPMS GeoDB Manager

Deep Learning of Semantic Image Labels on HDR Imagery in a Maritime Environment

Situational awareness in the maritime environment can be extremely challenging. The maritime environment is highly dynamic and largely undefined, requiring the perception of many potential hazards in the shared maritime environment. One particular challenge is the effect of direct-sunlight exposure and specular reflection causing degradation of camera reliability. It is for this reason then, in this work, the use of High-Dynamic Range imagery for deep learning of semantic image labels is studied in a littoral environment. This study theorizes that the use HDR imagery may be extremely beneficial for the purpose of situational awareness in maritime environments due to the inherent advantages of the technology. This study creates labels for a multi-class semantic segmentation process, and performs well on water and horizon identification in the littoral zone. Additionally, this work contributes proof that water can be reasonably identified using HDR imagery with semantic networks, which is useful for determining the navigable regions for a vessel. This result is a basis on which to build further semantic segmentation work upon in this environment, and could be further improved upon in future works with the introduction of additional data for multi-class segmentation problems

Ship detection in SAR images based on Maxtree representation and graph signal processing

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper discusses an image processing architecture and tools to address the problem of ship detection in synthetic-aperture radar images. The detection strategy relies on a tree-based representation of images, here a Maxtree, and graph signal processing tools. Radiometric as well as geometric attributes are evaluated and associated with the Maxtree nodes. They form graph attribute signals which are processed with graph filters. The goal of this filtering step is to exploit the correlation existing between attribute values on neighboring tree nodes. Considering that trees are specific graphs where the connectivity toward ancestors and descendants may have a different meaning, we analyze several linear, nonlinear, and morphological filtering strategies. Beside graph filters, two new filtering notions emerge from this analysis: tree and branch filters. Finally, we discuss a ship detection architecture that involves graph signal filters and machine learning tools. This architecture demonstrates the interest of applying graph signal processing tools on the tree-based representation of images and of going beyond classical graph filters. The resulting approach significantly outperforms state-of-the-art algorithms. Finally, a MATLAB toolbox allowing users to experiment with the tools discussed in this paper on Maxtree or Mintree has been created and made public.Peer ReviewedPostprint (author's final draft