Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly "intelligent" behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication

Reliable Navigational Scene Perception for Autonomous Ships in Maritime Environment

Due to significant advances in robotics and transportation, research on autonomous ships has attracted considerable attention. The most critical task is to make the ships capable of accurately, reliably, and intelligently detecting their surroundings to achieve high levels of autonomy. Three deep learning-based models are constructed in this thesis to perform complex perceptual tasks such as identifying ships, analysing encounter situations, and recognising water surface objects. In this thesis, sensors, including the Automatic Identification System (AIS) and cameras, provide critical information for scene perception. Specifically, the AIS enables mid-range and long-range detection, assisting the decision-making system to take suitable and decisive action. A Convolutional Neural Network-Ship Movement Modes Classification (CNN-SMMC) is used to detect ships or objects. Following that, a Semi- Supervised Convolutional Encoder-Decoder Network (SCEDN) is developed to classify ship encounter situations and make a collision avoidance plan for the moving ships or objects. Additionally, cameras are used to detect short-range objects, a supplementary solution to ships or objects not equipped with an AIS. A Water Obstacle Detection Network based on Image Segmentation (WODIS) is developed to find potential threat targets. A series of quantifiable experiments have demonstrated that these models can provide reliable scene perception for autonomous ships

A semi-supervised deep learning model for ship encounter situation classification

Maritime safety is an important issue for global shipping industries. Currently, most of collision accidents at sea are caused by the misjudgement of the ship’s operators. The deployment of maritime autonomous surface ships (MASS) can greatly reduce ships’ reliance on human operators by using an automated intelligent collision avoidance system to replace human decision-making. To successfully develop such a system, the capability of autonomously identifying other ships and evaluating their associated encountering situation is of paramount importance. In this paper, we aim to identify ships’ encounter situation modes using deep learning methods based upon the Automatic Identification System (AIS) data. First, a segmentation process is developed to divide each ship’s AIS data into different segments that contain only one encounter situation mode. This is different to the majority of studies that have proposed encounter situation mode classification using hand-crafted features, which may not reflect the actual ship’s movement states. Furthermore, a number of present classification tasks are conducted using substantial labelled AIS data followed by a supervised training paradigm, which is not applicable to our dataset as it contains a large number of unlabelled AIS data. Therefore, a method called Semi-Supervised Convolutional Encoder–Decoder Network (SCEDN) for ship encounter situation classification based on AIS data is proposed. The structure of the network is not only able to automatically extract features from AIS segments but also share training parameters for the unlabelled data. The SCEDN uses an encoder–decoder convolutional structure with four channels for each segment (distance, speed, Time to the Closed Point of Approach (TCPA) and Distance to the Closed Point of Approach (DCPA)) been developed. The performance of the SCEDN model are evaluated by comparing to several baselines with the experimental results demonstrating a higher accuracy can be achieved by our proposed model

LSTM Pose Machines

We observed that recent state-of-the-art results on single image human pose estimation were achieved by multi-stage Convolution Neural Networks (CNN). Notwithstanding the superior performance on static images, the application of these models on videos is not only computationally intensive, it also suffers from performance degeneration and flicking. Such suboptimal results are mainly attributed to the inability of imposing sequential geometric consistency, handling severe image quality degradation (e.g. motion blur and occlusion) as well as the inability of capturing the temporal correlation among video frames. In this paper, we proposed a novel recurrent network to tackle these problems. We showed that if we were to impose the weight sharing scheme to the multi-stage CNN, it could be re-written as a Recurrent Neural Network (RNN). This property decouples the relationship among multiple network stages and results in significantly faster speed in invoking the network for videos. It also enables the adoption of Long Short-Term Memory (LSTM) units between video frames. We found such memory augmented RNN is very effective in imposing geometric consistency among frames. It also well handles input quality degradation in videos while successfully stabilizes the sequential outputs. The experiments showed that our approach significantly outperformed current state-of-the-art methods on two large-scale video pose estimation benchmarks. We also explored the memory cells inside the LSTM and provided insights on why such mechanism would benefit the prediction for video-based pose estimations.Comment: Poster in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 201

A ship movement classification based on Automatic Identification System (AIS) data using Convolutional Neural Network

With a wide use of AIS data in maritime transportation, there is an increasing demand to develop algorithms to efficiently classify a ship’s AIS data into different movements (static, normal navigation and manoeuvring). To achieve this, several studies have been proposed to use labelled features but with the drawback of not being able to effectively extract the details of ship movement information. In addition, a ship movement is in a free space, which is different to a road vehicle’s movement in road grids, making it inconvenient to directly migrate the methods for GPS data classification into AIS data. To deal with these problems, a Convolutional Neural Network-Ship Movement Modes Classification (CNN-SMMC) algorithm is proposed in this paper. The underlying concept of this method is to train a neural network to learn from the labelled AIS data, and the unlabelled AIS data can be effectively classified by using this trained network. More specifically, a Ship Movement Image Generation and Labelling (SMIGL) algorithm is first designed to convert a ship’s AIS trajectories into different movement images to make a full use of the CNN’s classification ability. Then, a CNN-SMMC architecture is built with a series of functional layers (convolutional layer, max-pooling layer, dense layer etc.) for ship movement classification with seven experiments been designed to find the optimal parameters for the CNN-SMMC. Considering the imbalanced features of AIS data, three metrics (average accuracy, score and Area Under Curve (AUC)) are selected to evaluate the performance of the CNN-SMMC. Finally, several benchmark classification algorithms (K-Nearest Neighbours (KNN), Support Vector Machine (SVM) and Decision Tree (DT)) are selected to compare with CNN-SMMC. The results demonstrate that the proposed CNN-SMMC has a better performance in the classification of AIS data

Hybrid Satellite-Terrestrial Communication Networks for the Maritime Internet of Things: Key Technologies, Opportunities, and Challenges

With the rapid development of marine activities, there has been an increasing number of maritime mobile terminals, as well as a growing demand for high-speed and ultra-reliable maritime communications to keep them connected. Traditionally, the maritime Internet of Things (IoT) is enabled by maritime satellites. However, satellites are seriously restricted by their high latency and relatively low data rate. As an alternative, shore & island-based base stations (BSs) can be built to extend the coverage of terrestrial networks using fourth-generation (4G), fifth-generation (5G), and beyond 5G services. Unmanned aerial vehicles can also be exploited to serve as aerial maritime BSs. Despite of all these approaches, there are still open issues for an efficient maritime communication network (MCN). For example, due to the complicated electromagnetic propagation environment, the limited geometrically available BS sites, and rigorous service demands from mission-critical applications, conventional communication and networking theories and methods should be tailored for maritime scenarios. Towards this end, we provide a survey on the demand for maritime communications, the state-of-the-art MCNs, and key technologies for enhancing transmission efficiency, extending network coverage, and provisioning maritime-specific services. Future challenges in developing an environment-aware, service-driven, and integrated satellite-air-ground MCN to be smart enough to utilize external auxiliary information, e.g., sea state and atmosphere conditions, are also discussed