Temporally stable feature clusters for maritime object tracking in visible and thermal imagery

This paper describes a new approach to detect and track maritime objects in real time. The approach particularly addresses the highly dynamic maritime environment, panning cameras, target scale changes, and operates on both visible and thermal imagery. Object detection is based on agglomerative clustering of temporally stable features. Object extents are first determined based on persistence of detected features and their relative separation and motion attributes. An explicit cluster merging and splitting process handles object creation and separation. Stable object clus- ters are tracked frame-to-frame. The effectiveness of the approach is demonstrated on four challenging real-world public datasets

Evaluating deep semantic segmentation networks for object detection in maritime surveillance

Maritime surveillance is important for applications in safety and security, but the visual detection of objects in maritime scenes remains challenging due to the diverse and unconstrained nature of such environments, and the need to operate in near real-time. Recent work on deep neural networks for semantic segmentation has achieved good performance in the road/urban scene parsing task. Driven by the potential application in autonomous vehicle navigation, many of the architectures are designed to be fast and lightweight. In this paper, we evaluate semantic segmentation networks in the context of an object detection system for maritime surveillance. Using data from the ADE20k scene parsing dataset, we train a selection of recent semantic segmentation network architectures to compare their performance on a number of publicly available maritime surveillance datasets

Deep Learning-Based Maritime Environment Segmentation for Unmanned Surface Vehicles Using Superpixel Algorithms

Unmanned surface vehicles (USVs) are receiving increasing attention in recent years from both academia and industry. To make a high-level autonomy for USVs, the environment situational awareness is a key capability. However, due to the richness of the features in marine environments, as well as the complexity of the environment influenced by sun glare and sea fog, the development of a reliable situational awareness system remains a challenging problem that requires further studies. This paper, therefore, proposes a new deep semantic segmentation model together with a Simple Linear Iterative Clustering (SLIC) algorithm, for an accurate perception for various maritime environments. More specifically, powered by the SLIC algorithm, the new segmentation model can achieve refined results around obstacle edges and improved accuracy for water surface obstacle segmentation. The overall structure of the new model employs an encoder–decoder layout, and a superpixel refinement is embedded before final outputs. Three publicly available maritime image datasets are used in this paper to train and validate the segmentation model. The final output demonstrates that the proposed model can provide accurate results for obstacle segmentation

Modified inter prediction H.264 video encoding for maritime surveillance

Video compression has evolved since it is first being standardized. The most popular CODEC, H.264 can compress video effectively according to the quality that is required. This is due to the motion estimation (ME) process that has impressive features like variable block sizes varying from 4×4 to 16×16 and quarter pixel motion compensation. However, the disadvantage of H.264 is that, it is very complex and impractical for hardware implementation. Many efforts have been made to produce low complexity encoding by compromising on the bitrate and decoded quality. Two notable methods are Fast Search Mode and Early Termination. In Early Termination concept, the encoder does not have to perform ME on every macroblock for every block size. If certain criteria are reached, the process could be terminated and the Mode Decision could select the best block size much faster. This project proposes on using background subtraction to maximize the Early Termination process. When recording using static camera, the background remains the same for a long period of time where most macroblocks will produce minimum residual. Thus in this thesis, the ME process for the background macroblock is terminated much earlier using the maximum 16×16 macroblock size. The accuracy of the background segmentation for maritime surveillance video case study is 88.43% and the true foreground rate is at 41.74%. The proposed encoder manages to reduce 73.5% of the encoding time and 80.5% of the encoder complexity. The bitrate of the output is also reduced, in the range of 20%, compared to the H.264 baseline encoder. The results show that the proposed method achieves the objectives of improving the compression rate and the encoding time

Detecting obstacles from camera image at open sea

While self-driving cars are a hot topic in these days, fewer people know that the same level of automation is being developed in the maritime industry. To enhance safety on board and to ensure the optimal utilization of crew members, automated assistant solutions are implemented on cargo ships and vessels. This thesis deals with a monocular camera-based system, that is capable of detection obstacles in open sea scenarios, and to estimate surrounding vehicles’ distance and bearing. After a solid research of existing methods and literature, an algorithm has been developed, containing three main parts. First, the real-world measurement data and camera images are being processed. Secondly, object detection is achieved with the YOLO deep learning methods that achieves at a high framerate and can be used for real-time applications. Lastly, distance and bearing values of detected obstacles are estimated based on geometrical calculations and mathematical equations that are validated with ground truth measurement data. Having multiple weeks of recorded measurement data from a RoPax vessel operating from Helsinki, allowed testing and validation already during the development phase. Results have shown that the systems’ detection capability is highly affected by the image resolution, and that distance estimation performance is reliable until 2-3 kilometers, but estimation errors rise at farther distances, due to physical limitations of cameras. In addition, as an interesting evaluation method, a survey has been conducted with industry professionals, to compare human distance estimation capability with the developed system. As a conclusion it can be stated that a significant need and huge potential can be found in automated safety solution in the maritime industry

Robust Multi-sensor Data Fusion for Practical Unmanned Surface Vehicles (USVs) Navigation

The development of practical Unmanned Surface Vehicles (USVs) are attracting increasing attention driven by their assorted military and commercial application potential. However, addressing the uncertainties presented in practical navigational sensor measurements of an USV in maritime environment remain the main challenge of the development. This research aims to develop a multi-sensor data fusion system to autonomously provide an USV reliable navigational information on its own positions and headings as well as to detect dynamic target ships in the surrounding environment in a holistic fashion. A multi-sensor data fusion algorithm based on Unscented Kalman Filter (UKF) has been developed to generate more accurate estimations of USV’s navigational data considering practical environmental disturbances. A novel covariance matching adaptive estimation algorithm has been proposed to deal with the issues caused by unknown and varying sensor noise in practice to improve system robustness. Certain measures have been designed to determine the system reliability numerically, to recover USV trajectory during short term sensor signal loss, and to autonomously detect and discard permanently malfunctioned sensors, and thereby enabling potential sensor faults tolerance. The performance of the algorithms have been assessed by carrying out theoretical simulations as well as using experimental data collected from a real-world USV projected collaborated with Plymouth University. To increase the degree of autonomy of USVs in perceiving surrounding environments, target detection and prediction algorithms using an Automatic Identification System (AIS) in conjunction with a marine radar have been proposed to provide full detections of multiple dynamic targets in a wider coverage range, remedying the narrow detection range and sensor uncertainties of the AIS. The detection algorithms have been validated in simulations using practical environments with water current effects. The performance of developed multi-senor data fusion system in providing reliable navigational data and perceiving surrounding environment for USV navigation have been comprehensively demonstrated

Marine Robots for Underwater Surveillance

Abstract Purpose of Review The paper reviews the role of marine robots, in particular unmanned vehicles, in underwater surveillance, i.e. the control and monitoring of an area of competence aimed at identifying potential threats in support of homeland defence, antiterrorism, force protection and Explosive Ordnance Disposal (EOD). Recent Findings The paper explores separately robotic missions for identification and classification of threats lying on the seabed (e.g. EOD) and anti-intrusion robotic systems. The current main scientific challenge is identified in terms of enhancing autonomy and team/swarm mission capabilities by improving interoperability among robotic vehicles and providing communication networking capabilities, a non-trivial task, giving the severe limitations in bandwidth and latency of acoustic underwater messaging. Summary The work is intended to be a critical guide to the recent prolific bibliography on the topic, providing pointers to the main recent advancements in the field, and to give also a set of references in terms of mission and stakeholders' requirements (port authorities, coastal guards, navies)

Feature-based object tracking in maritime scenes.

A monitoring of presence, location and activity of various objects on the sea is essential for maritime navigation and collision avoidance. Mariners normally rely on two complementary methods of the monitoring: radar and satellite-based aids and human observation. Though radar aids are relatively accurate at long distances, their capability of detecting small, unmanned or non-metallic craft that generally do not reflect radar waves sufficiently enough, is limited. The mariners, therefore, rely in such cases on visual observations. The visual observation is often facilitated by using cameras overlooking the sea that can also provide intensified infra-red images. These systems or nevertheless merely enhance the image and the burden of the tedious and error-prone monitoring task still rests with the operator. This thesis addresses the drawbacks of both methods by presenting a framework consisting of a set of machine vision algorithms that facilitate the monitoring tasks in maritime environment. The framework detects and tracks objects in a sequence of images captured by a camera mounted either on a board of a vessel or on a static platform over-looking the sea. The detection of objects is independent of their appearance and conditions such as weather and time of the day. The output of the framework consists of locations and motions of all detected objects with respect to a fixed point in the scene. All values are estimated in real-world units, i. e. location is expressed in metres and velocity in knots. The consistency of the estimates is maintained by compensating for spurious effects such as vibration of the camera. In addition, the framework continuously checks for predefined events such as collision threats or area intrusions, raising an alarm when any such event occurs. The development and evaluation of the framework is based on sequences captured under conditions corresponding to a designated application. The independence of the detection and tracking on the appearance of the sceneand objects is confirmed by a final cross-validation of the framework on previously unused sequences. Potential applications of the framework in various areas of maritime environment including navigation, security, surveillance and others are outlined. Limitations to the presented framework are identified and possible solutions suggested. The thesis concludes with suggestions to further directions of the research presented

3D reconstruction and motion estimation using forward looking sonar

Autonomous Underwater Vehicles (AUVs) are increasingly used in different domains including archaeology, oil and gas industry, coral reef monitoring, harbour’s security, and mine countermeasure missions. As electromagnetic signals do not penetrate underwater environment, GPS signals cannot be used for AUV navigation, and optical cameras have very short range underwater which limits their use in most underwater environments. Motion estimation for AUVs is a critical requirement for successful vehicle recovery and meaningful data collection. Classical inertial sensors, usually used for AUV motion estimation, suffer from large drift error. On the other hand, accurate inertial sensors are very expensive which limits their deployment to costly AUVs. Furthermore, acoustic positioning systems (APS) used for AUV navigation require costly installation and calibration. Moreover, they have poor performance in terms of the inferred resolution. Underwater 3D imaging is another challenge in AUV industry as 3D information is increasingly demanded to accomplish different AUV missions. Different systems have been proposed for underwater 3D imaging, such as planar-array sonar and T-configured 3D sonar. While the former features good resolution in general, it is very expensive and requires huge computational power, the later is cheaper implementation but requires long time for full 3D scan even in short ranges. In this thesis, we aim to tackle AUV motion estimation and underwater 3D imaging by proposing relatively affordable methodologies and study different parameters affecting their performance. We introduce a new motion estimation framework for AUVs which relies on the successive acoustic images to infer AUV ego-motion. Also, we propose an Acoustic Stereo Imaging (ASI) system for underwater 3D reconstruction based on forward looking sonars; the proposed system features cheaper implementation than planar array sonars and solves the delay problem in T configured 3D sonars

A Self-Guided Docking Architecture for Autonomous Surface Vehicles

Autonomous Surface Vehicles (ASVs) provide the ideal platform to further explore the many opportunities in the cargo shipping industry, by making it more profitable and safer. Information retrieved from a 3D LIDAR, IMU, GPS, and Camera is combined to extract the geometric features of the floating platform and to estimate the relative position and orientation of the moor to the ASV. Then, a trajectory is planned to a specific target position, guaranteeing that the ASV will not collide with the mooring facility. To ensure that the sensors are within range of operation, a module has been developed to generate a trajectory that will deliver the ASV to a catch zone where it is able to function properly.A High-Level controler is also implemented, resorting to an heuristic to evaluate if the ASV is within this operating range and also its current orientation relative to the docking platform