Unsupervised maritime target detection

The unsupervised detection of maritime targets in grey scale video is a difficult problem in maritime video surveillance. Most approaches assume that the camera is static and employ pixel-wise background modelling techniques for foreground detection; other methods rely on colour or thermal information to detect targets. These methods fail in real-world situations when the static camera assumption is violated, and colour or thermal data is unavailable. In defence and security applications, prior information and training samples of targets may be unavailable for training a classifier; the learning of a one class classifier for the background may be impossible as well. Thus, an unsupervised online approach that attempts to learn from the scene data is highly desirable. In this thesis, the characteristics of the maritime scene and the ocean texture are exploited for foreground detection. Two fast and effective methods are investigated for target detection. Firstly, online regionbased background texture models are explored for describing the appearance of the ocean. This approach avoids the need for frame registration because the model is built spatially rather than temporally. The texture appearance of the ocean is described using Local Binary Pattern (LBP) descriptors. Two models are proposed: one model is a Gaussian Mixture (GMM) and the other, referred to as a Sparse Texture Model (STM), is a set of histogram texture distributions. The foreground detections are optimized using a Graph Cut (GC) that enforces spatial coherence. Secondly, feature tracking is investigated as a means of detecting stable features in an image frame that typically correspond to maritime targets; unstable features are background regions. This approach is a Track-Before-Detect (TBD) concept and it is implemented using a hierarchical scheme for motion estimation, and matching of Scale- Invariant Feature Transform (SIFT) appearance features. The experimental results show that these approaches are feasible for foreground detection in maritime video when the camera is either static or moving. Receiver Operating Characteristic (ROC) curves were generated for five test sequences and the Area Under the ROC Curve (AUC) was analyzed for the performance of the proposed methods. The texture models, without GC optimization, achieved an AUC of 0.85 or greater on four out of the five test videos. At 50% True Positive Rate (TPR), these four test scenarios had a False Positive Rate (FPR) of less than 2%. With the GC optimization, an AUC of greater than 0.8 was achieved for all the test cases and the FPR was reduced in all cases when compared to the results without the GC. In comparison to the state of the art in background modelling for maritime scenes, our texture model methods achieved the best performance or comparable performance. The two texture models executed at a reasonable processing frame rate. The experimental results for TBD show that one may detect target features using a simple track score based on the track length. At 50% TPR a FPR of less than 4% is achieved for four out of the five test scenarios. These results are very promising for maritime target detection

Modelling false positive reduction in maritime object detection

Target detection has become a very significant research area in computer vision with its applications in military, maritime surveillance, and defense and security. Maritime target detection during critical sea conditions produces a number of false positives when using the existing algorithms due to sea waves, dynamic nature of the ocean, camera motion, sea glint, sensor noise, sea spray, swell and the presence of birds. The main question that has been addressed in this research is how can object detection be improved in maritime environment by reducing false positives and promoting detection rate. Most of Previous work on object detection still fails to address the problem of false positives and false negatives due to background clutter. Most of the researchers tried to reduce false positives by applying filters but filtering degrades the quality of an image leading to more false alarms during detection. As much as radar technology has previously been the most utilized method, it still fails to detect very small objects and it may be applied in special circumstances. In trying to improve the implementation of target detection in maritime, empirical research method was proposed to answer questions about existing target detection algorithms and techniques used to reduce false positives in object detection. Visible images were retrained on a pre-trained Faster R-CNN with inception v2. The pre-trained model was retrained on five different sample data with increasing size, however for the last two samples the data was duplicated to increase size. For testing purposes 20 test images were utilized to evaluate all the models. The results of this study showed that the deep learning method used performed best in detecting maritime vessels and the increase of dataset improved detection performance and false positives were reduced. The duplication of images did not yield the best results; however, the results were promising for the first three models with increasing data

Modelling false positive reduction in maritime object detection

Target detection has become a very significant research area in computer vision with its applications in military, maritime surveillance, and defense and security. Maritime target detection during critical sea conditions produces a number of false positives when using the existing algorithms due to sea waves, dynamic nature of the ocean, camera motion, sea glint, sensor noise, sea spray, swell and the presence of birds. The main question that has been addressed in this research is how can object detection be improved in maritime environment by reducing false positives and promoting detection rate. Most of Previous work on object detection still fails to address the problem of false positives and false negatives due to background clutter. Most of the researchers tried to reduce false positives by applying filters but filtering degrades the quality of an image leading to more false alarms during detection. As much as radar technology has previously been the most utilized method, it still fails to detect very small objects and it may be applied in special circumstances. In trying to improve the implementation of target detection in maritime, empirical research method was proposed to answer questions about existing target detection algorithms and techniques used to reduce false positives in object detection. Visible images were retrained on a pre-trained Faster R-CNN with inception v2. The pre-trained model was retrained on five different sample data with increasing size, however for the last two samples the data was duplicated to increase size. For testing purposes 20 test images were utilized to evaluate all the models. The results of this study showed that the deep learning method used performed best in detecting maritime vessels and the increase of dataset improved detection performance and false positives were reduced. The duplication of images did not yield the best results; however, the results were promising for the first three models with increasing data

Sea-Surface Object Detection Based on Electro-Optical Sensors: A Review

Sea-surface object detection is critical for navigation safety of autonomous ships. Electrooptical (EO) sensors, such as video cameras, complement radar on board in detecting small obstacle sea-surface objects. Traditionally, researchers have used horizon detection, background subtraction, and foreground segmentation techniques to detect sea-surface objects. Recently, deep learning-based object detection technologies have been gradually applied to sea-surface object detection. This article demonstrates a comprehensive overview of sea-surface object-detection approaches where the advantages and drawbacks of each technique are compared, covering four essential aspects: EO sensors and image types, traditional object-detection methods, deep learning methods, and maritime datasets collection. In particular, sea-surface object detections based on deep learning methods are thoroughly analyzed and compared with highly influential public datasets introduced as benchmarks to verify the effectiveness of these approaches. The arti

Automatic human behaviour anomaly detection in surveillance video

This thesis work focusses upon developing the capability to automatically evaluate and detect anomalies in human behaviour from surveillance video. We work with static monocular cameras in crowded urban surveillance scenarios, particularly air- ports and commercial shopping areas. Typically a person is 100 to 200 pixels high in a scene ranging from 10 - 20 meters width and depth, populated by 5 to 40 peo- ple at any given time. Our procedure evaluates human behaviour unobtrusively to determine outlying behavioural events, agging abnormal events to the operator. In order to achieve automatic human behaviour anomaly detection we address the challenge of interpreting behaviour within the context of the social and physical environment. We develop and evaluate a process for measuring social connectivity between individuals in a scene using motion and visual attention features. To do this we use mutual information and Euclidean distance to build a social similarity matrix which encodes the social connection strength between any two individuals. We de- velop a second contextual basis which acts by segmenting a surveillance environment into behaviourally homogeneous subregions which represent high tra c slow regions and queuing areas. We model the heterogeneous scene in homogeneous subgroups using both contextual elements. We bring the social contextual information, the scene context, the motion, and visual attention features together to demonstrate a novel human behaviour anomaly detection process which nds outlier behaviour from a short sequence of video. The method, Nearest Neighbour Ranked Outlier Clusters (NN-RCO), is based upon modelling behaviour as a time independent se- quence of behaviour events, can be trained in advance or set upon a single sequence. We nd that in a crowded scene the application of Mutual Information-based social context permits the ability to prevent self-justifying groups and propagate anomalies in a social network, granting a greater anomaly detection capability. Scene context uniformly improves the detection of anomalies in all the datasets we test upon. We additionally demonstrate that our work is applicable to other data domains. We demonstrate upon the Automatic Identi cation Signal data in the maritime domain. Our work is capable of identifying abnormal shipping behaviour using joint motion dependency as analogous for social connectivity, and similarly segmenting the shipping environment into homogeneous regions

Survey on video anomaly detection in dynamic scenes with moving cameras

The increasing popularity of compact and inexpensive cameras, e.g.~dash cameras, body cameras, and cameras equipped on robots, has sparked a growing interest in detecting anomalies within dynamic scenes recorded by moving cameras. However, existing reviews primarily concentrate on Video Anomaly Detection (VAD) methods assuming static cameras. The VAD literature with moving cameras remains fragmented, lacking comprehensive reviews to date. To address this gap, we endeavor to present the first comprehensive survey on Moving Camera Video Anomaly Detection (MC-VAD). We delve into the research papers related to MC-VAD, critically assessing their limitations and highlighting associated challenges. Our exploration encompasses three application domains: security, urban transportation, and marine environments, which in turn cover six specific tasks. We compile an extensive list of 25 publicly-available datasets spanning four distinct environments: underwater, water surface, ground, and aerial. We summarize the types of anomalies these datasets correspond to or contain, and present five main categories of approaches for detecting such anomalies. Lastly, we identify future research directions and discuss novel contributions that could advance the field of MC-VAD. With this survey, we aim to offer a valuable reference for researchers and practitioners striving to develop and advance state-of-the-art MC-VAD methods.Comment: Under revie

Real-time embedded reconstruction of dynamic objects for a 3D maritime situational awareness picture

Monitoring maritime infrastructures is an important part of maritime safety and security. To best assess the security status of these facilities, detailed information should be made available to stakeholders, such as port authorities, law enforcement agencies and emergency services in a concise and easily understandable format. In this work, we propose a novel real-time 3D reconstruction framework for enhancing maritime situational awareness. We introduce and verify a pipeline prototype for dynamic 3D reconstruction of maritime objects using a static observer and stereoscopic cameras on an GPU-accelerated embedded device. Our pipeline runs with approx. 6Hz on a Nvidia Jetson Xavier AGX embedded system and is verified using a simulated dataset of a harbor basin