A framework based on Gaussian mixture models and Kalman filters for the segmentation and tracking of anomalous events in shipboard video

Anomalous indications in monitoring equipment on board U.S. Navy vessels must be handled in a timely manner to prevent catastrophic system failure. The development of sensor data analysis techniques to assist a ship\u27s crew in monitoring machinery and summon required ship-to-shore assistance is of considerable benefit to the Navy. In addition, the Navy has a large interest in the development of distance support technology in its ongoing efforts to reduce manning on ships. In this thesis, algorithms have been developed for the detection of anomalous events that can be identified from the analysis of monochromatic stationary ship surveillance video streams. The specific anomalies that we have focused on are the presence and growth of smoke and fire events inside the frames of the video stream. The algorithm consists of the following steps. First, a foreground segmentation algorithm based on adaptive Gaussian mixture models is employed to detect the presence of motion in a scene. The algorithm is adapted to emphasize gray-level characteristics related to smoke and fire events in the frame. Next, shape discriminant features in the foreground are enhanced using morphological operations. Following this step, the anomalous indication is tracked between frames using Kalman filtering. Finally, gray level shape and motion features corresponding to the anomaly are subjected to principal component analysis and classified using a multilayer perceptron neural network. The algorithm is exercised on 68 video streams that include the presence of anomalous events (such as fire and smoke) and benign/nuisance events (such as humans walking the field of view). Initial results show that the algorithm is successful in detecting anomalies in video streams, and is suitable for application in shipboard environments

Sea-Surface Object Detection Based on Electro-Optical Sensors: A Review

Sea-surface object detection is critical for navigation safety of autonomous ships. Electrooptical (EO) sensors, such as video cameras, complement radar on board in detecting small obstacle sea-surface objects. Traditionally, researchers have used horizon detection, background subtraction, and foreground segmentation techniques to detect sea-surface objects. Recently, deep learning-based object detection technologies have been gradually applied to sea-surface object detection. This article demonstrates a comprehensive overview of sea-surface object-detection approaches where the advantages and drawbacks of each technique are compared, covering four essential aspects: EO sensors and image types, traditional object-detection methods, deep learning methods, and maritime datasets collection. In particular, sea-surface object detections based on deep learning methods are thoroughly analyzed and compared with highly influential public datasets introduced as benchmarks to verify the effectiveness of these approaches. The arti

Intelligent Vision-based Autonomous Ship Landing of VTOL UAVs

The paper discusses an intelligent vision-based control solution for autonomous tracking and landing of Vertical Take-Off and Landing (VTOL) capable Unmanned Aerial Vehicles (UAVs) on ships without utilizing GPS signal. The central idea involves automating the Navy helicopter ship landing procedure where the pilot utilizes the ship as the visual reference for long-range tracking; however, refers to a standardized visual cue installed on most Navy ships called the "horizon bar" for the final approach and landing phases. This idea is implemented using a uniquely designed nonlinear controller integrated with machine vision. The vision system utilizes machine learning-based object detection for long-range ship tracking and classical computer vision for the estimation of aircraft relative position and orientation utilizing the horizon bar during the final approach and landing phases. The nonlinear controller operates based on the information estimated by the vision system and has demonstrated robust tracking performance even in the presence of uncertainties. The developed autonomous ship landing system was implemented on a quad-rotor UAV equipped with an onboard camera, and approach and landing were successfully demonstrated on a moving deck, which imitates realistic ship deck motions. Extensive simulations and flight tests were conducted to demonstrate vertical landing safety, tracking capability, and landing accuracy

Image Pre-processing Algorithms for Detection of Small/Point Airborne Targets

The problem of detecting small/point targets in infra-red imagery is an important research area for defence applications. The challenge is to achieve high sensitivity for detection of dim point like small targets with low false alarms and high detection probability. To detect the target in such scenario, pre-processing algorithms are used to predict the complex background and then to subtract predicted background from the original image. The difference image is passed to the detection algorithm to further distinguish between target and background and/or noise. The aim of the study is to fit the background as closely as possible in the original image without diminishing the target signal. A number of pre-processing algorithms (spatial, temporal and spatio-temporal) have been reported in the literature. In this paper a survey of different pre-processing algorithm is presented. An improved hybrid morphological filter, which provides high gain in signal-to-noise plus clutter ratio (SCNR), has been proposed for detection of small/point targets.Defence Science Journal, 2009, 59(2), pp.166-174, DOI:http://dx.doi.org/10.14429/dsj.59.150

Research Brief

Approved for public release; distribution is unlimited

A self-selective correlation ship tracking method for smart ocean systems

In recent years, with the development of the marine industry, the ship navigation environment has become more complicated. Some artificial intelligence technologies, such as computer vision, can recognize, track and count sailing ships to ensure maritime security and facilitate management for Smart Ocean systems. Aiming at the scaling problem and boundary effect problem of traditional correlation filtering methods, we propose a self-selective correlation filtering method based on box regression (BRCF). The proposed method mainly includes: (1) A self-selective model with a negative samples mining method which effectively reduces the boundary effect in strengthening the classification ability of the classifier at the same time; (2) a bounding box regression method combined with a key points matching method for the scale prediction, leading to a fast and efficient calculation. The experimental results show that the proposed method can effectively deal with the problem of ship size changes and background interference. The success rates and precisions were over 8 % higher than Discriminative Scale Space Tracking (DSST) on the marine traffic dataset of our laboratory. In terms of processing speed, the proposed method is higher than DSST by nearly 22 frames per second (FPS).This research was supported by the National Natural Science Foundation of China under Grant (No. 61772387 and No. 61802296), the Fundamental Research Funds of Ministry of Education and China Mobile (MCM20170202), the Fundamental Research Funds for the Central Universities (JB180101), China Postdoctoral Science Foundation Grant (No. 2017M620438), and supported by ISN State Key Laboratory

Fish detection automation from ARIS and DIDSON SONAR data

Abstract. The goal of this thesis is to analyse SONAR files produced by ARIS and DIDSON manufactured by Sound Metrics Co. which are ultrasonic, monostatic and multibeam echo-sounders. They are used to capture the behaviour of Atlantic salmon, which recently has been on the lists of endangered species. These SONARs can work in dark lighting conditions and provide high resolution images due to their high frequencies that ranges from 1.1 MHz to 1.8 MHz. The thesis goes through extracting data from file, redrawing it, and visualising it in human friendly format. Next, images are analysed to search for fish. Results of analysis are saved in formats such as JSON, to allow harmony with other legacy systems. Also the output helps in future development due to the support for JSON in multitude of programming languages. Eventually, a user-friendly user interface is introduced, which helps making the process easier. The software is tested against data-sets from rivers in Finland, that are rich in Atlantic salmon