Multiple Fish Tracking via Viterbi Data Association for Low-Frame-Rate Underwater Camera Systems †

Abstract-Non-extractive fish abundance estimation with the aid of visual analysis has drawn increasing attention. Low frame rate and variable illumination in the underwater environment, however, makes conventional tracking methods unreliable. In this paper, a robust multiple fish tracking system for low-framerate underwater stereo cameras is proposed. With the result of fish segmentation, a computationally efficient block-matching method is applied to perform successful stereo matching. A multiple-feature matching cost function is utilized to give a simple but effective metric for finding the temporal match of each target. Built upon reliable stereo matching, a multipletarget tracking algorithm via the Viterbi data association is developed to overcome the poor motion continuity of targets. Experimental results show that an accurate underwater live fish tracking result with stereo cameras is achieved

Change detection in combination with spatial models and its effectiveness on underwater scenarios

This thesis proposes a novel change detection approach for underwater scenarios and combines it with different especially developed spatial models, this allows accurate and spatially coherent detection of any moving objects with a static camera in arbitrary environments. To deal with the special problems of underwater imaging pre-segmentations based on the optical flow and other special adaptions were added to the change detection algorithm so that it can better handle typical underwater scenarios like a scene crowded by a whole fish swarm

Video Analysis : Techniques for Semi-Supervised Video Object Instance Segmentation and Tracking-by-Detection in the Wild

This thesis consists of two major components. The first part is concerned with video object instance segmentation (VOS), which is the task of assigning per-pixel labels perframe of a video sequence to indicate foreground object instance membership, given the first frame ground truth mask. VOS has myriad applications, from video post-processing to action recognition, and is an active area of research. A novel end-to-end trainable, online algorithm utilizing a bilinear LSTM to learn long-term appearance models is presented. The bilinear LSTM is used to guide the learned CNN features, integrating temporal information and building more discriminative appearance features for specific objects during inference. The second part of this thesis examines computer vision's potential applications for performing automated ecological inference for endemic flat-fish populations. Specifically, it looks at the construction of a visual tracking dataset, NHFish, consisting of underwater beam trawl videos collected along the Newport Hydrographic Line of Oregon coast benthos and the application of automated methods for video analysis of the beam trawl videos

Selected Papers from the 2018 IEEE International Workshop on Metrology for the Sea

This Special Issue is devoted to recent developments in instrumentation and measurement techniques applied to the marine field. ¶The sea is the medium that has allowed people to travel from one continent to another using vessels, even today despite the use of aircraft. It has also been acting as a great reservoir and source of food for all living beings. However, for many generations, it served as a landfill for depositing conventional and nuclear wastes, especially in its deep seabeds, and we are assisting in a race to exploit minerals and resources, different from foods, encompassed in it. Its health is a great challenge for the survival of all humanity since it is one of the most important environmental components targeted by global warming. ¶ As everyone may know, measuring is a step that generates substantial knowledge about a phenomenon or an asset, which is the basis for proposing correct solutions and making proper decisions. However, measurements in the sea environment pose unique difficulties and opportunities, which is made clear from the research results presented in this Special Issue

MSGNet: multi-source guidance network for fish segmentation in underwater videos

Fish segmentation in underwater videos provides basic data for fish measurements, which is vital information that supports fish habitat monitoring and fishery resources survey. However, because of water turbidity and insufficient lighting, fish segmentation in underwater videos has low accuracy and poor robustness. Most previous work has utilized static fish appearance information while ignoring fish motion in underwater videos. Considering that motion contains more detail, this paper proposes a method that simultaneously combines appearance and motion information to guide fish segmentation in underwater videos. First, underwater videos are preprocessed to highlight fish in motion, and obtain high-quality underwater optical flow. Then, a multi-source guidance network (MSGNet) is presented to segment fish in complex underwater videos with degraded visual features. To enhance both fish appearance and motion information, a non-local-based multiple co-attention guidance module (M-CAGM) is applied in the encoder stage, in which the appearance and motion features from the intra-frame salient fish and the moving fish in video sequences are reciprocally enhanced. In addition, a feature adaptive fusion module (FAFM) is introduced in the decoder stage to avoid errors accumulated in the video sequences due to blurred fish or inaccurate optical flow. Experiments based on three publicly available datasets were designed to test the performance of the proposed model. The mean pixel accuracy (mPA) and mean intersection over union (mIoU) of MSGNet were 91.89% and 88.91% respectively with the mixed dataset. Compared with those of the advanced underwater fish segmentation and video object segmentation models, the mPA and mIoU of the proposed model significantly improved. The results showed that MSGNet achieves excellent segmentation performance in complex underwater videos and can provide an effective segmentation solution for fisheries resource assessment and ocean observation. The proposed model and code are exposed via Github1