FishMOT: A Simple and Effective Method for Fish Tracking Based on IoU Matching

The tracking of various fish species plays a profoundly significant role in understanding the behavior of individual fish and their groups. Present tracking methods suffer from issues of low accuracy or poor robustness. In order to address these concerns, this paper proposes a novel tracking approach, named FishMOT (Fish Multiple Object Tracking). This method combines object detection techniques with the IoU matching algorithm, thereby achieving efficient, precise, and robust fish detection and tracking. Diverging from other approaches, this method eliminates the need for multiple feature extractions and identity assignments for each individual, instead directly utilizing the output results of the detector for tracking, thereby significantly reducing computational time and storage space. Furthermore, this method imposes minimal requirements on factors such as video quality and variations in individual appearance. As long as the detector can accurately locate and identify fish, effective tracking can be achieved. This approach enhances robustness and generalizability. Moreover, the algorithm employed in this method addresses the issue of missed detections without relying on complex feature matching or graph optimization algorithms. This contributes to improved accuracy and reliability. Experimental trials were conducted in the open-source video dataset provided by idtracker.ai, and comparisons were made with state-of-the-art detector-based multi-object tracking methods. Additionally, comparisons were made with idtracker.ai and TRex, two tools that demonstrate exceptional performance in the field of animal tracking. The experimental results demonstrate that the proposed method outperforms other approaches in various evaluation metrics, exhibiting faster speed and lower memory requirements. The source codes and pre-trained models are available at: https://github.com/gakkistar/FishMO

Deep Learning for Multi-Animal Tracking

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Engenharia Clínica e Instrumentação Médica) Universidade de Lisboa, Faculdade de Ciências, 2021In collective behaviour studies, the use of multi-animal tracking systems is extremely valuable. To be able to identify and track each individual in a group helps in the study and understanding of their behaviour in the collective. For this, researchers can use tracking systems, which can use sensors to detect the individuals; or they can use image-based tracking, with or without the need to mark the individuals. idtracker.ai is a state-of-the-art multi-animal image-based tracking system that uses convolutional neural networks to identify each of the individuals in a video. In videos with a higher density of individuals, idtracker.ai cannot extract enough frames of the single individuals (few frames ≈ 30) and the training of the identification network is slower. With the idea of decreasing this training time, here we propose to test three different machine learning methods. The first method is to use Transfer Learning models, with the expectation that the training can be done with few data. The second method is to use the ensemble method to join the results of various models of the idtracker.ai identification network, and thus decrease the variability of classification. Finally, the third method is to use not only multi-class labels but also pairwise-labels to increase the amount of information the network has available for training. The three methods are compared to the idtracker.ai model in terms of image classification accuracy and training time. Transfer learning and ensemble improved the accuracy of classification, but failed to reduce the time of training of the identification network. The pairwise method increased accuracy and time of training was comparable to the one of idtracker.ai. More specifically, by training the identification network with multi-class labeled images and pairwise-labeled images, the information the network can have from few images leads to an average classification accuracy of 94% (for 3000 images per class with 30 multi-labeled images per class). This is comparable to idtracker.ai trained with 3000 multi-labeled images (per class) - 98% accuracy, and is better than when idtracker.ai is trained with 30 multi-labeled images - 56% accuracy

A review on the use of computer vision and artificial intelligence for fish recognition, monitoring, and management.

Abstract: Computer vision has been applied to fish recognition for at least three decades. With the inception of deep learning techniques in the early 2010s, the use of digital images grew strongly, and this trend is likely to continue. As the number of articles published grows, it becomes harder to keep track of the current state of the art and to determine the best course of action for new studies. In this context, this article characterizes the current state of the art by identifying the main studies on the subject and briefly describing their approach. In contrast with most previous reviews related to technology applied to fish recognition, monitoring, and management, rather than providing a detailed overview of the techniques being proposed, this work focuses heavily on the main challenges and research gaps that still remain. Emphasis is given to prevalent weaknesses that prevent more widespread use of this type of technology in practical operations under real-world conditions. Some possible solutions and potential directions for future research are suggested, as an effort to bring the techniques developed in the academy closer to meeting the requirements found in practice