Application of Deep Learning Methods for Detection and Tracking of Players

This chapter deals with the application of deep learning methods in sports scenes for the purpose of detecting and tracking the athletes and recognizing their activities. The scenes recorded during handball games and training activities will be used as an example. Handball is a team sport played with the ball with well-defined goals and rules, with a given number of players who can participate in the game as well as their roles. Athletes move quickly throughout the field during the game, change position and roles from defensive to offensive, use different techniques and actions, and very often are partially or completely occluded by another athlete. If artificial lighting and cluttered background are additionally taken into account, it is clear that these are very challenging tasks for object detectors and trackers. The chapter will present the results of various experiments that include player and ball detection using state-of-the-art deep convolutional neural networks such as YOLO v3 or Mask R-CNN, player tracking using Deep Sort, key player determination using activity measures, and action recognition using LSTM. In the conclusion, open issues and challenges in applying deep learning methods in such a dynamic sports environment will be discussed

Dataset preparation for swimmer detection

Velika količina podataka koja se svaki dan kreira može se upotrijebiti za razvoj algoritama umjetne inteligencije u domeni računalnog vida koji rješavaju zadatke poput klasifikacije slika, detekcije osoba i raspoznavanja akcija. Ti skupovi podataka su najčešće izrađeni od videozapisa i slika preuzetih s televizijskih kanala ili s društvene mreže YouTube i prikupljeni su i pripremljeni za odgovarajući zadatak. Nas je zanimao zadatak detekcije plivača, kako bi se model mogao koristiti za raspoznavanje i unaprjeđenje plivačkih tehnika. Iako danas postoje ogromne otvorene baze slika poput COCO i ImageNet, pripremljene za nadzirano strojno učenje te baze sportskih scena poput Olympic Sports Dataset, UCF Action Sport dataset ili Sport-1M koje uključuju slike popularnijih (gledanijih) sportova, nijedna od njih ne uključuje slike koje bi se mogle koristiti za izradu našeg modela za detekciju plivača. Stoga je u ovom radu opisan postupak snimanja i prikupljanja video materijala te priprema skupa slika UNIRI-SWM za detekciju plivača. Skup uključuje snimke plivača u realnim, situacijskim uvjetima treninga i natjecanja snimljenih akcijskim kamerama iz različitih kutova snimanja. U radu su dani rezultati detekcije plivača korištenjem dubokih konvolucijskih neuronskih mreža Mask R-CNN i Yolov3, naučenim na skupu općih slika prije i nakon učenja na skupu UNIRI-SWM. Rezultati pokazuju da se nakon prilagodbe modela na odgovarajućem skupu slika iz domene plivanja mogu postići jako dobri rezultati detekcije plivača.The large amount of data that is created every day can be used to develop artificial intelligence algorithms in the domain of computer vision that solve tasks such as image classification, face detection and action recognition. These datasets are most often created from videos and images downloaded from television channels or the YouTube social network and are collected and prepared for the appropriate task. We were interested in the task of detecting swimmers, so that the model could be used to recognize and improve swimming techniques. Although today there are huge open image databases like COCO and ImageNet, prepared for supervised machine learning and sports scene databases like Olympic Sports Dataset, UCF Action Sport dataset or Sport-1M that include images of more popular (watched) sports, none of them include images that could be used to make our swimmer detection model. Therefore, this paper describes the process of recording and collecting video material and preparing a set of UNIRI-SWM images for swimmer detection. The set includes shots of swimmers in real, situational training and competition conditions filmed by action cameras from different shooting angles. The paper presents the results of swimmer detection using deep convolutional neural networks Mask R-CNN and Yolo v3, learned in the set of general images before and after learning in the set UNIRI-SWM. The results show that after adjusting the model on the appropriate set of images from the swimming domain, very good results of swimmer detection can be achieved

Building a labeled dataset for recognition of handball actions using mask R-CNN and STIPS

Building successful machine learning models depends on large amounts of training data that often needs to be labelled manually. We propose a method to efficiently build an action recognition dataset in the handball domain, focusing on minimizing the manual labor required to label the individual players performing the chosen actions. The method uses existing deep learning object recognition methods for player detection and combines the obtained location information with a player activity measure based on spatio-temporal interest points to track players that are performing the currently relevant action, here called active players. The method was successfully used on a challenging dataset of real-world handball practice videos, where the leading active player was correctly tracked and labeled in 84 % of cases