Automatic annotation of tennis games: An integration of audio, vision, and learning

Fully automatic annotation of tennis game using broadcast video is a task with a great potential but with enormous challenges. In this paper we describe our approach to this task, which integrates computer vision, machine listening, and machine learning. At the low level processing, we improve upon our previously proposed state-of-the-art tennis ball tracking algorithm and employ audio signal processing techniques to detect key events and construct features for classifying the events. At high level analysis, we model event classification as a sequence labelling problem, and investigate four machine learning techniques using simulated event sequences. Finally, we evaluate our proposed approach on three real world tennis games, and discuss the interplay between audio, vision and learning. To the best of our knowledge, our system is the only one that can annotate tennis game at such a detailed level

Automatic annotation of tennis games: an integration of audio, vision, and learning

Fully automatic annotation of tennis game using broadcast video is a task with a great potential but with enormous challenges. In this paper we describe our approach to this task, which integrates computer vision, machine listening, and machine learning. At the low level processing, we improve upon our previously proposed state-of-the-art tennis ball tracking algorithm and employ audio signal processing techniques to detect key events and construct features for classifying the events. At high level analysis, we model event classification as a sequence labelling problem, and investigate four machine learning techniques using simulated event sequences. Finally, we evaluate our proposed approach on three real world tennis games, and discuss the interplay between audio, vision and learning. To the best of our knowledge, our system is the only one that can annotate tennis game at such a detailed level

Automatic annotation of tennis games: an integration of audio, vision, and learning

Fully automatic annotation of tennis game using broadcast video is a task with a great potential but with enormous challenges. In this paper we describe our approach to this task, which integrates computer vision, machine listening, and machine learning. At the low level processing, we improve upon our previously proposed state-of-the-art tennis ball tracking algorithm and employ audio signal processing techniques to detect key events and construct features for classifying the events. At high level analysis, we model event classification as a sequence labelling problem, and investigate four machine learning techniques using simulated event sequences. Finally, we evaluate our proposed approach on three real world tennis games, and discuss the interplay between audio, vision and learning. To the best of our knowledge, our system is the only one that can annotate tennis game at such a detailed level

TennisSense: a platform for extracting semantic information from multi-camera tennis data

In this paper, we introduce TennisSense, a technology platform for the digital capture, analysis and retrieval of tennis training and matches. Our algorithms for extracting useful metadata from the overhead court camera are described and evaluated. We track the tennis ball using motion images for ball candidate detection and then link ball candidates into locally linear tracks. From these tracks we can infer when serves and rallies take place. Using background subtraction and hysteresis-type blob tracking, we track the tennis players positions. The performance of both modules is evaluated using ground-truthed data. The extracted metadata provides valuable information for indexing and efficient browsing of hours of multi-camera tennis footage and we briefly illustrative how this data is used by our tennis-coach playback interface

Towards Structured Analysis of Broadcast Badminton Videos

Sports video data is recorded for nearly every major tournament but remains archived and inaccessible to large scale data mining and analytics. It can only be viewed sequentially or manually tagged with higher-level labels which is time consuming and prone to errors. In this work, we propose an end-to-end framework for automatic attributes tagging and analysis of sport videos. We use commonly available broadcast videos of matches and, unlike previous approaches, does not rely on special camera setups or additional sensors. Our focus is on Badminton as the sport of interest. We propose a method to analyze a large corpus of badminton broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective badminton strokes. We evaluate the performance on 10 Olympic matches with 20 players and achieved 95.44% point segmentation accuracy, 97.38% player detection score ([email protected]), 97.98% player identification accuracy, and stroke segmentation edit scores of 80.48%. We further show that the automatically annotated videos alone could enable the gameplay analysis and inference by computing understandable metrics such as player's reaction time, speed, and footwork around the court, etc.Comment: 9 page

Image segmentation and feature extraction for recognizing strokes in tennis game videos

This paper addresses the problem of recognizing human actions from video. Particularly, the case of recognizing events in tennis game videos is analyzed. Driven by our domain knowledge, a robust player segmentation algorithm is developed real video data. Further, we introduce a number of novel features to be extracted for our particular application. Different feature combinations are investigated in order to find the optimal one. Finally, recognition results for different classes of tennis strokes using automatic learning capability of Hidden Markov Models (HMMs) are presented. The experimental results demonstrate that our method is close to realizing statistics of tennis games automatically using ordinary TV broadcast videos

Content-based Video Retrieval by Integrating Spatio-Temporal and Stochastic Recognition of Events

As amounts of publicly available video data grow the need to query this data efficiently becomes significant. Consequently content-based retrieval of video data turns out to be a challenging and important problem. We address the specific aspect of inferring semantics automatically from raw video data. In particular, we introduce a new video data model that supports the integrated use of two different approaches for mapping low-level features to high-level concepts. Firstly, the model is extended with a rule-based approach that supports spatio-temporal formalization of high-level concepts, and then with a stochastic approach. Furthermore, results on real tennis video data are presented, demonstrating the validity of both approaches, as well us advantages of their integrated us

PENERAPAN OBJECT TRACKING DENGAN METODE ADAPTIVE PARTICLE FILTER UNTUK PELACAKAN BOLA PADA PERMAINAN

Data pergerakan bola dapat dimanfaatkan sebagai panduan untuk mengamati kejadian-kejadian pada pertandingan tenis yang telah berlangsung. Namun, untuk mendapatkan data pergerakan bola dari video pertandingan rentan terjadi kesalahan dalam pendeteksian objek, sehingga data yang dihasilkan terdapat noise. Berdasarkan alasan tesebut, penulis melakukan mining terhadap video pertandingan bola tenis dengan pendekatan object tracking, sehingga kesalahan deteksi ketika mendeteksi bola dapat dikurangi. Pendekatan tersebut diwujudkan dengan merancang model pelacakan bola dengan metode circle hough transform untuk mendeteksi lingkaran, kemudian dilanjutkan dengan metode pelacakan adaptive particle filter yang berfungsi untuk menghilangkan noise yang dihasilkan ketika melakukan deteksi lingkaran. Model tersebut dijalankan melalui proses-proses yang diantaranya adalah segmentasi citra, deteksi lingkaran, pelacakan objek dan diakhiri dengan koreksi lintasan. Model yang dirancang kemudian diimplementasikan pada bahasa pemrograman Phyton dan library OpenCV. Tahap terakhir dalam penelitian ini adalah melakukan eksperimen, eksperimen ini bertujuan untuk mendapatkan parameter masukan terbaik pada perangkat lunak, sehingga dapat diketahui efektifitas dari model yang telah diimplementasikan. Hasil eksperimen menunjukan bahwa video dengan jenis siaran pada lapangan hard court outdoor menghasilkan keluaran terbaik dengan rata-rata error sebesar 0,344, sedangkan hasil pengujian pada parameter lainnya harus disesuaikan dengan jenis video masukan agar mendapat error minimal.----------Ball movement data can be utilized as a guide for observing the events on the tennis matches that has lasted. However, the movement of the ball to get the data from the video game of the vulnerable object detection in error, so that the resulting data there is noise. Based on the reasons are, the author does mining against video game tennis ball with object tracking approach, so the error detection when it detects the ball can be reduced. The approach embodied by designing a model tracking ball with hough transform for circle method to detect circles, then proceed with adaptive particle filter tracking method that serves to eliminate noise generated when the detection loop. The model is run through processes such as image segmentation, object tracking, circle detection and end with correction trajectory. Model designed then implemented in the programming language Python and OpenCV library. The last stage in this research is doing experiments, this experiment aims to get the best input parameters in the software, so it can be known to the effectiveness of the model that has been implemented. Experimental results show that the type of video broadcast on an outdoor hard court field produce the best output with an average error of 0.344, whereas the test results on the other parameters must be adjusted to the type of video input so that it gets the error minimal