Detection of Ball Hits in a Tennis Game Using Audio and Visual Information

Abstract-In this paper we describe a framework to improve the detection of ball hit events in tennis games by combining audio and visual information. Detection of the presence and timing of these events is crucial for the understanding of the game. However, neither modality on its own gives satisfactory results: audio information is often corrupted by noise and also suffers from acoustic mismatch between the training and test data, and visual information is corrupted by complex backgrounds, camera calibration, and the presence of multiple moving objects. Our approach is to first attempt to track the ball visually and hence estimate a sequence of candidate positions for the ball, and to then locate putative ball hits by analysing the ball's position in this trajectory. To handle the severe interferences caused by false ball candidates, we smooth the trajectory by using linear regression and removing the frames where there are no candidates. We use Gaussian mixture models to generate estimates of the times of hits using the audio information, and then integrate these two sources of information in a probabilistic framework. Testing our approach on three complete tennis games shows significant improvements in detection over a range of conditions when compared with using a single modality

A maximum a posteriori probability viterbi data association algorithm for ball tracking in sports video

In this paper, we derive a data association algorithm for object tracking in a maximum a posteriori framework: the output of the algorithm is the sequence of measurement-totarget associations with maximum a posteriori probability. We model the object motion as a Markov process, and solve this otherwise combinatorially complex problem efficiently by applying the Viterbi algorithm. A method for combining forward and backward tracking results is also developed, to recover from tracking errors caused by abrupt motion changes of the object. The proposed algorithm is applied to broadcast tennis video to track a tennis ball. Experiments show that its performance is comparable to that of a computationally more expensive particle-filter-based algorithm. 1