Shot Classification in Broadcast Soccer Video

In this paper, we present an effective hierarchical shot classification scheme for broadcast soccer video. We first partition a video into replay and non-replay shots with replay logo detection. Then, non-replay shots are further classified into Long, Medium, Close-up or Out-field types with color and texture features based on a decision tree. We tested the method on real broadcast FIFA soccer videos, and the experimental results demonstrate its effectiveness.

An Overview of Multimodal Techniques for the Characterization of Sport Programmes

The problem of content characterization of sports videos is of great interest because sports video appeals to large audiences and its efficient distribution over various networks should contribute to widespread usage of multimedia services. In this paper we analyze several techniques proposed in literature for content characterization of sports videos. We focus this analysis on the typology of the signal (audio, video, text captions, ...) from which the low-level features are extracted. First we consider the techniques based on visual information, then the methods based on audio information, and finally the algorithms based on audio-visual cues, used in a multi-modal fashion. This analysis shows that each type of signal carries some peculiar information, and the multi-modal approach can fully exploit the multimedia information associated to the sports video. Moreover, we observe that the characterization is performed either considering what happens in a specific time segment, observing therefore the features in a "static" way, or trying to capture their "dynamic" evolution in time. The effectiveness of each approach depends mainly on the kind of sports it relates to, and the type of highlights we are focusing on

Automatic Summarization of Soccer Highlights Using Audio-visual Descriptors

Automatic summarization generation of sports video content has been object of great interest for many years. Although semantic descriptions techniques have been proposed, many of the approaches still rely on low-level video descriptors that render quite limited results due to the complexity of the problem and to the low capability of the descriptors to represent semantic content. In this paper, a new approach for automatic highlights summarization generation of soccer videos using audio-visual descriptors is presented. The approach is based on the segmentation of the video sequence into shots that will be further analyzed to determine its relevance and interest. Of special interest in the approach is the use of the audio information that provides additional robustness to the overall performance of the summarization system. For every video shot a set of low and mid level audio-visual descriptors are computed and lately adequately combined in order to obtain different relevance measures based on empirical knowledge rules. The final summary is generated by selecting those shots with highest interest according to the specifications of the user and the results of relevance measures. A variety of results are presented with real soccer video sequences that prove the validity of the approach

Video analysis for replay detection in sport events

The postproduction cost of a sport event video requires lots of resources dedication and expenses of time trying to find the best highlights moments that will be used, for instance, in creating the summary of the event. This process can be optimized and improved in efficiency. During the event, the most important moments are repeated to offer to the audience the outstanding scene several times and from different points of view. The objective of the project is to automatically find the replays in live or pre-recorded transmission and accelerating the post-production process. The results will be part of the project CENIT-E BUSCAMEDIA CEN20091026, developed in the studios of Televisió de Catalunya (TVC) and which are focused on automated generation through content analysis. A software has been developed to detect the replays for different kind of sport events, principally soccer. This, implements many operation modes detailed during this report. We find from a mode rather manual to a full automatic mode, and moreover the percentages of success are presented after testing then using some videos from the TVC database. The structure of the work has been divided into five major sections: The first chapter begins by introducing us to the context in which it places the project, proposing the objectives to be achieved, and also discusses the data and tools used for their development. Subsequently, there is exposed the state of the art with a collection of methods used for the detection of repeats, which are the foundations on which we developed our methodology. The third chapter is the longest and complex. This contains the entire process of experimentation and improvements planned from the inception until the system implemented. In addition, the following section talks about the technical and exhibits the algorithm implemented in form of block diagram detailing all the operation modes. Finally, the last chapter contains all the results and conclusions after applying the algorithm on a set of videos taken from the database o f TVC, as well as its application in other areas such as Formula1 videos.Català: El cost de postproducció d‟un vídeo d‟un esdeveniment esportiu requereix la dedicació de molt recursos i temps en situar sobre el vídeo els moments destacats que s‟utilitzaran, per exemple, en la creació del resum del l‟esdeveniment. Aquest procés pot ser optimitzat i millorat en quant a eficiència. Durant el transcurs d‟aquest, els moments més destacats solen repetir-se per tal d‟oferir l‟escena varies vegades i des de diferents punts de vista. Aquest treball té com a objectiu principal la detecció d‟aquestes repeticions per tal d‟identificar els moments destacats i senyalitzar-ho per tal d‟agilitzar el procés de postproducció. Els resultats formaran part del projecte CENIT-E BUSCAMEDIA CEN20091026, desenvolupat als estudis de Televisió de Catalunya (TVC) i que tracta de generació automàtica mitjançant l‟anàlisi de continguts. S‟ha desenvolupat un software capaç de detectar les repeticions que apareixen en diferents tipus d‟esdeveniments esportius, principalment futbol. Aquest, implementa diferents modes d‟operació que veurem explicats en detall al llarg de la memòria. Trobem des d‟un mode mes aviat manual fins a un completament automàtic i es mostren els percentatge d‟èxit obtinguts després de realitzar proves funcionals utilitzant vídeos de la basa de dades de TVC. L‟estructura del treball s‟ha dividit en cinc grans apartats: El primer capítol comença introduint-nos en el context on es situa el projecte, proposant els objectius que es volen assolir, així com també parla sobre les dades i eines utilitzades pel seu desenvolupament. Posteriorment, s‟exposarà l‟estat de l‟art amb un recull dels mètodes més emprats per la detecció de repeticions i que han estat els fonaments sobre els que hem desenvolupat la nostra metodologia. El tercer capítol és el més llarg i complex. Conté tot el procés d‟experimentació i millores plantejat des de l‟inici fins arribar al sistema que s‟ha implementat. D‟altra banda, el següent apartat ens fa cinc cèntims de la part tècnica i exposa en forma de diagrama de blocs l‟algorisme implementat, explicant els mètodes possibles per utilitzar el sistema. Finalment, l‟últim capítol recull tot els resultats i conclusions extretes després d‟aplicar l‟algorisme en un conjunt de vídeos extrets de la base de dades de TVC, així com també l‟aplicació del mateix en altres àmbits com vídeos de Formula1

Audiovisual framework for automatic soccer highlights generation

Extracting low-level and mid-level descriptors from a soccer match to generate a summary of soccer highlights.Automatic generation of sports highlights from recorded audiovisual content has been object of great interest in recent years. The problem is indeed especially important in the production of second and third division highlights videos where the quantity of raw material is significant and does not contain manual annotations. In this thesis, a new approach for automatic generation of soccer highlights is proposed. The approach is based on the segmentation of the video sequence into shots that will be further ana- lyzed to determine its relevance and interest. For every video shot a set of low and mid level audio-visual descriptors are computed and combined in order to obtain different relevance measures based on empirical knowledge rules. The final summary is generated by selecting those shots with highest interest according to the specifications of the user and the results of relevance measures. The main novelties of this work have been the temporal combination of two shot boundary detectors; the selection of keyframes using motion and color features; the generation of new soccer audio mid-level descriptors; the robust detection of soccer players; the employment of a novel object detection technique to spot goal-posts and finally, the creation of a flexible and user-friendly highlight gen- eration framework. The thesis is mainly devoted to the description of the global visual segmentation module, the selection of audiovisual descriptors and the general scheme for evaluating the measures of relevance. Several results have been produced using real soccer video sequences that prove the validity of the proposed framework

Audio-visual football video analysis, from structure detection to attention analysis

Sport video is an important video genre. Content-based sports video analysis attracts great interest from both industry and academic fields. A sports video is characterised by repetitive temporal structures, relatively plain contents, and strong spatio-temporal variations, such as quick camera switches and swift local motions. It is necessary to develop specific techniques for content-based sports video analysis to utilise these characteristics. For an efficient and effective sports video analysis system, there are three fundamental questions: (1) what are key stories for sports videos; (2) what incurs viewer’s interest; and (3) how to identify game highlights. This thesis is developed around these questions. We approached these questions from two different perspectives and in turn three research contributions are presented, namely, replay detection, attack temporal structure decomposition, and attention-based highlight identification. Replay segments convey the most important contents in sports videos. It is an efficient approach to collect game highlights by detecting replay segments. However, replay is an artefact of editing, which improves with advances in video editing tools. The composition of replay is complex, which includes logo transitions, slow motions, viewpoint switches and normal speed video clips. Since logo transition clips are pervasive in game collections of FIFA World Cup 2002, FIFA World Cup 2006 and UEFA Championship 2006, we take logo transition detection as an effective replacement of replay detection. A two-pass system was developed, including a five-layer adaboost classifier and a logo template matching throughout an entire video. The five-layer adaboost utilises shot duration, average game pitch ratio, average motion, sequential colour histogram and shot frequency between two neighbouring logo transitions, to filter out logo transition candidates. Subsequently, a logo template is constructed and employed to find all transition logo sequences. The precision and recall of this system in replay detection is 100% in a five-game evaluation collection. An attack structure is a team competition for a score. Hence, this structure is a conceptually fundamental unit of a football video as well as other sports videos. We review the literature of content-based temporal structures, such as play-break structure, and develop a three-step system for automatic attack structure decomposition. Four content-based shot classes, namely, play, focus, replay and break were identified by low level visual features. A four-state hidden Markov model was trained to simulate transition processes among these shot classes. Since attack structures are the longest repetitive temporal unit in a sports video, a suffix tree is proposed to find the longest repetitive substring in the label sequence of shot class transitions. These occurrences of this substring are regarded as a kernel of an attack hidden Markov process. Therefore, the decomposition of attack structure becomes a boundary likelihood comparison between two Markov chains. Highlights are what attract notice. Attention is a psychological measurement of “notice ”. A brief survey of attention psychological background, attention estimation from vision and auditory, and multiple modality attention fusion is presented. We propose two attention models for sports video analysis, namely, the role-based attention model and the multiresolution autoregressive framework. The role-based attention model is based on the perception structure during watching video. This model removes reflection bias among modality salient signals and combines these signals by reflectors. The multiresolution autoregressive framework (MAR) treats salient signals as a group of smooth random processes, which follow a similar trend but are filled with noise. This framework tries to estimate a noise-less signal from these coarse noisy observations by a multiple resolution analysis. Related algorithms are developed, such as event segmentation on a MAR tree and real time event detection. The experiment shows that these attention-based approach can find goal events at a high precision. Moreover, results of MAR-based highlight detection on the final game of FIFA 2002 and 2006 are highly similar to professionally labelled highlights by BBC and FIFA

Integrated analysis of audiovisual signals and external information sources for event detection in team sports video

Ph.DDOCTOR OF PHILOSOPH

Audiovisual processing for sports-video summarisation technology

In this thesis a novel audiovisual feature-based scheme is proposed for the automatic summarization of sports-video content The scope of operability of the scheme is designed to encompass the wide variety o f sports genres that come under the description ‘field-sports’. Given the assumption that, in terms of conveying the narrative of a field-sports-video, score-update events constitute the most significant moments, it is proposed that their detection should thus yield a favourable summarisation solution. To this end, a generic methodology is proposed for the automatic identification of score-update events in field-sports-video content. The scheme is based on the development of robust extractors for a set of critical features, which are shown to reliably indicate their locations. The evidence gathered by the feature extractors is combined and analysed using a Support Vector Machine (SVM), which performs the event detection process. An SVM is chosen on the basis that its underlying technology represents an implementation of the latest generation of machine learning algorithms, based on the recent advances in statistical learning. Effectively, an SVM offers a solution to optimising the classification performance of a decision hypothesis, inferred from a given set of training data. Via a learning phase that utilizes a 90-hour field-sports-video trainmg-corpus, the SVM infers a score-update event model by observing patterns in the extracted feature evidence. Using a similar but distinct 90-hour evaluation corpus, the effectiveness of this model is then tested genencally across multiple genres of fieldsports- video including soccer, rugby, field hockey, hurling, and Gaelic football. The results suggest that in terms o f the summarization task, both high event retrieval and content rejection statistics are achievable