Examining the functionality of peripheral vision: From fundamental understandings to applied sport science

In sports, it is important not only to locate gaze on the right location to utilize the high acuity of foveal vision, but also to attend to other objects in the environment without looking directly at them, accordingly, using peripheral vision. Peripheral vision becomes especially important if, for example, the processing of information from more than one location (e.g. players) is decisive in making accurate decisions. Since such decisions generally must be made under high spatio-temporal demands, costly eye-movements might be advantageously avoided by using peripheral vision for information pick-up from multiple cues. In a series of studies, we aimed to translate the demands found in sports and to investigate the functionality of peripheral vision in a well-controlled experimental paradigm, the multiple object tracking (MOT) task. MOT was implemented in a dual task, along with an additional event-detection task. The present article first presents an overview of sport-specific studies focusing on the functionality of peripheral vision and following, summarizes a series of three published MOT studies. These studies show that peripheral vision is used for simultaneous target monitoring and target-change detection and that visual and attentional demands affect gaze anchoring and change-detection rates. Results also reveal a dysfunctionality of saccades, and further suggest an event- and distance-optimized gaze-anchoring position. In the final portion of this article, we derive specific applications for future sports-specific research. Specifically, we suggest to: (a) use dual-task situations in sport-specific settings, such as monitoring multiple players in soccer and playing a pass at specific moments, (b) investigate the costs of saccades in sports situations with high spatio-temporal demands, as in martial arts, and finally, (c) manipulate attentional and visual demands. For each of these avenues of research, we sketch sports-specific experiments currently being conducted in our research group

Content-based video indexing for sports applications using integrated multi-modal approach

This thesis presents a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple (audio-visual) modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s). The main objective is to contribute to the three major components of sports video indexing systems. The first component is a set of powerful techniques to extract audio-visual features and semantic contents automatically. The main purposes are to reduce manual annotations and to summarize the lengthy contents into a compact, meaningful and more enjoyable presentation. The second component is an expressive and flexible indexing technique that supports gradual index construction. Indexing scheme is essential to determine the methods by which users can access a video database. The third and last component is a query language that can generate dynamic video summaries for smart browsing and support user-oriented retrievals

Audio-visual football video analysis, from structure detection to attention analysis

Sport video is an important video genre. Content-based sports video analysis attracts great interest from both industry and academic fields. A sports video is characterised by repetitive temporal structures, relatively plain contents, and strong spatio-temporal variations, such as quick camera switches and swift local motions. It is necessary to develop specific techniques for content-based sports video analysis to utilise these characteristics. For an efficient and effective sports video analysis system, there are three fundamental questions: (1) what are key stories for sports videos; (2) what incurs viewer’s interest; and (3) how to identify game highlights. This thesis is developed around these questions. We approached these questions from two different perspectives and in turn three research contributions are presented, namely, replay detection, attack temporal structure decomposition, and attention-based highlight identification. Replay segments convey the most important contents in sports videos. It is an efficient approach to collect game highlights by detecting replay segments. However, replay is an artefact of editing, which improves with advances in video editing tools. The composition of replay is complex, which includes logo transitions, slow motions, viewpoint switches and normal speed video clips. Since logo transition clips are pervasive in game collections of FIFA World Cup 2002, FIFA World Cup 2006 and UEFA Championship 2006, we take logo transition detection as an effective replacement of replay detection. A two-pass system was developed, including a five-layer adaboost classifier and a logo template matching throughout an entire video. The five-layer adaboost utilises shot duration, average game pitch ratio, average motion, sequential colour histogram and shot frequency between two neighbouring logo transitions, to filter out logo transition candidates. Subsequently, a logo template is constructed and employed to find all transition logo sequences. The precision and recall of this system in replay detection is 100% in a five-game evaluation collection. An attack structure is a team competition for a score. Hence, this structure is a conceptually fundamental unit of a football video as well as other sports videos. We review the literature of content-based temporal structures, such as play-break structure, and develop a three-step system for automatic attack structure decomposition. Four content-based shot classes, namely, play, focus, replay and break were identified by low level visual features. A four-state hidden Markov model was trained to simulate transition processes among these shot classes. Since attack structures are the longest repetitive temporal unit in a sports video, a suffix tree is proposed to find the longest repetitive substring in the label sequence of shot class transitions. These occurrences of this substring are regarded as a kernel of an attack hidden Markov process. Therefore, the decomposition of attack structure becomes a boundary likelihood comparison between two Markov chains. Highlights are what attract notice. Attention is a psychological measurement of “notice ”. A brief survey of attention psychological background, attention estimation from vision and auditory, and multiple modality attention fusion is presented. We propose two attention models for sports video analysis, namely, the role-based attention model and the multiresolution autoregressive framework. The role-based attention model is based on the perception structure during watching video. This model removes reflection bias among modality salient signals and combines these signals by reflectors. The multiresolution autoregressive framework (MAR) treats salient signals as a group of smooth random processes, which follow a similar trend but are filled with noise. This framework tries to estimate a noise-less signal from these coarse noisy observations by a multiple resolution analysis. Related algorithms are developed, such as event segmentation on a MAR tree and real time event detection. The experiment shows that these attention-based approach can find goal events at a high precision. Moreover, results of MAR-based highlight detection on the final game of FIFA 2002 and 2006 are highly similar to professionally labelled highlights by BBC and FIFA

Automated Detection of Complex Tactical Patterns in Football—Using Machine Learning Techniques to Identify Tactical Behavior

Football tactics is a topic of public interest, where decisions are predominantly made based on gut instincts from domain-experts. Sport science literature often highlights the need for evidence-based research on football tactics, however the limited capabilities in modeling the dynamics of football has prevented researchers from gaining usable insights. Recent technological advances have made high quality football data more available and affordable. Particularly, positional data providing player and ball coordinates at every instance of a match can be combined with event data containing spatio-temporal information on any event taking place on the pitch (e.g. passes, shots, fouls). On the other hand, the application of machine learning methods to domain-specific problems yields a paradigm shift in many industries including sports. The need for more informed decisions as well as automating time consuming processes—accelerated by the availability of data—has motivated many scientific investigations in football analytics. This thesis is part of a research program combining methodologies from sports and data science to address the following problems: the synchronization of positional and event data, objectively quantifying offensive actions, as well as the detection of tactical patterns. Although various basic insights from the overall research program are integrated, this thesis focuses primarily on the latter one. Specifically, positional and event data are used to apply machine learning techniques to identify eight established tactical patterns in football: namely high-/mid-/low-block defending, build-up/attacking play in the offense, counterpressing and counterattacks during transitions, and patterns when defending corner-kicks, e.g. player-/zonal- or post-marking. For each pattern, we consolidate definitions with football experts and label large amounts of data manually using video recordings. The inter-labeler reliability is used to ensure that each pattern is well-defined. Unsupervised techniques are used for the purpose of exploration, and supervised machine learning methods based on expert-labeled data for the final detection. As an outlook, semi-supervised methods were used to reduce the labeling effort. This thesis proves that the detection of tactical patterns can optimize everyday processes in professional clubs, and leverage the domain of tactical analysis in sport science by gaining unseen insights. Additionally, we add value to the machine learning domain by evaluating recent methods in supervised and semi supervised machine learning on challenging, real-world problems

A dialogue model of the resolution of inter-individual conflicts : implications for computer based collaborative learning

As a result of empirical research into the benefits of peer interaction for learning and development, there is growing interest in designing systems to support collaborative learning. How best to do this for classroom learning is still unclear, and the aim of this thesis is to marry theory with empirical study to produce a set of guide-lines for informing the design of computer based collaborative learning.The resolution of inter-individual conflicts has long been proposed as a mechanism for learning in peer interaction and research indicates that in certain circumstances it does lead to learning. How this finding can be employed in the design of systems is problematic because researchers have typically not unpacked the notion of conflict and not explained how they are resolved.To overcome this a dialogue model of the resolution of inter-individual conflicts in joint planning is proposed. There are three key components in this model; the Task Representation, the Task Focus and the Dialogue focus. Using this model I begin to differentiate between three different types of conflict. Conflicts are caused by Task Representation Differences, Intersection Differences or Task Focus Differences . All three types of interindividual difference can be resolved with a set of discourse transactions and a set of internal resolution procedures. I propose that the resolution of all three can facilitate joint planning and subsequent individual planning.Three experiments are reported which investigate these claims. The first experiment shows that pairs performed better than individuals in a planning task and this beneficial effect carries over to subsequent individual planning. The proposed model is used to investigate this facilitative effect. A corpus of inter-individual conflicts is reported. Most of the conflicts identified in this corpus are either intersection differences or task focus differences and evidence is reported which supports the claim that their resolution can lead to learning. The second and third experiments investigate the claims made from the model about task representation differences and evidence was found to support them. The research reported in this thesis is then related to the design of software to support collaborative learning by presenting a set of guide-lines for informing their design. The work also has important implications for developmental psychology and classroom practice.