Semantic analysis of field sports video using a petri-net of audio-visual concepts

The most common approach to automatic summarisation and highlight detection in sports video is to train an automatic classifier to detect semantic highlights based on occurrences of low-level features such as action replays, excited commentators or changes in a scoreboard. We propose an alternative approach based on the detection of perception concepts (PCs) and the construction of Petri-Nets which can be used for both semantic description and event detection within sports videos. Low-level algorithms for the detection of perception concepts using visual, aural and motion characteristics are proposed, and a series of Petri-Nets composed of perception concepts is formally defined to describe video content. We call this a Perception Concept Network-Petri Net (PCN-PN) model. Using PCN-PNs, personalized high-level semantic descriptions of video highlights can be facilitated and queries on high-level semantics can be achieved. A particular strength of this framework is that we can easily build semantic detectors based on PCN-PNs to search within sports videos and locate interesting events. Experimental results based on recorded sports video data across three types of sports games (soccer, basketball and rugby), and each from multiple broadcasters, are used to illustrate the potential of this framework

Social Network Analysis using Cultural Algorithms and its Variants

Finding relationships between social entities and discovering the underlying structures of networks are fundamental tasks for analyzing social networks. In recent years, various methods have been suggested to study these networks efficiently, however, due to the dynamic and complex nature that these networks have, a lot of open problems still exist in the field. The aim of this research is to propose an integrated computational model to study the structure and behavior of the complex social network. The focus of this research work is on two major classic problems in the field which are called community detection and link prediction. Moreover, a problem of population adaptation through knowledge migration in real-life social systems has been identified to model and study through the proposed method. To the best of our knowledge, this is the first work in the field which is exploring this concept through this approach. In this research, a new adaptive knowledge-based evolutionary framework is defined to investigate the structure of social networks by adopting a multi-population cultural algorithm. The core of the model is designed based on a unique community-oriented approach to estimate the existence of a relationship between social entities in the network. In each evolutionary cycle, the normative knowledge is shaped through the extraction of the topological knowledge from the structure of the network. This source of knowledge is utilized for the various network analysis tasks such as estimating the quality of relation between social entities, related studies regarding the link prediction, population adaption, and knowledge formation. The main contributions of this work can be summarized in introducing a novel method to define, extract and represent different sources of knowledge from a snapshot of a given network to determine the range of the optimal solution, and building a probability matrix to show the quality of relations between pairs of actors in the system. Introducing a new similarity metric, utilizing the prior knowledge in dynamic social network analysis and study the co-evolution of societies in a case of individual migration are another major contributions of this work. According to the obtained results, utilizing the proposed approach in community detection problem can reduce the search space size by 80%. It also can improve the accuracy of the search process in high dense networks by up to 30% compared with the other well-known methods. Addressing the link prediction problem through the proposed approach also can reach the comparable results with other methods and predict the next state of the system with a notably high accuracy. In addition, the obtained results from the study of population adaption through knowledge migration indicate that population with prior knowledge about an environment can adapt themselves to the new environment faster than the ones who do not have this knowledge if the level of changes between the two environments is less than 25%. Therefore, utilizing this approach in dynamic social network analysis can reduce the search time and space significantly (up to above 90%), if the snapshots of the system are taken when the level of changes in the network structure is within 25%. In summary, the experimental results indicate that this knowledge-based approach is capable of exploring the evolution and structure of the network with the high level of accuracy while it improves the performance by reducing the search space and processing time

Wavelet methods and statistical applications: network security and bioinformatics

Wavelet methods possess versatile properties for statistical applications. We would like to explore the advantages of using wavelets in the analyses in two different research areas. First of all, we develop an integrated tool for online detection of network anomalies. We consider statistical change point detection algorithms, for both local changes in the variance and for jumps detection, and propose modified versions of these algorithms based on moving window techniques. We investigate performances on simulated data and on network traffic data with several superimposed attacks. All detection methods are based on wavelet packets transformations. We also propose a Bayesian model for the analysis of high-throughput data where the outcome of interest has a natural ordering. The method provides a unified approach for identifying relevant markers and predicting class memberships. This is accomplished by building a stochastic search variable selection method into an ordinal model. We apply the methodology to the analysis of proteomic studies in prostate cancer. We explore wavelet-based techniques to remove noise from the protein mass spectra. The goal is to identify protein markers associated with prostate-specific antigen (PSA) level, an ordinal diagnostic measure currently used to stratify patients into different risk groups

An audio-based sports video segmentation and event detection algorithm

In this paper, we present an audio-based event detection algorithm shown to be effective when applied to Soccer video. The main benefit of this approach is the ability to recognise patterns that display high levels of crowd response correlated to key events. The soundtrack from a Soccer sequence is first parameterised using Mel-frequency Cepstral coefficients. It is then segmented into homogenous components using a windowing algorithm with a decision process based on Bayesian model selection. This decision process eliminated the need for defining a heuristic set of rules for segmentation. Each audio segment is then labelled using a series of Hidden Markov model (HMM) classifiers, each a representation of one of 6 predefined semantic content classes found in Soccer video. Exciting events are identified as those segments belonging to a crowd cheering class. Experimentation indicated that the algorithm was more effective for classifying crowd response when compared to traditional model-based segmentation and classification techniques

Mathematical and computer modeling of electro-optic systems using a generic modeling approach

The conventional approach to modelling electro-optic sensor systems is to develop separate models for individual systems or classes of system, depending on the detector technology employed in the sensor and the application. However, this ignores commonality in design and in components of these systems. A generic approach is presented for modelling a variety of sensor systems operating in the infrared waveband that also allows systems to be modelled with different levels of detail and at different stages of the product lifecycle. The provision of different model types (parametric and image-flow descriptions) within the generic framework can allow valuable insights to be gained

A detection theory account of change detection

Previous studies have suggested that visual short-term memory (VSTM) has a storage limit of approximately four items. However, the type of high-threshold (HT) model used to derive this estimate is based on a number of assumptions that have been criticized in other experimental paradigms (e.g., visual search). Here we report findings from nine experiments in which VSTM for color, spatial frequency, and orientation was modeled using a signal detection theory (SDT) approach. In Experiments 1-6, two arrays composed of multiple stimulus elements were presented for 100 ms with a 1500 ms ISI. Observers were asked to report in a yes/no fashion whether there was any difference between the first and second arrays, and to rate their confidence in their response on a 1-4 scale. In Experiments 1-3, only one stimulus element difference could occur (T = 1) while set size was varied. In Experiments 4-6, set size was fixed while the number of stimuli that might change was varied (T = 1, 2, 3, and 4). Three general models were tested against the receiver operating characteristics generated by the six experiments. In addition to the HT model, two SDT models were tried: one assuming summation of signals prior to a decision, the other using a max rule. In Experiments 7-9, observers were asked to directly report the relevant feature attribute of a stimulus presented 1500 ms previously, from an array of varying set size. Overall, the results suggest that observers encode stimuli independently and in parallel, and that performance is limited by internal noise, which is a function of set size

Video information retrieval using objects and ostensive relevance feedback

In this paper, we present a brief overview of current approaches to video information retrieval (IR) and we highlight its limitations and drawbacks in terms of satisfying user needs. We then describe a method of incorporating object-based relevance feedback into video IR which we believe opens up new possibilities for helping users find information in video archives. Following this we describe our own work on shot retrieval from video archives which uses object detection, object-based relevance feedback and a variation of relevance feedback called ostensive RF which is particularly appropriate for this type of retrieval