Statistical Skimming of Feature Films

We present a statistical framework based on Hidden Markov Models (HMMs) for skimming feature films. A chain of HMMs is used to model subsequent story units: HMM states represent different visual-concepts, transitions model the temporal dependencies in each story unit, and stochastic observations are given by single shots. The skim is generated as an observation sequence, where, in order to privilege more informative segments for entering the skim, shots are assigned higher probability of observation if endowed with salient features related to specific film genres. The effectiveness of the method is demonstrated by skimming the first thirty minutes of a wide set of action and dramatic movies, in order to create previews for users useful for assessing whether they would like to see that movie or not, but without revealing the movie central part and plot details. Results are evaluated and compared through extensive user tests in terms of metrics that estimate the content representational value of the obtained video skims and their utility for assessing the user's interest in the observed movie

Audio-visual football video analysis, from structure detection to attention analysis

Sport video is an important video genre. Content-based sports video analysis attracts great interest from both industry and academic fields. A sports video is characterised by repetitive temporal structures, relatively plain contents, and strong spatio-temporal variations, such as quick camera switches and swift local motions. It is necessary to develop specific techniques for content-based sports video analysis to utilise these characteristics. For an efficient and effective sports video analysis system, there are three fundamental questions: (1) what are key stories for sports videos; (2) what incurs viewer’s interest; and (3) how to identify game highlights. This thesis is developed around these questions. We approached these questions from two different perspectives and in turn three research contributions are presented, namely, replay detection, attack temporal structure decomposition, and attention-based highlight identification. Replay segments convey the most important contents in sports videos. It is an efficient approach to collect game highlights by detecting replay segments. However, replay is an artefact of editing, which improves with advances in video editing tools. The composition of replay is complex, which includes logo transitions, slow motions, viewpoint switches and normal speed video clips. Since logo transition clips are pervasive in game collections of FIFA World Cup 2002, FIFA World Cup 2006 and UEFA Championship 2006, we take logo transition detection as an effective replacement of replay detection. A two-pass system was developed, including a five-layer adaboost classifier and a logo template matching throughout an entire video. The five-layer adaboost utilises shot duration, average game pitch ratio, average motion, sequential colour histogram and shot frequency between two neighbouring logo transitions, to filter out logo transition candidates. Subsequently, a logo template is constructed and employed to find all transition logo sequences. The precision and recall of this system in replay detection is 100% in a five-game evaluation collection. An attack structure is a team competition for a score. Hence, this structure is a conceptually fundamental unit of a football video as well as other sports videos. We review the literature of content-based temporal structures, such as play-break structure, and develop a three-step system for automatic attack structure decomposition. Four content-based shot classes, namely, play, focus, replay and break were identified by low level visual features. A four-state hidden Markov model was trained to simulate transition processes among these shot classes. Since attack structures are the longest repetitive temporal unit in a sports video, a suffix tree is proposed to find the longest repetitive substring in the label sequence of shot class transitions. These occurrences of this substring are regarded as a kernel of an attack hidden Markov process. Therefore, the decomposition of attack structure becomes a boundary likelihood comparison between two Markov chains. Highlights are what attract notice. Attention is a psychological measurement of “notice ”. A brief survey of attention psychological background, attention estimation from vision and auditory, and multiple modality attention fusion is presented. We propose two attention models for sports video analysis, namely, the role-based attention model and the multiresolution autoregressive framework. The role-based attention model is based on the perception structure during watching video. This model removes reflection bias among modality salient signals and combines these signals by reflectors. The multiresolution autoregressive framework (MAR) treats salient signals as a group of smooth random processes, which follow a similar trend but are filled with noise. This framework tries to estimate a noise-less signal from these coarse noisy observations by a multiple resolution analysis. Related algorithms are developed, such as event segmentation on a MAR tree and real time event detection. The experiment shows that these attention-based approach can find goal events at a high precision. Moreover, results of MAR-based highlight detection on the final game of FIFA 2002 and 2006 are highly similar to professionally labelled highlights by BBC and FIFA

Laboratory Studies of Hypervelocity Impacts on Solar System Analogues

Impact cratering and asteroid collisions are major processes throughout the Solar System. Although previous collision-related impact investigations exist (Flynn et al. 2015, Holsapple et al. 2002 and Burchell et al. 1998 are good examples), in the works covering this broad range of investigation, the targets are non-rotating (for the purposes of catastrophic disruption) and different temperature conditions are not considered (for impact cratering). Accordingly, I present experimental processes and data, regarding hypervelocity impact experiments into analogues of (1) rotating asteroids and (2) temperature dependant terrestrial planetary rock. During the course of this work, it was necessary to develop new apparatus and new experimental techniques such as three separate target holders to aid in both catastrophic disruption and heated impact projects, a 3-dimensional model analysis of craters and a completely new, statistically robust, technique to determine a complete crater profile called the KDM method where KDM is Kinnear-Deller-Morris. The main result from this work showed that during an asteroid impact collision where the asteroid is not rotating, the impact energy density for catastrophic disruption is Q*static = 1442 ± 90 J kg-1. However, when the target asteroid was rotating, the condition Q*rotation = 1097 ± 296 J kg-1. The mean value of Q* had thus reduced, but the spread in the data on individual experiments was larger. This leads to two conclusions. The mean value for Q*, based on measurements of many impacts, falls, due to the internal forces acting in the body which are associated with the rotation. This energy term reduction means that the amount of energy to instigate catastrophic disruption is lower and that a rotating asteroid is effectively weaker upon impact than a stationary asteroid. However, the spread in the results indicates that this is not a uniform process, and an individual result for Q* for a rotating or spinning target may be spread over a large range. For the temperature related impacts, as the targets were heated to approximately 1000 K, the target rocks showed an impact dependence more similar to a plastic phase-state than to solidus, due to being held close to temperatures associated with semi-plastic phases. Basalt impact craters displayed this relationship greatest with crater sizes becoming smaller at the higher temperature ranges but larger in the colder brittle solidus temperatures, partly explained in experiments by increased spallation

MAC-REALM: A video content feature extraction and modelling framework

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.A consequence of the ‘data deluge’ is the exponential increase in digital video footage, while the ability to find relevant video clips diminishes. Traditional text based search engines are no longer optimal for searching, as they cannot provide a granular search of the content inside video footage. To be able to search the video in a content based manner, the content features of the video need to be extracted and modelled into a content model, which can then act as a searchable proxy for the video content. This thesis focuses on the extraction of syntactic and semantic content features and content modelling, using machine driven processes, with either little or no user interaction. Our abstract framework design extracts syntactic and semantic content features and compiles them into an integrated content model. The framework integrates a four plane strategy that consists of a pre-processing plane that removes redundant data and filters the media to improve the feature extraction properties of the media; a syntactic feature extraction plane that extracts low level syntactic feature and mid-level syntactic features that have semantic attributes; a semantic relationship analysis and linkage plane, where the spatial and temporal relationships of all the content features are defined, and finally a content modelling stage where the syntactic and semantic content features are integrated into a content model. Each of the four planes can be split into three layers namely, the content layer, where the content to be processed is stored; the application layer, where the content is converted into content descriptions, and the MPEG-7 layer, where content descriptions are serialised. Using MPEG-7 standards to produce the content model will provide wide-ranging interoperability, while facilitating granular multi-content type searches. The framework is aiming to ‘bridge’ the semantic gap, by integrating the syntactic and semantic content features from extraction through to modelling. The design of the framework has been implemented into a prototype called MAC-REALM, which has been tested and evaluated for its effectiveness to extract and model content features. Conclusions are drawn about the research output as a whole and whether they have met the objectives. Finally, future work is presented on how concept detection and crowd sourcing can be used with MAC-REALM

Activity Recognition Based on Micro-Doppler Signature with In-Home Wi-Fi

Device free activity recognition and monitoring has become a promising research area with increasing public interest in pattern of life monitoring and chronic health conditions. This paper proposes a novel framework for inhome Wi-Fi signal-based activity recognition in e-healthcare applications using passive micro-Doppler (m-D) signature classification. The framework includes signal modeling, Doppler extraction and m-D classification. A data collection campaign was designed to verify the framework where six m-D signatures corresponding to typical daily activities are sucessfully detected and classified using our software defined radio (SDR) demo system. Analysis of the data focussed on potential discriminative characteristics, such as maximum Doppler frequency and time duration of activity. Finally, a sparsity induced classifier is applied for adaptting the method in healthcare application scenarios and the results are compared with those from the well-known Support Vector Machine (SVM) method

Recent Trends in Computational Intelligence

Traditional models struggle to cope with complexity, noise, and the existence of a changing environment, while Computational Intelligence (CI) offers solutions to complicated problems as well as reverse problems. The main feature of CI is adaptability, spanning the fields of machine learning and computational neuroscience. CI also comprises biologically-inspired technologies such as the intellect of swarm as part of evolutionary computation and encompassing wider areas such as image processing, data collection, and natural language processing. This book aims to discuss the usage of CI for optimal solving of various applications proving its wide reach and relevance. Bounding of optimization methods and data mining strategies make a strong and reliable prediction tool for handling real-life applications

Informative Data Fusion: Beyond Canonical Correlation Analysis

Multi-modal data fusion is a challenging but common problem arising in fields such as economics, statistical signal processing, medical imaging, and machine learning. In such applications, we have access to multiple datasets that use different data modalities to describe some system feature. Canonical correlation analysis (CCA) is a multidimensional joint dimensionality reduction algorithm for exactly two datasets. CCA finds a linear transformation for each feature vector set such that the correlation between the two transformed feature sets is maximized. These linear transformations are easily found by solving the SVD of a matrix that only involves the covariance and cross-covariance matrices of the feature vector sets. When these covariance matrices are unknown, an empirical version of CCA substitutes sample covariance estimates formed from training data. However, when the number of training samples is less than the combined dimension of the datasets, CCA fails to reliably detect correlation between the datasets. This thesis explores the the problem of detecting correlations from data modeled by the ubiquitous signal-plus noise data model. We present a modification to CCA, which we call informative CCA (ICCA) that first projects each dataset onto a low-dimensional informative signal subspace. We verify the superior performance of ICCA on real-world datasets and argue the optimality of trim-then-fuse over fuse-then-trim correlation analysis strategies. We provide a significance test for the correlations returned by ICCA and derive improved estimates of the population canonical vectors using insights from random matrix theory. We then extend the analysis of CCA to regularized CCA (RCCA) and demonstrate that setting the regularization parameter to infinity results in the best performance and has the same solution as taking the SVD of the cross-covariance matrix of the two datasets. Finally, we apply the ideas learned from ICCA to multiset CCA (MCCA), which analyzes correlations for more than two datasets. There are multiple formulations of multiset CCA (MCCA), each using a different combination of objective function and constraint function to describe a notion of multiset correlation. We consider MAXVAR, provide an informative version of the algorithm, which we call informative MCCA (IMCCA), and demonstrate its superiority on a real-world dataset.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113419/1/asendorf_1.pd

Kepler Mission Stellar and Instrument Noise Properties

Kepler Mission results are rapidly contributing to fundamentally new discoveries in both the exoplanet and asteroseismology fields. The data returned from Kepler are unique in terms of the number of stars observed, precision of photometry for time series observations, and the temporal extent of high duty cycle observations. As the first mission to provide extensive time series measurements on thousands of stars over months to years at a level hitherto possible only for the Sun, the results from Kepler will vastly increase our knowledge of stellar variability for quiet solar-type stars. Here we report on the stellar noise inferred on the timescale of a few hours of most interest for detection of exoplanets via transits. By design the data from moderately bright Kepler stars are expected to have roughly comparable levels of noise intrinsic to the stars and arising from a combination of fundamental limitations such as Poisson statistics and any instrument noise. The noise levels attained by Kepler on-orbit exceed by some 50% the target levels for solar-type, quiet stars. We provide a decomposition of observed noise for an ensemble of 12th magnitude stars arising from fundamental terms (Poisson and readout noise), added noise due to the instrument and that intrinsic to the stars. The largest factor in the modestly higher than anticipated noise follows from intrinsic stellar noise. We show that using stellar parameters from galactic stellar synthesis models, and projections to stellar rotation, activity and hence noise levels reproduces the primary intrinsic stellar noise features.Comment: Accepted by ApJ; 26 pages, 20 figure