On Evidence Weighted Mixture Classification

2005 Joint Annual Meeting of the Interface and the Classification Society of North America, St. Louis, Missouri, 8-12 June 2005Calculation of the marginal likelihood or evidence is a problem central to model selection and model averaging in a Bayesian framework. Many sampling methods, especially (Reversible Jump) Markov chain Monte Carlo techniques, have been devised to avoid explicit calculation of the evidence, but they are limited to models with a common parameterisation. It is desirable to extend model averaging to models with disparate architectures and parameterisations. In this paper we present a straightforward general computational scheme for calculating the evidence, applicable to any model for which samples can be drawn from the posterior distribution of parameters conditioned on the data. The scheme is demonstrated on a simple feature subset selection example

Variational inference for robust sequential learning of multilayered perceptron neural network

U radu je prikazan i izveden novi sekvencijalni algoritam za obučavanje višeslojnog perceptrona u prisustvu autlajera. Autlajeri predstavljaju značajan problem, posebno ukoliko sprovodimo sekvencijalno obučavanje ili obučavanje u realnom vremenu. Linearizovani Kalmanov filtar robustan na autlajere (LKF-RA), je statistički generativni model u kome je matrica kovarijansi šuma merenja modelovana kao stohastički proces, a apriorna informacija usvojena kao inverzna Višartova raspodela. Izvođenje svih jednakosti je bazirano na prvim principima Bajesovske metodologije. Da bi se rešio korak modifikacije primenjen je varijacioni metod, u kome rešenje problema tražimo u familiji raspodela odgovarajuće funkcionalne forme. Eksperimentalni rezultati primene LKF-RA, dobijeni korišćenjem stvarnih vremenskih serija, pokazuju da je LKF-RA bolji od konvencionalnog linearizovanog Kalmanovog filtra u smislu generisanja niže greške na test skupu podataka. Prosečna vrednost poboljšanja određena u eksperimentalnom procesu je 7%.We derive a new sequential learning algorithm for Multilayered Perceptron (MLP) neural network robust to outliers. Presence of outliers in data results in failure of the model especially if data processing is performed on-line or in real time. Extended Kalman filter robust to outliers (EKF-OR) is probabilistic generative model in which measurement noise covariance is modeled as stochastic process over the set of symmetric positive-definite matrices in which prior is given as inverse Wishart distribution. Derivation of expressions comes straight form first principles, within Bayesian framework. Analytical intractability of Bayes' update step is solved using Variational Inference (VI). Experimental results obtained using real world stochastic data show that MLP network trained with proposed algorithm achieves low error and average improvement rate of 7% when compared directly to conventional EKF learning algorithm

Variational inference for robust sequential learning of multilayered perceptron neural network

U radu je prikazan i izveden novi sekvencijalni algoritam za obučavanje višeslojnog perceptrona u prisustvu autlajera. Autlajeri predstavljaju značajan problem, posebno ukoliko sprovodimo sekvencijalno obučavanje ili obučavanje u realnom vremenu. Linearizovani Kalmanov filtar robustan na autlajere (LKF-RA), je statistički generativni model u kome je matrica kovarijansi šuma merenja modelovana kao stohastički proces, a apriorna informacija usvojena kao inverzna Višartova raspodela. Izvođenje svih jednakosti je bazirano na prvim principima Bajesovske metodologije. Da bi se rešio korak modifikacije primenjen je varijacioni metod, u kome rešenje problema tražimo u familiji raspodela odgovarajuće funkcionalne forme. Eksperimentalni rezultati primene LKF-RA, dobijeni korišćenjem stvarnih vremenskih serija, pokazuju da je LKF-RA bolji od konvencionalnog linearizovanog Kalmanovog filtra u smislu generisanja niže greške na test skupu podataka. Prosečna vrednost poboljšanja određena u eksperimentalnom procesu je 7%.We derive a new sequential learning algorithm for Multilayered Perceptron (MLP) neural network robust to outliers. Presence of outliers in data results in failure of the model especially if data processing is performed on-line or in real time. Extended Kalman filter robust to outliers (EKF-OR) is probabilistic generative model in which measurement noise covariance is modeled as stochastic process over the set of symmetric positive-definite matrices in which prior is given as inverse Wishart distribution. Derivation of expressions comes straight form first principles, within Bayesian framework. Analytical intractability of Bayes' update step is solved using Variational Inference (VI). Experimental results obtained using real world stochastic data show that MLP network trained with proposed algorithm achieves low error and average improvement rate of 7% when compared directly to conventional EKF learning algorithm

Unsupervised discovery of multilevel statistical video structures using hierarchical hidden Markov models

Structure elements in a time sequence (e.g. video) are repetitive segments with consistent deterministic or stochastic characteristics. While most existing work in detecting structures follow a supervised paradigm, we propose a fully unsupervised statistical solution in this paper. We present a unified approach to structure discovery from long video sequences as simultaneously finding the statistical descriptions of structure and locating segments that matches the descriptions. We model the multilevel statistical structure as hierarchical hidden Markov models, and present efficient algorithms for learning both the parameters and the model structure. When tested on a specific domain, soccer video, the unsupervised learning scheme achieves very promising results: it automatically discovers the statistical descriptions of high-level structures, and at the same time achieves even slightly better accuracy in detecting discovered structures in unlabelled videos than a supervised approach designed with domain knowledge and trained with comparable hidden Markov models

Radial Basis Function Neural Networks : A Review

Radial Basis Function neural networks (RBFNNs) represent an attractive alternative to other neural network models. One reason is that they form a unifying link between function approximation, regularization, noisy interpolation, classification and density estimation. It is also the case that training RBF neural networks is faster than training multi-layer perceptron networks. RBFNN learning is usually split into an unsupervised part, where center and widths of the Gaussian basis functions are set, and a linear supervised part for weight computation. This paper reviews various learning methods for determining centers, widths, and synaptic weights of RBFNN. In addition, we will point to some applications of RBFNN in various fields. In the end, we name software that can be used for implementing RBFNNs

Audio-visual football video analysis, from structure detection to attention analysis

Sport video is an important video genre. Content-based sports video analysis attracts great interest from both industry and academic fields. A sports video is characterised by repetitive temporal structures, relatively plain contents, and strong spatio-temporal variations, such as quick camera switches and swift local motions. It is necessary to develop specific techniques for content-based sports video analysis to utilise these characteristics. For an efficient and effective sports video analysis system, there are three fundamental questions: (1) what are key stories for sports videos; (2) what incurs viewer’s interest; and (3) how to identify game highlights. This thesis is developed around these questions. We approached these questions from two different perspectives and in turn three research contributions are presented, namely, replay detection, attack temporal structure decomposition, and attention-based highlight identification. Replay segments convey the most important contents in sports videos. It is an efficient approach to collect game highlights by detecting replay segments. However, replay is an artefact of editing, which improves with advances in video editing tools. The composition of replay is complex, which includes logo transitions, slow motions, viewpoint switches and normal speed video clips. Since logo transition clips are pervasive in game collections of FIFA World Cup 2002, FIFA World Cup 2006 and UEFA Championship 2006, we take logo transition detection as an effective replacement of replay detection. A two-pass system was developed, including a five-layer adaboost classifier and a logo template matching throughout an entire video. The five-layer adaboost utilises shot duration, average game pitch ratio, average motion, sequential colour histogram and shot frequency between two neighbouring logo transitions, to filter out logo transition candidates. Subsequently, a logo template is constructed and employed to find all transition logo sequences. The precision and recall of this system in replay detection is 100% in a five-game evaluation collection. An attack structure is a team competition for a score. Hence, this structure is a conceptually fundamental unit of a football video as well as other sports videos. We review the literature of content-based temporal structures, such as play-break structure, and develop a three-step system for automatic attack structure decomposition. Four content-based shot classes, namely, play, focus, replay and break were identified by low level visual features. A four-state hidden Markov model was trained to simulate transition processes among these shot classes. Since attack structures are the longest repetitive temporal unit in a sports video, a suffix tree is proposed to find the longest repetitive substring in the label sequence of shot class transitions. These occurrences of this substring are regarded as a kernel of an attack hidden Markov process. Therefore, the decomposition of attack structure becomes a boundary likelihood comparison between two Markov chains. Highlights are what attract notice. Attention is a psychological measurement of “notice ”. A brief survey of attention psychological background, attention estimation from vision and auditory, and multiple modality attention fusion is presented. We propose two attention models for sports video analysis, namely, the role-based attention model and the multiresolution autoregressive framework. The role-based attention model is based on the perception structure during watching video. This model removes reflection bias among modality salient signals and combines these signals by reflectors. The multiresolution autoregressive framework (MAR) treats salient signals as a group of smooth random processes, which follow a similar trend but are filled with noise. This framework tries to estimate a noise-less signal from these coarse noisy observations by a multiple resolution analysis. Related algorithms are developed, such as event segmentation on a MAR tree and real time event detection. The experiment shows that these attention-based approach can find goal events at a high precision. Moreover, results of MAR-based highlight detection on the final game of FIFA 2002 and 2006 are highly similar to professionally labelled highlights by BBC and FIFA