Video Classification Using Spatial-Temporal Features and PCA

We investigate the problem of automated video classification by analysing the low-level audio-visual signal patterns along the time course in a holistic manner. Five popular TV broadcast genre are studied including sports, cartoon, news, commercial and music. A novel statistically based approach is proposed comprising two important ingredients designed for implicit semantic content characterisation and class identities modelling. First, a spatial-temporal audio-visual "super" feature vector is computed, capturing crucial clip-level video structure information inherent in a video genre. Second, the feature vector is further processed using Principal Component Analysis to reduce the spatial-temporal redundancy while exploiting the correlations between feature elements, which give rise to a compact representation for effective probabilistic modelling of each video genre. Extensive experiments are conducted assessing various aspects of the approach and their influence on the overall system performance

Visual tracking via efficient kernel discriminant subspace learning

© 2005 IEEERobustly tracking moving objects in video sequences is one of the key problems in computer vision. In this paper we introduce a computationally efficient nonlinear kernel learning strategy to find a discriminative model which distinguishes the tracked object from the background. Principal component analysis and linear discriminant analysis have been applied to this problem with some success. These techniques are limited, however, by the fact that they are capable only of identifying linear subspaces within the data. Kernel based methods, in contrast, are able to extract nonlinear subspaces, and thus represent more complex characteristics of the tracked object and background. This is a particular advantage when tracking deformable objects and where appearance changes due to the unstable illumination and pose occur. An efficient approximation to kernel discriminant analysis using QR decomposition proposed by Xiong et al. (2004) makes possible real-time updating of the optimal nonlinear subspace. We present a tracking method based on this result and show promising experimental results on real videos undergoing large pose and illumination changes

Method for solving nonlinearity in recognising tropical wood species

Classifying tropical wood species pose a considerable economic challenge and failure to classify the wood species accurately can have significant effects on timber industries. Hence, an automatic tropical wood species recognition system was developed at Centre for Artificial Intelligence and Robotics (CAIRO), Universiti Teknologi Malaysia. The system classifies wood species based on texture analysis whereby wood surface images are captured and wood features are extracted from these images which will be used for classification. Previous research on tropical wood species recognition systems considered methods for wood species classification based on linear features. Since wood species are known to exhibit nonlinear features, a Kernel-Genetic Algorithm (Kernel-GA) is proposed in this thesis to perform nonlinear feature selection. This method combines the Kernel Discriminant Analysis (KDA) technique with Genetic Algorithm (GA) to generate nonlinear wood features and also reduce dimension of the wood database. The proposed system achieved classification accuracy of 98.69%, showing marked improvement to the work done previously. Besides, a fuzzy logic-based pre-classifier is also proposed in this thesis to mimic human interpretation on wood pores which have been proven to aid the data acquisition bottleneck and serve as a clustering mechanism for large database simplifying the classification. The fuzzy logic-based pre-classifier managed to reduce the processing time for training and testing by more than 75% and 26% respectively. Finally, the fuzzy pre-classifier is combined with the Kernal-GA algorithm to improve the performance of the tropical wood species recognition system. The experimental results show that the combination of fuzzy preclassifier and nonlinear feature selection improves the performance of the tropical wood species recognition system in terms of memory space, processing time and classification accuracy

Significant Association of Urinary Toxic Metals and Autism-Related Symptoms—A Nonlinear Statistical Analysis with Cross Validation

abstract: Introduction A number of previous studies examined a possible association of toxic metals and autism, and over half of those studies suggest that toxic metal levels are different in individuals with Autism Spectrum Disorders (ASD). Additionally, several studies found that those levels correlate with the severity of ASD. Methods In order to further investigate these points, this paper performs the most detailed statistical analysis to date of a data set in this field. First morning urine samples were collected from 67 children and adults with ASD and 50 neurotypical controls of similar age and gender. The samples were analyzed to determine the levels of 10 urinary toxic metals (UTM). Autism-related symptoms were assessed with eleven behavioral measures. Statistical analysis was used to distinguish participants on the ASD spectrum and neurotypical participants based upon the UTM data alone. The analysis also included examining the association of autism severity with toxic metal excretion data using linear and nonlinear analysis. “Leave-one-out” cross-validation was used to ensure statistical independence of results. Results and Discussion Average excretion levels of several toxic metals (lead, tin, thallium, antimony) were significantly higher in the ASD group. However, ASD classification using univariate statistics proved difficult due to large variability, but nonlinear multivariate statistical analysis significantly improved ASD classification with Type I/II errors of 15% and 18%, respectively. These results clearly indicate that the urinary toxic metal excretion profiles of participants in the ASD group were significantly different from those of the neurotypical participants. Similarly, nonlinear methods determined a significantly stronger association between the behavioral measures and toxic metal excretion. The association was strongest for the Aberrant Behavior Checklist (including subscales on Irritability, Stereotypy, Hyperactivity, and Inappropriate Speech), but significant associations were found for UTM with all eleven autism-related assessments with cross-validation R[superscript 2] values ranging from 0.12–0.48.The article is published at http://journals.plos.org/plosone/article?id=10.1371/journal.pone.016952

Recognising trajectories of facial identities using kernel discriminant analysis

We present a comprehensive approach to address three challenging problems in face recognition: modelling faces across multi-views, extracting the non-linear discriminating features, and recognising moving faces dynamically in image sequences. A multi-view dynamic face model is designed to extract the shape-and-pose-free facial texture patterns. Kernel Discriminant Analysis, which employs the kernel technique to perform Linear Discriminant Analysis in a high-dimensional feature space, is developed to extract the significant non-linear features which maximise the between-class variance and minimise the within-class variance. Finally, an identity surface based face recognition is performed dynamically from video input by matching object and model trajectories

Automated Post-Earthquake Building Damage Assessment Using Smart Devices

Conventional practices to evaluate post-earthquake damage to buildings rely on reconnaissance teams deployed to the affected areas to carry out visual inspections of buildings. These inspections are done manually and are therefore time consuming and error prone. Motivated by these drawbacks, this dissertation explores the possibility and means for conducting automated inspections using smart devices, which are ubiquitous in modern communities. The premise is that smart devices can record acceleration data using their onboard sensors. The records can then be double integrated and processed to yield interstory drift ratios (IDRs), which are key indicators of building damage. The dynamic behavior of a smart device on an underlying surface subjected to seismic motion is studied first. The smart device and its frictional interactions with the underlying surface are represented using a modified friction model. The conditions under which the smart device slides on or sticks to the surface for a given earthquake intensity are investigated. The concept of a ‘probability of exceeding the slip limit curve’ is introduced to relate the probability of exceeding a given slip limit for a given structure and location. The presence of sliding motions in an acceleration record can contaminate the recorded data and make it impossible to estimate the motion of the underlying floor from smartphone measurements. To resolve this problem, stick-slip motion identification methods are studied based on two approaches. The first method relies on the theoretical observation that acceleration is constant during sliding. The second method employs two different types of machine learning algorithms to differentiate between sticking and slipping motions. It is shown that the developed techniques can yield reasonably high classification accuracies. Computation of IDR requires multiple steps, each of which is theoretically investigated and experimentally validated by using a shake table and multiple types of smart devices with different types of protective shells. The needed steps include record synchronization and warping, data fusion, and compensation for errors that are magnified by double integration (needed to compute IDR). The abilities of different types of smart devices to estimate displacement were compared and the error in displacement was shown to have a strong relationship to their mean square amplitude of stationary noise. The proposed IDR estimation process is validated using the results from previously published shake table experiments of a four-story steel frame structure. It is shown that reasonable estimates of IDR can be achieved by using the developed methods.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/149867/1/yunsu_1.pd

Hierarchical information representation and efficient classification of gene expression microarray data

In the field of computational biology, microarryas are used to measure the activity of thousands of genes at once and create a global picture of cellular function. Microarrays allow scientists to analyze expression of many genes in a single experiment quickly and eficiently. Even if microarrays are a consolidated research technology nowadays and the trends in high-throughput data analysis are shifting towards new technologies like Next Generation Sequencing (NGS), an optimum method for sample classification has not been found yet. Microarray classification is a complicated task, not only due to the high dimensionality of the feature set, but also to an apparent lack of data structure. This characteristic limits the applicability of processing techniques, such as wavelet filtering or other filtering techniques that take advantage of known structural relation. On the other hand, it is well known that genes are not expressed independently from other each other: genes have a high interdependence related to the involved regulating biological process. This thesis aims to improve the current state of the art in microarray classification and to contribute to understand how signal processing techniques can be developed and applied to analyze microarray data. The goal of building a classification framework needs an exploratory work in which algorithms are constantly tried and adapted to the analyzed data. The developed algorithms and classification frameworks in this thesis tackle the problem with two essential building blocks. The first one deals with the lack of a priori structure by inferring a data-driven structure with unsupervised hierarchical clustering tools. The second key element is a proper feature selection tool to produce a precise classifier as an output and to reduce the overfitting risk. The main focus in this thesis is the binary data classification, field in which we obtained relevant improvements to the state of the art. The first key element is the data-driven structure, obtained by modifying hierarchical clustering algorithms derived from the Treelets algorithm from the literature. Several alternatives to the original reference algorithm have been tested, changing either the similarity metric to merge the feature or the way two feature are merged. Moreover, the possibility to include external sources of information from publicly available biological knowledge and ontologies to improve the structure generation has been studied too. About the feature selection, two alternative approaches have been studied: the first one is a modification of the IFFS algorithm as a wrapper feature selection, while the second approach involved an ensemble learning focus. To obtain good results, the IFFS algorithm has been adapted to the data characteristics by introducing new elements to the selection process like a reliability measure and a scoring system to better select the best feature at each iteration. The second feature selection approach is based on Ensemble learning, taking advantage of the microarryas feature abundance to implement a different selection scheme. New algorithms have been studied in this field, improving state of the art algorithms to the microarray data characteristic of small sample and high feature numbers. In addition to the binary classification problem, the multiclass case has been addressed too. A new algorithm combining multiple binary classifiers has been evaluated, exploiting the redundancy offered by multiple classifiers to obtain better predictions. All the studied algorithm throughout this thesis have been evaluated using high quality publicly available data, following established testing protocols from the literature to offer a proper benchmarking with the state of the art. Whenever possible, multiple Monte Carlo simulations have been performed to increase the robustness of the obtained results.En el campo de la biología computacional, los microarrays son utilizados para medir la actividad de miles de genes a la vez y producir una representación global de la función celular. Los microarrays permiten analizar la expresión de muchos genes en un solo experimento, rápidamente y eficazmente. Aunque los microarrays sean una tecnología de investigación consolidada hoy en día y la tendencia es en utilizar nuevas tecnologías como Next Generation Sequencing (NGS), aun no se ha encontrado un método óptimo para la clasificación de muestras. La clasificación de muestras de microarray es una tarea complicada, debido al alto número de variables y a la falta de estructura entre los datos. Esta característica impide la aplicación de técnicas de procesado que se basan en relaciones estructurales, como el filtrado con wavelet u otras técnicas de filltrado. Por otro lado, los genes no se expresen independientemente unos de otros: los genes están inter-relacionados según el proceso biológico que les regula. El objetivo de esta tesis es mejorar el estado del arte en la clasi cación de microarrays y contribuir a entender cómo se pueden diseñar y aplicar técnicas de procesado de señal para analizar microarrays. El objetivo de construir un algoritmo de clasi cación, necesita un estudio de comprobaciones y adaptaciones de algoritmos existentes a los datos analizados. Los algoritmo desarrollados en esta tesis encaran el problema con dos bloques esenciales. El primero ataca la falta de estructura, derivando un árbol binario usando herramientas de clustering no supervisado. El segundo elemento fundamental para obtener clasificadores precisos reduciendo el riesgo de overfitting es un elemento de selección de variables. La principal tarea en esta tesis es la clasificación de datos binarios en la cual hemos obtenido mejoras relevantes al estado del arte. El primer paso es la generación de una estructura, para eso se ha utilizado el algoritmo Treelets disponible en la literatura. Múltiples alternativas a este algoritmo original han sido propuestas y evaluadas, cambiando las métricas de similitud o las reglas de fusión durante el proceso. Además, se ha estudiado la posibilidad de usar fuentes de información externas, como ontologías de información biológica, para mejorar la inferencia de la estructura. Se han estudiado dos enfoques diferentes para la selección de variables: el primero es una modificación del algoritmo IFFS y el segundo utiliza un esquema de aprendizaje con “ensembles”. El algoritmo IFFS ha sido adaptado a las características de microarrays para obtener mejores resultados, añadiendo elementos como la medida de fiabilidad y un sistema de evaluación para seleccionar la mejor variable en cada iteración. El método que utiliza “ensembles” aprovecha la abundancia de features de los microarrays para implementar una selección diferente. En este campo se han estudiado diferentes algoritmos, mejorando alternativas ya existentes al escaso número de muestras y al alto número de variables, típicos de los microarrays. El problema de clasificación con más de dos clases ha sido también tratado al estudiar un nuevo algoritmo que combina múltiples clasificadores binarios. El algoritmo propuesto aprovecha la redundancia ofrecida por múltiples clasificadores para obtener predicciones más fiables. Todos los algoritmos propuestos en esta tesis han sido evaluados con datos públicos y de alta calidad, siguiendo protocolos establecidos en la literatura para poder ofrecer una comparación fiable con el estado del arte. Cuando ha sido posible, se han aplicado simulaciones Monte Carlo para mejorar la robustez de los resultados

Deliverable D1.1 State of the art and requirements analysis for hypervideo

This deliverable presents a state-of-art and requirements analysis report for hypervideo authored as part of the WP1 of the LinkedTV project. Initially, we present some use-case (viewers) scenarios in the LinkedTV project and through the analysis of the distinctive needs and demands of each scenario we point out the technical requirements from a user-side perspective. Subsequently we study methods for the automatic and semi-automatic decomposition of the audiovisual content in order to effectively support the annotation process. Considering that the multimedia content comprises of different types of information, i.e., visual, textual and audio, we report various methods for the analysis of these three different streams. Finally we present various annotation tools which could integrate the developed analysis results so as to effectively support users (video producers) in the semi-automatic linking of hypervideo content, and based on them we report on the initial progress in building the LinkedTV annotation tool. For each one of the different classes of techniques being discussed in the deliverable we present the evaluation results from the application of one such method of the literature to a dataset well-suited to the needs of the LinkedTV project, and we indicate the future technical requirements that should be addressed in order to achieve higher levels of performance (e.g., in terms of accuracy and time-efficiency), as necessary