Behavior Discovery and Alignment of Articulated Object Classes from Unstructured Video

We propose an automatic system for organizing the content of a collection of unstructured videos of an articulated object class (e.g. tiger, horse). By exploiting the recurring motion patterns of the class across videos, our system: 1) identifies its characteristic behaviors; and 2) recovers pixel-to-pixel alignments across different instances. Our system can be useful for organizing video collections for indexing and retrieval. Moreover, it can be a platform for learning the appearance or behaviors of object classes from Internet video. Traditional supervised techniques cannot exploit this wealth of data directly, as they require a large amount of time-consuming manual annotations. The behavior discovery stage generates temporal video intervals, each automatically trimmed to one instance of the discovered behavior, clustered by type. It relies on our novel motion representation for articulated motion based on the displacement of ordered pairs of trajectories (PoTs). The alignment stage aligns hundreds of instances of the class to a great accuracy despite considerable appearance variations (e.g. an adult tiger and a cub). It uses a flexible Thin Plate Spline deformation model that can vary through time. We carefully evaluate each step of our system on a new, fully annotated dataset. On behavior discovery, we outperform the state-of-the-art Improved DTF descriptor. On spatial alignment, we outperform the popular SIFT Flow algorithm.Comment: 19 pages, 19 figure, 3 tables. arXiv admin note: substantial text overlap with arXiv:1411.788

An Information Theoretic Framework for Camera and Lidar Sensor Data Fusion and its Applications in Autonomous Navigation of Vehicles.

This thesis develops an information theoretic framework for multi-modal sensor data fusion for robust autonomous navigation of vehicles. In particular we focus on the registration of 3D lidar and camera data, which are commonly used perception sensors in mobile robotics. This thesis presents a framework that allows the fusion of the two modalities, and uses this fused information to enhance state-of-the-art registration algorithms used in robotics applications. It is important to note that the time-aligned discrete signals (3D points and their reflectivity from lidar, and pixel location and color from camera) are generated by sampling the same physical scene, but in a different manner. Thus, although these signals look quite different at a high level (2D image from a camera looks entirely different than a 3D point cloud of the same scene from a lidar), since they are generated from the same physical scene, they are statistically dependent upon each other at the signal level. This thesis exploits this statistical dependence in an information theoretic framework to solve some of the common problems encountered in autonomous navigation tasks such as sensor calibration, scan registration and place recognition. In a general sense we consider these perception sensors as a source of information (i.e., sensor data), and the statistical dependence of this information (obtained from different modalities) is used to solve problems related to multi-modal sensor data registration.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/107286/1/pgaurav_1.pd

Computational and Imaging Methods for Studying Neuronal Populations during Behavior

One of the central questions in neuroscience is how the nervous system generates and regulates behavior. To understand the neural code for any behavior, an ideal experiment would entail (i) quantitatively defining that behavior, (ii) recording neuronal activity in relevant brain regions to identify the underlying neuronal circuits and eventually (iii) manipulating them to test their function. Novel methods in neuroscience have greatly advanced our abilities to conduct such experiments but are still insufficient. In this thesis, I developed methods for these three goals. In Chapter 2, I describe an automatic behavior identification and classification method for the cnidarian Hydra vulgaris using machine learning. In Chapter 3, I describe a fast volumetric two-photon microscope with dual-color laser excitation that can image in 3D the activity of populations of neurons from visual cortex of awake mice. In Chapter 4, I present a machine learning method that identifies cortical ensembles and pattern completion neurons in mouse visual cortex, using two-photon calcium imaging data. These methods advance current technologies, providing opportunities for new discoveries

Multi-Dimensional Task Recognition for Human-Robot Teaming

Human-robot teams involve humans and robots collaborating to achieve tasks under various environmental conditions. Successful teaming requires robots to adapt autonomously in real-time to a human teammate's state. An important element of such adaptation is the ability for the robot to infer the tasks performed by their human teammates. Human-robot teams often perform a wide variety of tasks, involving multiple activity components, and may even perform two or more tasks concurrently. A robot’s ability to recognize the human’s composite tasks that occur concurrently is a key requirement for realizing successful collaboration. Existing task recognition algorithms are not viable for human-robot teams, as they only detect tasks from a subset of activity components and rarely detect concurrent, composite tasks. This dissertation developed a multi-dimensional task recognition algorithm capable of detecting concurrent, composite tasks across the cognitive, speech, auditory, visual, gross motor, fine-grained motor, and tactile components by incorporating metrics that are sensitive, versatile, and suitable across human-robot teaming paradigms. The developed algorithm addresses a foundational problem of understanding an individual's task engagement state in human-robot teams operating in dynamic, unstructured environments

Proceedings of the Eighth Workshop on Information Theoretic Methods in Science and Engineering

Proceedings of the Eighth Workshop on Information Theoretic Methods in Science and Engineering (WITMSE 2015) held in Copenhagen, Denmark, 24-26 June 2015; published in the series of the Department of Computer Science, University of Helsinki.Peer reviewe

Permutation distribution clustering and structural equation model trees

The primary goal of this thesis is to present novel methodologies for the exploratory analysis of psychological data sets that support researchers in informed theory development. Psychological data analysis bears a long tradition of confirming hypotheses generated prior to data collection. However, in practical research, the following two situations are commonly observed: In the first instance, there are no initial hypotheses about the data. In that case, there is no model available and one has to resort to uninformed methods to reveal structure in the data. In the second instance, existing models that reflect prior hypotheses need to be extended and improved, thereby altering and renewing hypotheses about the data and refining descriptions of the observed phenomena. This dissertation introduces a novel method for the exploratory analysis of psychological data sets for each of the two situations. Both methods focus on time series analysis, which is particularly interesting for the analysis of psychophysiological data and longitudinal data typically collected by developmental psychologists. Nonetheless, the methods are generally applicable and useful for other fields that analyze time series data, e.g., sociology, economics, neuroscience, and genetics. The first part of the dissertation proposes a clustering method for time series. A dissimilarity measure of time series based on the permutation distribution is developed. Employing this measure in a hierarchical scheme allows for a novel clustering method for time series based on their relative complexity: Permutation Distribution Clustering (PDC). Two methods for the determination of the number of distinct clusters are discussed based on a statistical and an information-theoretic criterion. Structural Equation Models (SEMs) constitute a versatile modeling technique, which is frequently employed in psychological research. The second part of the dissertation introduces an extension of SEMs to Structural Equation Modeling Trees (SEM Trees). SEM Trees describe partitions of a covariate-space which explain differences in the model parameters. They can provide solutions in situations in which hypotheses in the form of a model exist but may potentially be refined by integrating other variables. By harnessing the full power of SEM, they represent a general data analysis technique that can be used for both time series and non-time series data. SEM Trees algorithmically refine initial models of the sample and thus support researchers in theory development. This thesis includes demonstrations of the methods on simulated as well as on real data sets, including applications of SEM Trees to longitudinal models of cognitive development and cross-sectional cognitive factor models, and applications of PDC on psychophysiological data, including electroencephalographic, electrocardiographic, and genetic data.Ziel dieser Arbeit ist der Entwurf von explorativen Analysemethoden für Datensätze aus der Psychologie, um Wissenschaftler bei der Entwicklung fundierter Theorien zu unterstützen. Die Arbeit ist motiviert durch die Beobachtung, dass die klassischen Auswertungsmethoden für psychologische Datensätze auf der Tradition gründen, Hypothesen zu testen, die vor der Datenerhebung aufgestellt wurden. Allerdings treten die folgenden beiden Situationen im Alltag der Datenauswertung häufig auf: (1) es existieren keine Hypothesen über die Daten und damit auch kein Modelle. Der Wissenschaftler muss also auf uninformierte Methoden zurückgreifen, um Strukturen und Ähnlichkeiten in den Daten aufzudecken. (2) Modelle sind vorhanden, die Hypothesen über die Daten widerspiegeln, aber die Stichprobe nur unzureichend abbilden. In diesen Fällen müssen die existierenden Modelle und damit Hypothesen verändert und erweitert werden, um die Beschreibung der beobachteten Phänomene zu verfeinern. Die vorliegende Dissertation führt für beide Fälle je eine neue Methode ein, die auf die explorative Analyse psychologischer Daten zugeschnitten ist. Gleichwohl sind beide Methoden für alle Bereiche nützlich, in denen Zeitreihendaten analysiert werden, wie z.B. in der Soziologie, den Wirtschaftswissenschaften, den Neurowissenschaften und der Genetik. Der erste Teil der Arbeit schlägt ein Clusteringverfahren für Zeitreihen vor. Dieses basiert auf einem Ähnlichkeitsmaß zwischen Zeitreihen, das auf die Permutationsverteilung der eingebetteten Zeitreihen zurückgeht. Dieses Maß wird mit einem hierarchischen Clusteralgorithmus kombiniert, um Zeitreihen nach ihrer Komplexität in homogene Gruppen zu ordnen. Auf diese Weise entsteht die neue Methode der Permutationsverteilungs-basierten Clusteranalyse (PDC). Zwei Methoden zur Bestimmung der Anzahl von separaten Clustern werden hergeleitet, einmal auf Grundlage von statistischen Tests und einmal basierend auf informationstheoretischen Kriterien. Der zweite Teil der Arbeit erweitert Strukturgleichungsmodelle (SEM), eine vielseitige Modellierungstechnik, die in der Psychologie weit verbreitet ist, zu Strukturgleichungsmodell-Bäumen (SEM Trees). SEM Trees beschreiben rekursive Partitionen eines Raumes beobachteter Variablen mit maximalen Unterschieden in den Modellparametern eines SEMs. In Situationen, in denen Hypothesen in Form eines Modells existieren, können SEM Trees sie verfeinern, indem sie automatisch Variablen finden, die Unterschiede in den Modellparametern erklären. Durch die hohe Flexibilität von SEMs, können eine Vielzahl verschiedener Modelle mit SEM Trees erweitert werden. Die Methode eignet sich damit für die Analyse sowohl von Zeitreihen als auch von Nicht-Zeitreihen. SEM Trees verfeinern algorithmisch anfängliche Hypothesen und unterstützen Forscher in der Weiterentwicklung ihrer Theorien. Die vorliegende Arbeit beinhaltet Demonstrationen der vorgeschlagenen Methoden auf realen Datensätzen, darunter Anwendungen von SEM Trees auf einem längsschnittlichen Wachstumsmodell kognitiver Fähigkeiten und einem querschnittlichen kognitiven Faktor Modell, sowie Anwendungen des PDC auf verschiedenen psychophsyiologischen Zeitreihen

High-Level Codewords Based on Granger Causality for Video Event Detection

Video event detection is a challenging problem in many applications, such as video surveillance and video content analysis. In this paper, we propose a new framework to perceive high-level codewords by analyzing temporal relationship between different channels of video features. The low-level vocabulary words are firstly generated after different audio and visual feature extraction. A weighted undirected graph is constructed by exploring the Granger Causality between low-level words. Then, a greedy agglomerative graph-partitioning method is used to discover low-level word groups which have similar temporal pattern. The high-level codebooks representation is obtained by quantification of low-level words groups. Finally, multiple kernel learning, combined with our high-level codewords, is used to detect the video event. Extensive experimental results show that the proposed method achieves preferable results in video event detection

SIMULATING SEISMIC WAVE PROPAGATION IN TWO-DIMENSIONAL MEDIA USING DISCONTINUOUS SPECTRAL ELEMENT METHODS

We introduce a discontinuous spectral element method for simulating seismic wave in 2- dimensional elastic media. The methods combine the flexibility of a discontinuous finite element method with the accuracy of a spectral method. The elastodynamic equations are discretized using high-degree of Lagrange interpolants and integration over an element is accomplished based upon the Gauss-Lobatto-Legendre integration rule. This combination of discretization and integration results in a diagonal mass matrix and the use of discontinuous finite element method makes the calculation can be done locally in each element. Thus, the algorithm is simplified drastically. We validated the results of one-dimensional problem by comparing them with finite-difference time-domain method and exact solution. The comparisons show excellent agreement