ACTIVITY ANALYSIS OF SPECTATOR PERFORMER VIDEOS USING MOTION TRAJECTORIES

Spectator Performer Space (SPS) is a frequently occurring crowd dynamics, composed of one or more central performers, and a peripheral crowd of spectators. Analysis of videos in this space is often complicated due to occlusion and high density of people. Although there are many video analysis approaches, they are targeted for individual actors or low-density crowd and hence are not suitable for SPS videos. In this work, we present two trajectory-based features: Histogram of Trajectories (HoT) and Histogram of Trajectory Clusters (HoTC) to analyze SPS videos. HoT is calculated from the distribution of length and orientation of motion trajectories in a video. For HoTC, we compute the features derived from the motion trajectory clusters in the videos. So, HoTC characterizes different spatial region which may contain different action categories, inside a video. We have extended DBSCAN, a well-known clustering algorithm, to cluster short trajectories, common in SPS videos. The derived features are then used to classify the SPS videos based on their activities. In addition to using NaïveBayes and support vector machines (SVM), we have experimented with ensemble based classifiers and a deep learning approach using the videos directly for training. The efficacy of our algorithms is demonstrated using a dataset consisting of 4000 real life videos each from spectator and performer spaces. The classification accuracies for spectator videos (HoT: 87%; HoTC: 92%) and performer videos (HoT: 91%; HoTC: 90%) show that our approach out-performs t­­he state of the art techniques based on deep learning. Advisor: Ashok Sama

ACTIVITY ANALYSIS OF SPECTATOR PERFORMER VIDEOS USING MOTION TRAJECTORIES

Spectator Performer Space (SPS) is a frequently occurring crowd dynamics, composed of one or more central performers, and a peripheral crowd of spectators. Analysis of videos in this space is often complicated due to occlusion and high density of people. Although there are many video analysis approaches, they are targeted for individual actors or low-density crowd and hence are not suitable for SPS videos. In this work, we present two trajectory-based features: Histogram of Trajectories (HoT) and Histogram of Trajectory Clusters (HoTC) to analyze SPS videos. HoT is calculated from the distribution of length and orientation of motion trajectories in a video. For HoTC, we compute the features derived from the motion trajectory clusters in the videos. So, HoTC characterizes different spatial region which may contain different action categories, inside a video. We have extended DBSCAN, a well-known clustering algorithm, to cluster short trajectories, common in SPS videos. The derived features are then used to classify the SPS videos based on their activities. In addition to using NaïveBayes and support vector machines (SVM), we have experimented with ensemble based classifiers and a deep learning approach using the videos directly for training. The efficacy of our algorithms is demonstrated using a dataset consisting of 4000 real life videos each from spectator and performer spaces. The classification accuracies for spectator videos (HoT: 87%; HoTC: 92%) and performer videos (HoT: 91%; HoTC: 90%) show that our approach out-performs t­­he state of the art techniques based on deep learning. Advisor: Ashok Sama

Spatio-temporal coverage optimization of sensor networks

Les réseaux de capteurs sont formés d’un ensemble de dispositifs capables de prendre individuellement des mesures d’un environnement particulier et d’échanger de l’information afin d’obtenir une représentation de haut niveau sur les activités en cours dans la zone d’intérêt. Une telle détection distribuée, avec de nombreux appareils situés à proximité des phénomènes d’intérêt, est pertinente dans des domaines tels que la surveillance, l’agriculture, l’observation environnementale, la surveillance industrielle, etc. Nous proposons dans cette thèse plusieurs approches pour effectuer l’optimisation des opérations spatio-temporelles de ces dispositifs, en déterminant où les placer dans l’environnement et comment les contrôler au fil du temps afin de détecter les cibles mobiles d’intérêt. La première nouveauté consiste en un modèle de détection réaliste représentant la couverture d’un réseau de capteurs dans son environnement. Nous proposons pour cela un modèle 3D probabiliste de la capacité de détection d’un capteur sur ses abords. Ce modèle inègre également de l’information sur l’environnement grâce à l’évaluation de la visibilité selon le champ de vision. À partir de ce modèle de détection, l’optimisation spatiale est effectuée par la recherche du meilleur emplacement et l’orientation de chaque capteur du réseau. Pour ce faire, nous proposons un nouvel algorithme basé sur la descente du gradient qui a été favorablement comparée avec d’autres méthodes génériques d’optimisation «boites noires» sous l’aspect de la couverture du terrain, tout en étant plus efficace en terme de calculs. Une fois que les capteurs placés dans l’environnement, l’optimisation temporelle consiste à bien couvrir un groupe de cibles mobiles dans l’environnement. D’abord, on effectue la prédiction de la position future des cibles mobiles détectées par les capteurs. La prédiction se fait soit à l’aide de l’historique des autres cibles qui ont traversé le même environnement (prédiction à long terme), ou seulement en utilisant les déplacements précédents de la même cible (prédiction à court terme). Nous proposons de nouveaux algorithmes dans chaque catégorie qui performent mieux ou produits des résultats comparables par rapport aux méthodes existantes. Une fois que les futurs emplacements de cibles sont prédits, les paramètres des capteurs sont optimisés afin que les cibles soient correctement couvertes pendant un certain temps, selon les prédictions. À cet effet, nous proposons une méthode heuristique pour faire un contrôle de capteurs, qui se base sur les prévisions probabilistes de trajectoire des cibles et également sur la couverture probabiliste des capteurs des cibles. Et pour terminer, les méthodes d’optimisation spatiales et temporelles proposées ont été intégrées et appliquées avec succès, ce qui démontre une approche complète et efficace pour l’optimisation spatio-temporelle des réseaux de capteurs.Sensor networks consist in a set of devices able to individually capture information on a given environment and to exchange information in order to obtain a higher level representation on the activities going on in the area of interest. Such a distributed sensing with many devices close to the phenomena of interest is of great interest in domains such as surveillance, agriculture, environmental monitoring, industrial monitoring, etc. We are proposing in this thesis several approaches to achieve spatiotemporal optimization of the operations of these devices, by determining where to place them in the environment and how to control them over time in order to sense the moving targets of interest. The first novelty consists in a realistic sensing model representing the coverage of a sensor network in its environment. We are proposing for that a probabilistic 3D model of sensing capacity of a sensor over its surrounding area. This model also includes information on the environment through the evaluation of line-of-sight visibility. From this sensing model, spatial optimization is conducted by searching for the best location and direction of each sensor making a network. For that purpose, we are proposing a new algorithm based on gradient descent, which has been favourably compared to other generic black box optimization methods in term of performance, while being more effective when considering processing requirements. Once the sensors are placed in the environment, the temporal optimization consists in covering well a group of moving targets in the environment. That starts by predicting the future location of the mobile targets detected by the sensors. The prediction is done either by using the history of other targets who traversed the same environment (long term prediction), or only by using the previous displacements of the same target (short term prediction). We are proposing new algorithms under each category which outperformed or produced comparable results when compared to existing methods. Once future locations of targets are predicted, the parameters of the sensors are optimized so that targets are properly covered in some future time according to the predictions. For that purpose, we are proposing a heuristics for making such sensor control, which deals with both the probabilistic targets trajectory predictions and probabilistic coverage of sensors over the targets. In the final stage, both spatial and temporal optimization method have been successfully integrated and applied, demonstrating a complete and effective pipeline for spatiotemporal optimization of sensor networks

Survey of maps of dynamics for mobile robots

Robotic mapping provides spatial information for autonomous agents. Depending on the tasks they seek to enable, the maps created range from simple 2D representations of the environment geometry to complex, multilayered semantic maps. This survey article is about maps of dynamics (MoDs), which store semantic information about typical motion patterns in a given environment. Some MoDs use trajectories as input, and some can be built from short, disconnected observations of motion. Robots can use MoDs, for example, for global motion planning, improved localization, or human motion prediction. Accounting for the increasing importance of maps of dynamics, we present a comprehensive survey that organizes the knowledge accumulated in the field and identifies promising directions for future work. Specifically, we introduce field-specific vocabulary, summarize existing work according to a novel taxonomy, and describe possible applications and open research problems. We conclude that the field is mature enough, and we expect that maps of dynamics will be increasingly used to improve robot performance in real-world use cases. At the same time, the field is still in a phase of rapid development where novel contributions could significantly impact this research area

Data-driven task allocation for multi-robot deliveries

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 93-97).In this thesis, we present a distributed task allocation system for a team of robots serving queues of tasks in an environment. We consider how historical information about such a system's performance could be used to improve future allocations. Our model is representative of a multi-robot mail delivery service, in which teams of robots would have to cooperate to pick up and deliver packages in an environment. We provide a framework for task allocation, planning, and control of the system and analyze task switching as a method for improving a task allocation as the system is running. We first treat a system where robots exchange tasks as they encounter each other in the environment. We consider both cases where the number of robots matches the number of task queues being served and where it does not. Most importantly, for situations where an optimal task switching policy would be too computationally expensive, we provide heuristics that nonetheless guarantee task completion. Our simulations show that our heuristics also generally lower the costs of task completion. We incorporate historical data about system performance by looking at a spatial allocation of tasks to robots in the system. We propose an algorithm for partitioning the environment into regions of equal workload for the robots. In order to overcome communication constraints, we introduce hubs, locations where robots can pass tasks to each other. We simulate the system with this additional infrastructure and compare its performance to that without hubs. We find that hubs can significantly improve performance when the task queues themselves follow some spatial structure.by Cynthia Rueyi Sung.S.M

Lagrangian coherent structures and trajectory similarity: two important tools for scientific visualization

This thesis studies the computation and visualization of Lagrangian coherent structures (LCS), an emerging technique for analyzing time-varying velocity fields (e.g. blood vessels and airflows), and the measure of similarity for trajectories (e.g. hurricane paths). LCS surfaces and trajectory-based techniques (e.g. trajectory clustering) are complementary to each other for visualization, while velocity fields and trajectories are two important types of scientific data, which are more and more accessible by virtue of the technology development for both data collection and numerical simulation. A key step for LCS computation is tracing the paths of collections of particles through a flow field. When a flow field is interpolated from the nodes of an unstructured mesh, the process of advecting a particle must first find which cell in the unstructured mesh contains the particle. Since the paths of nearby particles often diverge, the parallelization of particle advection quickly leads to incoherent memory accesses of the unstructured mesh. We have developed a new block advection GPU approach that reorganizes particles into spatially coherent bundles as they follow their advection paths, which greatly improves memory coherence and thus shared-memory GPU performance. This approach works best for flows that meet the CFL criterion on unstructured meshes of uniformly sized elements, small enough to fit at least two timesteps in GPU memory. LCS surfaces provide insight into unsteady fluid flow, but their construction has posed many challenges. These structures can be characterized as ridges of a field, but their local definition utilizes an ambiguous eigenvector direction that can point in one of two directions, and its ambiguity can lead to noise and other problems. We overcome these issues with an application of a global ridge definition, applied using the hierarchical watershed transformation. We show results on a mathematical flow model and a simulated vascular flow dataset indicating the watershed method produces less noisy structures. Trajectory similarity has been shown to be a powerful tool for visualizing and analyzing trajectories. In this paper we propose a novel measure of trajectory similarity using both spatial and directional information. The similarity is asymmetric, bounded within [0,1], affine-invariant, and efficiently computed. Asymmetric mappings between a pair of trajectories can be derived from this similarity. Experimental results demonstrate that the measure is better than existing measures in both similarity scores and trajectory mappings. The measure also inspires a simple similarity-based clustering method for effectivly visualizing a large number of trajectories, which outperforms the state-of-the-art model-based clustering method (VFKM)