Geometric Cross-Modal Comparison of Heterogeneous Sensor Data

In this work, we address the problem of cross-modal comparison of aerial data streams. A variety of simulated automobile trajectories are sensed using two different modalities: full-motion video, and radio-frequency (RF) signals received by detectors at various locations. The information represented by the two modalities is compared using self-similarity matrices (SSMs) corresponding to time-ordered point clouds in feature spaces of each of these data sources; we note that these feature spaces can be of entirely different scale and dimensionality. Several metrics for comparing SSMs are explored, including a cutting-edge time-warping technique that can simultaneously handle local time warping and partial matches, while also controlling for the change in geometry between feature spaces of the two modalities. We note that this technique is quite general, and does not depend on the choice of modalities. In this particular setting, we demonstrate that the cross-modal distance between SSMs corresponding to the same trajectory type is smaller than the cross-modal distance between SSMs corresponding to distinct trajectory types, and we formalize this observation via precision-recall metrics in experiments. Finally, we comment on promising implications of these ideas for future integration into multiple-hypothesis tracking systems.Comment: 10 pages, 13 figures, Proceedings of IEEE Aeroconf 201

Vehicle-Rear: A New Dataset to Explore Feature Fusion for Vehicle Identification Using Convolutional Neural Networks

This work addresses the problem of vehicle identification through non-overlapping cameras. As our main contribution, we introduce a novel dataset for vehicle identification, called Vehicle-Rear, that contains more than three hours of high-resolution videos, with accurate information about the make, model, color and year of nearly 3,000 vehicles, in addition to the position and identification of their license plates. To explore our dataset we design a two-stream CNN that simultaneously uses two of the most distinctive and persistent features available: the vehicle's appearance and its license plate. This is an attempt to tackle a major problem: false alarms caused by vehicles with similar designs or by very close license plate identifiers. In the first network stream, shape similarities are identified by a Siamese CNN that uses a pair of low-resolution vehicle patches recorded by two different cameras. In the second stream, we use a CNN for OCR to extract textual information, confidence scores, and string similarities from a pair of high-resolution license plate patches. Then, features from both streams are merged by a sequence of fully connected layers for decision. In our experiments, we compared the two-stream network against several well-known CNN architectures using single or multiple vehicle features. The architectures, trained models, and dataset are publicly available at https://github.com/icarofua/vehicle-rear

Phase transitions during fruiting body formation in Myxococcus xanthus

The formation of a collectively moving group benefits individuals within a population in a variety of ways such as ultra-sensitivity to perturbation, collective modes of feeding, and protection from environmental stress. While some collective groups use a single organizing principle, others can dynamically shift the behavior of the group by modifying the interaction rules at the individual level. The surface-dwelling bacterium Myxococcus xanthus forms dynamic collective groups both to feed on prey and to aggregate during times of starvation. The latter behavior, termed fruiting-body formation, involves a complex, coordinated series of density changes that ultimately lead to three-dimensional aggregates comprising hundreds of thousands of cells and spores. This multi-step developmental process most likely involves several different single-celled behaviors as the population condenses from a loose, two-dimensional sheet to a three-dimensional mound. Here, we use high-resolution microscopy and computer vision software to spatiotemporally track the motion of thousands of individuals during the initial stages of fruiting body formation. We find that a combination of cell-contact-mediated alignment and internal timing mechanisms drive a phase transition from exploratory flocking, in which cell groups move rapidly and coherently over long distances, to a reversal-mediated localization into streams, which act as slow-spreading, quasi-one-dimensional nematic fluids. These observations lead us to an active liquid crystal description of the myxobacterial development cycle.Comment: 16 pages, 5 figure