Action Recognition in Videos: from Motion Capture Labs to the Web

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, "in the wild" videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4 table

Search Tracker: Human-derived object tracking in-the-wild through large-scale search and retrieval

Humans use context and scene knowledge to easily localize moving objects in conditions of complex illumination changes, scene clutter and occlusions. In this paper, we present a method to leverage human knowledge in the form of annotated video libraries in a novel search and retrieval based setting to track objects in unseen video sequences. For every video sequence, a document that represents motion information is generated. Documents of the unseen video are queried against the library at multiple scales to find videos with similar motion characteristics. This provides us with coarse localization of objects in the unseen video. We further adapt these retrieved object locations to the new video using an efficient warping scheme. The proposed method is validated on in-the-wild video surveillance datasets where we outperform state-of-the-art appearance-based trackers. We also introduce a new challenging dataset with complex object appearance changes.Comment: Under review with the IEEE Transactions on Circuits and Systems for Video Technolog

Neighborhood detection and rule selection from cellular automata patterns

Using genetic algorithms (GAs) to search for cellular automation (CA) rules from spatio-temporal patterns produced in CA evolution is usually complicated and time-consuming when both, the neighborhood structure and the local rule are searched simultaneously. The complexity of this problem motivates the development of a new search which separates the neighborhood detection from the GA search. In the paper, the neighborhood is determined by independently selecting terms from a large term set on the basis of the contribution each term makes to the next state of the cell to be updated. The GA search is then started with a considerably smaller set of candidate rules pre-defined by the detected neighhorhood. This approach is tested over a large set of one-dimensional (1-D) and two-dimensional (2-D) CA rules. Simulation results illustrate the efficiency of the new algorith

Efficient Data Collection in Multimedia Vehicular Sensing Platforms

Vehicles provide an ideal platform for urban sensing applications, as they can be equipped with all kinds of sensing devices that can continuously monitor the environment around the travelling vehicle. In this work we are particularly concerned with the use of vehicles as building blocks of a multimedia mobile sensor system able to capture camera snapshots of the streets to support traffic monitoring and urban surveillance tasks. However, cameras are high data-rate sensors while wireless infrastructures used for vehicular communications may face performance constraints. Thus, data redundancy mitigation is of paramount importance in such systems. To address this issue in this paper we exploit sub-modular optimisation techniques to design efficient and robust data collection schemes for multimedia vehicular sensor networks. We also explore an alternative approach for data collection that operates on longer time scales and relies only on localised decisions rather than centralised computations. We use network simulations with realistic vehicular mobility patterns to verify the performance gains of our proposed schemes compared to a baseline solution that ignores data redundancy. Simulation results show that our data collection techniques can ensure a more accurate coverage of the road network while significantly reducing the amount of transferred data