Action Recognition in Videos: from Motion Capture Labs to the Web

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, "in the wild" videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4 table

Application of computer vision for roller operation management

Compaction is the last and possibly the most important phase in construction of asphalt concrete (AC) pavements. Compaction densifies the loose (AC) mat, producing a stable surface with low permeability. The process strongly affects the AC performance properties. Too much compaction may cause aggregate degradation and low air void content facilitating bleeding and rutting. On the other hand too little compaction may result in higher air void content facilitating oxidation and water permeability issues, rutting due to further densification by traffic and reduced fatigue life. Therefore, compaction is a critical issue in AC pavement construction.;The common practice for compacting a mat is to establish a roller pattern that determines the number of passes and coverages needed to achieve the desired density. Once the pattern is established, the roller\u27s operator must maintain the roller pattern uniformly over the entire mat.;Despite the importance of uniform compaction to achieve the expected durability and performance of AC pavements, having the roller operator as the only mean to manage the operation can involve human errors.;With the advancement of technology in recent years, the concept of intelligent compaction (IC) was developed to assist the roller operators and improve the construction quality. Commercial IC packages for construction rollers are available from different manufacturers. They can provide precise mapping of a roller\u27s location and provide the roller operator with feedback during the compaction process.;Although, the IC packages are able to track the roller passes with impressive results, there are also major hindrances. The high cost of acquisition and potential negative impact on productivity has inhibited implementation of IC.;This study applied computer vision technology to build a versatile and affordable system to count and map roller passes. An infrared camera is mounted on top of the roller to capture the operator view. Then, in a near real-time process, image features were extracted and tracked to estimate the incremental rotation and translation of the roller. Image featured are categorized into near and distant features based on the user defined horizon. The optical flow is estimated for near features located in the region below the horizon. The change in roller\u27s heading is constantly estimated from the distant features located in the sky region. Using the roller\u27s rotation angle, the incremental translation between two frames will be calculated from the optical flow. The roller\u27s incremental rotation and translation will put together to develop a tracking map.;During system development, it was noted that in environments with thermal uniformity, the background of the IR images exhibit less featured as compared to images captured with optical cameras which are insensitive to temperature. This issue is more significant overnight, since nature elements are not able to reflect the heat energy from sun. Therefore to improve roller\u27s heading estimation where less features are available in the sky region a unique methodology that allows heading detection based on the asphalt mat edges was developed for this research. The heading measurements based on the slope of the asphalt hot edges will be added to the pool of the headings measured from sky region. The median of all heading measurements will be used as the incremental roller\u27s rotation for the tracking analysis.;The record of tracking data is used for QC/QA purposes and verifying the proper implementation of the roller pattern throughout a job constructed under the roller pass specifications.;The system developed during this research was successful in mapping roller location for few projects tested. However the system should be independently validated

Machine Analysis of Facial Expressions

No abstract

Autonomous Navigation of Distributed Spacecraft using Graph-based SLAM for Proximity Operations in Small Celestial Bodies

Establishment of a sustainable human presence beyond the cislunar space is a major milestone for mankind. Small celestial bodies (SCBs) like asteroids are known to contain valuable natural resources necessary for the development of space assets essential to the accomplishment of this goal. Consequently, future robotic spacecraft missions to SCBs are envisioned with the objective of commercial in-situ resource utilization (ISRU). In mission design, there is also an increasing interest in the utilization of the distributed spacecraft, to benefit from specialization and redundancy. The ability of distributed spacecraft to navigate autonomously in the proximity of a SCB is indispensable for the successful realization of ISRU mission objectives. Quasi-autonomous methods currently used for proximity navigation require extensive ground support for mapping and model development, which can be an impediment for large scale multi-spacecraft ISRU missions in the future. It is prudent to leverage the advances in terrestrial robotic navigation to investigate the development of novel methods for autonomous navigation of spacecraft. The primary objective of the work presented in this thesis is to evaluate the feasibility and investigate the development of methods based on graph-based simultaneous localization and mapping (SLAM), a popular algorithm used in terrestrial autonomous navigation, for the autonomous navigation of distributed spacecraft in the proximity of SCBs. To this end, recent research in graph-based SLAM is extensively studied to identify strategies used to enable multi-agent navigation. The spacecraft navigation requirement is formulated as a graph-based SLAM problem using metric GraphSLAM or topometric graph-based SLAM. Techniques developed based on the identified strategies namely, map merging, inter-spacecraft measurements and relative localization are then applied to this formulation to enable distributed spacecraft navigation. In each case, navigation is formulated in terms of its application to a proximity operation scenario that best suits the multi-agent navigation technique. Several challenges related to the application of graph-based SLAM for spacecraft navigation, such as computational cost and illumination variation are also identified and addressed in the development of these methods. Experiments are performed using simulated models of asteroids and spacecraft dynamics, comparing the estimated states of the spacecraft and landmarks to the assumed true states. The results from the experiments indicate a consistent and robust state determination process, suggesting the suitability of the application of multi-agent navigation techniques to graph-based SLAM for enabling the autonomous navigation of distributed spacecraft near SCBs

Recent Advances in Indoor Localization Systems and Technologies

Despite the enormous technical progress seen in the past few years, the maturity of indoor localization technologies has not yet reached the level of GNSS solutions. The 23 selected papers in this book present the recent advances and new developments in indoor localization systems and technologies, propose novel or improved methods with increased performance, provide insight into various aspects of quality control, and also introduce some unorthodox positioning methods

Augmenting Wiring Diagrams of Neural Circuits with Activity in Larval Drosophila

Neural circuit models explain an animal's behavior as evoked activity of different circuit elements of its nervous system. Synaptic wiring diagrams mapped from structural imaging of nervous systems guide modeling of neural circuits on the basis of connectivity. However, connectivity alone may not sufficiently constrain the set of plausible circuit hypotheses for empirical study. Combining structural imaging of synaptic connectivity with functional information from activity imaging can further constrain these hypotheses to create unequivocal neural circuit models. This thesis develops computational methods and tools to cross-reference structural and activity imaging of explant larval Drosophila central nervous systems at cellular resolution. Augmenting synaptic wiring diagrams with activity maps via these methods relates circuit structure and function at the neuronal level on a per-behavior basis. Neuronal activity of larval central nervous systems expressing pan-neuronal calcium indicators is imaged in a light sheet microscope, which are then structurally imaged with high throughput electron microscopy. Methods and tools are provided for the assembly of these image volumes, spatial registration between imaging modalities, automated detection of relevant tissue and cellular structures in each, extraction of activity time series, and morphological identification of neurons in structural imaging using reference wiring diagrams mapped from other larvae. Using these methods, existing wiring diagrams mapped from a reference first instar larva were identified with neurons in a larva augmented with activity information for a neural circuit involved in peristaltic motor control. This demonstrates the feasibility of the contributed methods to associate the wiring diagrams of arbitrary circuits of interest with activity timeseries across multiple individuals, behaviors, and behavioral bouts. To demonstrate capability to augment wiring diagrams with information besides activity, these methods are also applied to multiple larvae each expressing specific neurotransmitter labels rather than calcium indicators in the light sheet microscopy. This work scaffolds future modeling of circuits underlying behavior that can only be mechanistically understood in the context of multi-modal observation of synaptic connectivity, functional activity and molecular markers. The methods developed also enable comparative connectomics between multiple individuals, which is necessary to study inter-individual variability in circuits and to observe experimental intervention in the development, structure, and function of neural circuits.Howard Hughes Medical Institute Janelia Research Campu

Forests

In this paper, we provide an overview of positioning systems for moving resources in forest and fire management and review the related literature. Emphasis is placed on the accuracy and range of different localization and location-sharing methods, particularly in forested environments and in the absence of conventional cellular or internet connectivity. We then conduct a second review of literature and concepts related to several emerging, broad themes in data science, including the terms |, |, |, |, |, |, and |. Our objective in this second review is to inform how these broader concepts, with implications for networking and analytics, may help to advance natural resource management and science in the future. Based on methods, themes, and concepts that arose in our systematic reviews, we then augmented the paper with additional literature from wildlife and fisheries management, as well as concepts from video object detection, relative positioning, and inventory-tracking that are also used as forms of localization. Based on our reviews of positioning technologies and emerging data science themes, we present a hierarchical model for collecting and sharing data in forest and fire management, and more broadly in the field of natural resources. The model reflects tradeoffs in range and bandwidth when recording, processing, and communicating large quantities of data in time and space to support resource management, science, and public safety in remote areas. In the hierarchical approach, wearable devices and other sensors typically transmit data at short distances using Bluetooth, Bluetooth Low Energy (BLE), or ANT wireless, and smartphones and tablets serve as intermediate data collection and processing hubs for information that can be subsequently transmitted using radio networking systems or satellite communication. Data with greater spatial and temporal complexity is typically processed incrementally at lower tiers, then fused and summarized at higher levels of incident command or resource management. Lastly, we outline several priority areas for future research to advance big data analytics in natural resources.U01 OH010841/OH/NIOSH CDC HHSUnited States/U54 OH007544/OH/NIOSH CDC HHSUnited States