Going Deeper into Action Recognition: A Survey

Understanding human actions in visual data is tied to advances in complementary research areas including object recognition, human dynamics, domain adaptation and semantic segmentation. Over the last decade, human action analysis evolved from earlier schemes that are often limited to controlled environments to nowadays advanced solutions that can learn from millions of videos and apply to almost all daily activities. Given the broad range of applications from video surveillance to human-computer interaction, scientific milestones in action recognition are achieved more rapidly, eventually leading to the demise of what used to be good in a short time. This motivated us to provide a comprehensive review of the notable steps taken towards recognizing human actions. To this end, we start our discussion with the pioneering methods that use handcrafted representations, and then, navigate into the realm of deep learning based approaches. We aim to remain objective throughout this survey, touching upon encouraging improvements as well as inevitable fallbacks, in the hope of raising fresh questions and motivating new research directions for the reader

Unraveling spatiotemporal variability of arbuscular mycorrhizal fungi in a temperate grassland plot

© The Author(s), 2019. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Goldmann, K., Boeddinghaus, R. S., Klemmer, S., Regan, K. M., Heintz-Buschart, A., Fischer, M., Prati, D., Piepho, H., Berner, D., Marhan, S., Kandeler, E., Buscot, F., & Wubet, T. Unraveling spatiotemporal variability of arbuscular mycorrhizal fungi in a temperate grassland plot. Environmental Microbiology, 22(3),(2020): 873-888, doi:10.1111/1462-2920.14653.Soils provide a heterogeneous environment varying in space and time; consequently, the biodiversity of soil microorganisms also differs spatially and temporally. For soil microbes tightly associated with plant roots, such as arbuscular mycorrhizal fungi (AMF), the diversity of plant partners and seasonal variability in trophic exchanges between the symbionts introduce additional heterogeneity. To clarify the impact of such heterogeneity, we investigated spatiotemporal variation in AMF diversity on a plot scale (10 × 10 m) in a grassland managed at low intensity in southwest Germany. AMF diversity was determined using 18S rDNA pyrosequencing analysis of 360 soil samples taken at six time points within a year. We observed high AMF alpha‐ and beta‐diversity across the plot and at all investigated time points. Relationships were detected between spatiotemporal variation in AMF OTU richness and plant species richness, root biomass, minimal changes in soil texture and pH. The plot was characterized by high AMF turnover rates with a positive spatiotemporal relationship for AMF beta‐diversity. However, environmental variables explained only ≈20% of the variation in AMF communities. This indicates that the observed spatiotemporal richness and community variability of AMF was largely independent of the abiotic environment, but related to plant properties and the cooccurring microbiome.We thank the managers of the three Exploratories, Kirsten Reichel‐Jung, Swen Renner, Katrin Hartwich, Sonja Gockel, Kerstin Wiesner, and Martin Gorke for their work in maintaining the plot and project infrastructure; Christiane Fischer and Simone Pfeiffer for giving support through the central office, Michael Owonibi and Andreas Ostrowski for managing the central data base, and Eduard Linsenmair, Dominik Hessenmöller, Jens Nieschulze, Ernst‐Detlef Schulze, Wolfgang W. Weisser and the late Elisabeth Kalko for their role in setting up the Biodiversity Exploratories project. The work has been funded by the DFG Priority Program 1374 ‘Infrastructure‐Biodiversity‐Exploratories’ (BU 941/22‐1, BU 941/22‐3, KA 1590/8‐2, KA 1590/8‐3). Field work permits were issued by the responsible state environmental office of Baden‐Württemberg (according to § 72 BbgNatSchG). Likewise, we kindly thank Beatrix Schnabel, Melanie Günther and Sigrid Härtling for 454 sequencing in Halle. AHB gratefully acknowledges the support of the German Centre for Integrative Biodiversity Research (iDiv) Halle‐Jena‐Leipzig funded by the German Research Foundation (FZT 118). Authors declare no conflict of interests

Within-Subject Joint Independent Component Analysis of Simultaneous fMRI/ERP in an Auditory Oddball Paradigm

The integration of event-related potential (ERP) and functional magnetic resonance imaging (fMRI) can contribute to characterizing neural networks with high temporal and spatial resolution. This research aimed to determine the sensitivity and limitations of applying joint independent component analysis (jICA) within-subjects, for ERP and fMRI data collected simultaneously in a parametric auditory frequency oddball paradigm. In a group of 20 subjects, an increase in ERP peak amplitude ranging 1–8 μV in the time window of the P300 (350–700 ms), and a correlated increase in fMRI signal in a network of regions including the right superior temporal and supramarginal gyri, was observed with the increase in deviant frequency difference. JICA of the same ERP and fMRI group data revealed activity in a similar network, albeit with stronger amplitude and larger extent. In addition, activity in the left pre- and post-central gyri, likely associated with right hand somato-motor response, was observed only with the jICA approach. Within-subject, the jICA approach revealed significantly stronger and more extensive activity in the brain regions associated with the auditory P300 than the P300 linear regression analysis. The results suggest that with the incorporation of spatial and temporal information from both imaging modalities, jICA may be a more sensitive method for extracting common sources of activity between ERP and fMRI

The macroscopic effects of microscopic heterogeneity

Over the past decade, advances in super-resolution microscopy and particle-based modeling have driven an intense interest in investigating spatial heterogeneity at the level of single molecules in cells. Remarkably, it is becoming clear that spatiotemporal correlations between just a few molecules can have profound effects on the signaling behavior of the entire cell. While such correlations are often explicitly imposed by molecular structures such as rafts, clusters, or scaffolds, they also arise intrinsically, due strictly to the small numbers of molecules involved, the finite speed of diffusion, and the effects of macromolecular crowding. In this chapter we review examples of both explicitly imposed and intrinsic correlations, focusing on the mechanisms by which microscopic heterogeneity is amplified to macroscopic effect.Comment: 20 pages, 5 figures. To appear in Advances in Chemical Physic

Robust sound event detection in bioacoustic sensor networks

Bioacoustic sensors, sometimes known as autonomous recording units (ARUs), can record sounds of wildlife over long periods of time in scalable and minimally invasive ways. Deriving per-species abundance estimates from these sensors requires detection, classification, and quantification of animal vocalizations as individual acoustic events. Yet, variability in ambient noise, both over time and across sensors, hinders the reliability of current automated systems for sound event detection (SED), such as convolutional neural networks (CNN) in the time-frequency domain. In this article, we develop, benchmark, and combine several machine listening techniques to improve the generalizability of SED models across heterogeneous acoustic environments. As a case study, we consider the problem of detecting avian flight calls from a ten-hour recording of nocturnal bird migration, recorded by a network of six ARUs in the presence of heterogeneous background noise. Starting from a CNN yielding state-of-the-art accuracy on this task, we introduce two noise adaptation techniques, respectively integrating short-term (60 milliseconds) and long-term (30 minutes) context. First, we apply per-channel energy normalization (PCEN) in the time-frequency domain, which applies short-term automatic gain control to every subband in the mel-frequency spectrogram. Secondly, we replace the last dense layer in the network by a context-adaptive neural network (CA-NN) layer. Combining them yields state-of-the-art results that are unmatched by artificial data augmentation alone. We release a pre-trained version of our best performing system under the name of BirdVoxDetect, a ready-to-use detector of avian flight calls in field recordings.Comment: 32 pages, in English. Submitted to PLOS ONE journal in February 2019; revised August 2019; published October 201

Analysis of the visual spatiotemporal properties of American Sign Language.

Careful measurements of the temporal dynamics of speech have provided important insights into phonetic properties of spoken languages, which are important for understanding auditory perception. By contrast, analytic quantification of the visual properties of signed languages is still largely uncharted. Exposure to sign language is a unique experience that could shape and modify low-level visual processing for those who use it regularly (i.e., what we refer to as the Enhanced Exposure Hypothesis). The purpose of the current study was to characterize the visual spatiotemporal properties of American Sign Language (ASL) so that future studies can test the enhanced exposure hypothesis in signers, with the prediction that altered vision should be observed within, more so than outside, the range of properties found in ASL. Using an ultrasonic motion tracking system, we recorded the hand position in 3-dimensional space over time during sign language production of signs, sentences, and narratives. From these data, we calculated several metrics: hand position and eccentricity in space and hand motion speed. For individual signs, we also measured total distance travelled by the dominant hand and total duration of each sign. These metrics were found to fall within a selective range, suggesting that exposure to signs is a specific and unique visual experience, which might alter visual perceptual abilities in signers for visual information within the experienced range, even for non-language stimuli