Real-time human action recognition on an embedded, reconfigurable video processing architecture

Copyright @ 2008 Springer-Verlag.In recent years, automatic human motion recognition has been widely researched within the computer vision and image processing communities. Here we propose a real-time embedded vision solution for human motion recognition implemented on a ubiquitous device. There are three main contributions in this paper. Firstly, we have developed a fast human motion recognition system with simple motion features and a linear Support Vector Machine (SVM) classifier. The method has been tested on a large, public human action dataset and achieved competitive performance for the temporal template (eg. “motion history image”) class of approaches. Secondly, we have developed a reconfigurable, FPGA based video processing architecture. One advantage of this architecture is that the system processing performance can be reconfiured for a particular application, with the addition of new or replicated processing cores. Finally, we have successfully implemented a human motion recognition system on this reconfigurable architecture. With a small number of human actions (hand gestures), this stand-alone system is performing reliably, with an 80% average recognition rate using limited training data. This type of system has applications in security systems, man-machine communications and intelligent environments.DTI and Broadcom Ltd

Monocular Expressive Body Regression through Body-Driven Attention

To understand how people look, interact, or perform tasks, we need to quickly and accurately capture their 3D body, face, and hands together from an RGB image. Most existing methods focus only on parts of the body. A few recent approaches reconstruct full expressive 3D humans from images using 3D body models that include the face and hands. These methods are optimization-based and thus slow, prone to local optima, and require 2D keypoints as input. We address these limitations by introducing ExPose (EXpressive POse and Shape rEgression), which directly regresses the body, face, and hands, in SMPL-X format, from an RGB image. This is a hard problem due to the high dimensionality of the body and the lack of expressive training data. Additionally, hands and faces are much smaller than the body, occupying very few image pixels. This makes hand and face estimation hard when body images are downscaled for neural networks. We make three main contributions. First, we account for the lack of training data by curating a dataset of SMPL-X fits on in-the-wild images. Second, we observe that body estimation localizes the face and hands reasonably well. We introduce body-driven attention for face and hand regions in the original image to extract higher-resolution crops that are fed to dedicated refinement modules. Third, these modules exploit part-specific knowledge from existing face- and hand-only datasets. ExPose estimates expressive 3D humans more accurately than existing optimization methods at a small fraction of the computational cost. Our data, model and code are available for research at https://expose.is.tue.mpg.de .Comment: Accepted in ECCV'20. Project page: http://expose.is.tue.mpg.d

Biview learning for human posture segmentation from 3D points cloud

Posture segmentation plays an essential role in human motion analysis. The state-of-the-art method extracts sufficiently high-dimensional features from 3D depth images for each 3D point and learns an efficient body part classifier. However, high-dimensional features are memory-consuming and difficult to handle on large-scale training dataset. In this paper, we propose an efficient two-stage dimension reduction scheme, termed biview learning, to encode two independent views which are depth-difference features (DDF) and relative position features (RPF). Biview learning explores the complementary property of DDF and RPF, and uses two stages to learn a compact yet comprehensive low-dimensional feature space for posture segmentation. In the first stage, discriminative locality alignment (DLA) is applied to the high-dimensional DDF to learn a discriminative low-dimensional representation. In the second stage, canonical correlation analysis (CCA) is used to explore the complementary property of RPF and the dimensionality reduced DDF. Finally, we train a support vector machine (SVM) over the output of CCA. We carefully validate the effectiveness of DLA and CCA utilized in the two-stage scheme on our 3D human points cloud dataset. Experimental results show that the proposed biview learning scheme significantly outperforms the state-of-the-art method for human posture segmentation. © 2014 Qiao et al

Ellenberg-type indicator values for European vascular plant species

Aims: Ellenberg-type indicator values are expert-based rankings of plant species according to their ecological optima on main environmental gradients. Here we extend the indicator-value system proposed by Heinz Ellenberg and co-authors for Central Europe by incorporating other systems of Ellenberg-type indicator values (i.e., those using scales compatible with Ellenberg values) developed for other European regions. Our aim is to create a harmonized data set of Ellenberg-type indicator values applicable at the European scale. Methods: We collected European data sets of indicator values for vascular plants and selected 13 data sets that used the nine-, ten- or twelve-degree scales defined by Ellenberg for light, temperature, moisture, reaction, nutrients and salinity. We compared these values with the original Ellenberg values and used those that showed consistent trends in regression slope and coefficient of determination. We calculated the average value for each combination of species and indicator values from these data sets. Based on species’ co-occurrences in European vegetation plots, we also calculated new values for species that were not assigned an indicator value. Results: We provide a new data set of Ellenberg-type indicator values for 8908 European vascular plant species (8168 for light, 7400 for temperature, 8030 for moisture, 7282 for reaction, 7193 for nutrients, and 7507 for salinity), of which 398 species have been newly assigned to at least one indicator value. Conclusions: The newly introduced indicator values are compatible with the original Ellenberg values. They can be used for large-scale studies of the European flora and vegetation or for gap-filling in regional data sets. The European indicator values and the original and taxonomically harmonized regional data sets of Ellenberg-type indicator values are available in the Supporting Information and the Zenodo repository

Climatic predictors of species distributions neglect biophysiologically meaningful variables

This is the final version. Available on open access from Wiley via the DOI in this record.Aim: Species distribution models (SDMs) have played a pivotal role in predicting how species might respond to climate change. To generate reliable and realistic predictions from these models requires the use of climate variables that adequately capture physiological responses of species to climate and therefore provide a proximal link between climate and their distributions. Here, we examine whether the climate variables used in plant SDMs are different from those known to influence directly plant physiology. Location: Global. Methods: We carry out an extensive, systematic review of the climate variables used to model the distributions of plant species and provide comparison to the climate variables identified as important in the plant physiology literature. We calculate the top ten SDM and physiology variables at 2.5 degree spatial resolution for the globe and use principal component analyses and multiple regression to assess similarity between the climatic variation described by both variable sets. Results: We find that the most commonly used SDM variables do not reflect the most important physiological variables and differ in two main ways: (i) SDM variables rely on seasonal or annual rainfall as simple proxies of water available to plants and neglect more direct measures such as soil water content; and (ii) SDM variables are typically averaged across seasons or years and overlook the importance of climatic events within the critical growth period of plants. We identify notable differences in their spatial gradients globally and show where distal variables may be less reliable proxies for the variables to which species are known to respond. Main conclusions: There is a growing need for the development of accessible, fine-resolution global climate surfaces of physiological variables. This would provide a means to improve the reliability of future range predictions from SDMs and support efforts to conserve biodiversity in a changing climate

Micro, Meso, and Macro Data Collection and Analysis, as a Method for Speculative and Artistic Exploration

In this work, an attempt is made to explore the emerging computationally-enhanced private and public environments by analyzing their ecological transitions and its implications on practical, aesthetic, and speculative dimensions. The author has decided to methodologically dissect the multiplicity of information that exists on many possible-to-detect scales (micro, meso, macro), and utilize this extraction as a tool for experimentation and redefinition. With the use of custom-made hardware and software utilities (sensor devices, sentiment analysis algorithms, online APIs, and many more), a vast amount of data is collected and used as a multidimensional layered architecture that constantly shifts and transforms. The extracted and analyzed content of the collection becomes the essence of the work that is shaped and refined through digital and physical making – middleware, recursion, mapping – and by utilizing technological objects within the physical space, the creative process is augmented and amplified, exploring not only new practices and novel applications, but rather redefining behavior, thought-process, and context