A discussion on the validation tests employed to compare human action recognition methods using the MSR Action3D dataset

This paper aims to determine which is the best human action recognition method based on features extracted from RGB-D devices, such as the Microsoft Kinect. A review of all the papers that make reference to MSR Action3D, the most used dataset that includes depth information acquired from a RGB-D device, has been performed. We found that the validation method used by each work differs from the others. So, a direct comparison among works cannot be made. However, almost all the works present their results comparing them without taking into account this issue. Therefore, we present different rankings according to the methodology used for the validation in orden to clarify the existing confusion.Comment: 16 pages and 7 table

Understanding egocentric human actions with temporal decision forests

Understanding human actions is a fundamental task in computer vision with a wide range of applications including pervasive health-care, robotics and game control. This thesis focuses on the problem of egocentric action recognition from RGB-D data, wherein the world is viewed through the eyes of the actor whose hands describe the actions. The main contributions of this work are its findings regarding egocentric actions as described by hands in two application scenarios and a proposal of a new technique that is based on temporal decision forests. The thesis first introduces a novel framework to recognise fingertip writing in mid-air in the context of human-computer interaction. This framework detects whether the user is writing and tracks the fingertip over time to generate spatio-temporal trajectories that are recognised by using a Hough forest variant that encourages temporal consistency in prediction. A problem with using such forest approach for action recognition is that the learning of temporal dynamics is limited to hand-crafted temporal features and temporal regression, which may break the temporal continuity and lead to inconsistent predictions. To overcome this limitation, the thesis proposes transition forests. Besides any temporal information that is encoded in the feature space, the forest automatically learns the temporal dynamics during training, and it is exploited in inference in an online and efficient manner achieving state-of-the-art results. The last contribution of this thesis is its introduction of the first RGB-D benchmark to allow for the study of egocentric hand-object actions with both hand and object pose annotations. This study conducts an extensive evaluation of different baselines, state-of-the art approaches and temporal decision forest models using colour, depth and hand pose features. Furthermore, it extends the transition forest model to incorporate data from different modalities and demonstrates the benefit of using hand pose features to recognise egocentric human actions. The thesis concludes by discussing and analysing the contributions and proposing a few ideas for future work.Open Acces

Effect of anthropic disturbances on the activity pattern of two generalist mesocarnivores inhabiting Mediterranean forestry plantations

Humans have been altering the Mediterranean landscapes for millennia. To diminish the probability of encounters with domestic animals, humans and their activities, many species adjust their behavior to become more nocturnal. Even habitat-generalist species, such as red fox and stone marten that are somehow tolerant to environmental changes, might be affected by anthropic disturbances. Nevertheless, only a small number of studies were implemented in Iberia targeting these mesocarnivores’ activity patterns, and fewer have assessed the temporal ecology of these species in Eucalyptus plantations, the current main forest cover in Portugal. Based on camera traps, we aimed to analyze: 1) the temporal and spatio-temporal activity patterns of red fox and stone marten; and 2) how they are affected by distinct human disturbances (i.e., humans, livestock, dogs, plantations, and hunting). Foxes presented a higher crepuscular activity, while martens were entirely nocturnal, suggesting some avoidance behavior. Both mesocarnivores showed a higher overlap with dogs’ activity than with humans or livestock. Foxes’ activity patterns vary between seasons and habitats but were not influenced by the hunting period. Results suggest that both mesocarnivores, besides setting apart their activity from humans related disturbances, also show a tendency to temporally avoid each other. While the increase of nocturnality may indicate an anthropic disturbance impact, a reduction of activity overlap between mesocarnivores may be a strategy to reduce competition. These results may help support the sustainable management of landscapes by highlighting critical periods where activity overlaps may occur, and thus the anthropic impacts on wildlife are higher.info:eu-repo/semantics/publishedVersio

Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks

Human action recognition in 3D skeleton sequences has attracted a lot of research attention. Recently, Long Short-Term Memory (LSTM) networks have shown promising performance in this task due to their strengths in modeling the dependencies and dynamics in sequential data. As not all skeletal joints are informative for action recognition, and the irrelevant joints often bring noise which can degrade the performance, we need to pay more attention to the informative ones. However, the original LSTM network does not have explicit attention ability. In this paper, we propose a new class of LSTM network, Global Context-Aware Attention LSTM (GCA-LSTM), for skeleton based action recognition. This network is capable of selectively focusing on the informative joints in each frame of each skeleton sequence by using a global context memory cell. To further improve the attention capability of our network, we also introduce a recurrent attention mechanism, with which the attention performance of the network can be enhanced progressively. Moreover, we propose a stepwise training scheme in order to train our network effectively. Our approach achieves state-of-the-art performance on five challenging benchmark datasets for skeleton based action recognition