Activity-conditioned continuous human pose estimation for performance analysis of athletes using the example of swimming

In this paper we consider the problem of human pose estimation in real-world videos of swimmers. Swimming channels allow filming swimmers simultaneously above and below the water surface with a single stationary camera. These recordings can be used to quantitatively assess the athletes' performance. The quantitative evaluation, so far, requires manual annotations of body parts in each video frame. We therefore apply the concept of CNNs in order to automatically infer the required pose information. Starting with an off-the-shelf architecture, we develop extensions to leverage activity information - in our case the swimming style of an athlete - and the continuous nature of the video recordings. Our main contributions are threefold: (a) We apply and evaluate a fine-tuned Convolutional Pose Machine architecture as a baseline in our very challenging aquatic environment and discuss its error modes, (b) we propose an extension to input swimming style information into the fully convolutional architecture and (c) modify the architecture for continuous pose estimation in videos. With these additions we achieve reliable pose estimates with up to +16% more correct body joint detections compared to the baseline architecture.Comment: 10 pages, 9 figures, accepted at WACV 201

Indoor Activity Detection and Recognition for Sport Games Analysis

Activity recognition in sport is an attractive field for computer vision research. Game, player and team analysis are of great interest and research topics within this field emerge with the goal of automated analysis. The very specific underlying rules of sports can be used as prior knowledge for the recognition task and present a constrained environment for evaluation. This paper describes recognition of single player activities in sport with special emphasis on volleyball. Starting from a per-frame player-centered activity recognition, we incorporate geometry and contextual information via an activity context descriptor that collects information about all player's activities over a certain timespan relative to the investigated player. The benefit of this context information on single player activity recognition is evaluated on our new real-life dataset presenting a total amount of almost 36k annotated frames containing 7 activity classes within 6 videos of professional volleyball games. Our incorporation of the contextual information improves the average player-centered classification performance of 77.56% by up to 18.35% on specific classes, proving that spatio-temporal context is an important clue for activity recognition.Comment: Part of the OAGM 2014 proceedings (arXiv:1404.3538

Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly "intelligent" behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication

A pilot study comparing the metabolic profiles of elite-level athletes from different sporting disciplines

Background: The outstanding performance of an elite athlete might be associated with changes in their blood metabolic profile. The aims of this study were to compare the blood metabolic profiles between moderate- and high-power and endurance elite athletes and to identify the potential metabolic pathways underlying these differences. Methods: Metabolic profiling of serum samples from 191 elite athletes from different sports disciplines (121 high- and 70 moderate-endurance athletes, including 44 high- and 144 moderate-power athletes), who participated in national or international sports events and tested negative for doping abuse at anti-doping laboratories, was performed using non-targeted metabolomics-based mass spectroscopy combined with ultrahigh-performance liquid chromatography. Multivariate analysis was conducted using orthogonal partial least squares discriminant analysis. Differences in metabolic levels between high- and moderate-power and endurance sports were assessed by univariate linear models. Results: Out of 743 analyzed metabolites, gamma-glutamyl amino acids were significantly reduced in both high-power and high-endurance athletes compared to moderate counterparts, indicating active glutathione cycle. High-endurance athletes exhibited significant increases in the levels of several sex hormone steroids involved in testosterone and progesterone synthesis, but decreases in diacylglycerols and ecosanoids. High-power athletes had increased levels of phospholipids and xanthine metabolites compared to moderate-power counterparts. Conclusions: This pilot data provides evidence that high-power and high-endurance athletes exhibit a distinct metabolic profile that reflects steroid biosynthesis, fatty acid metabolism, oxidative stress, and energy-related metabolites. Replication studies are warranted to confirm differences in the metabolic profiles associated with athletes’ elite performance in independent data sets, aiming ultimately for deeper understanding of the underlying biochemical processes that could be utilized as biomarkers with potential therapeutic implications

Going Deeper into First-Person Activity Recognition

We bring together ideas from recent work on feature design for egocentric action recognition under one framework by exploring the use of deep convolutional neural networks (CNN). Recent work has shown that features such as hand appearance, object attributes, local hand motion and camera ego-motion are important for characterizing first-person actions. To integrate these ideas under one framework, we propose a twin stream network architecture, where one stream analyzes appearance information and the other stream analyzes motion information. Our appearance stream encodes prior knowledge of the egocentric paradigm by explicitly training the network to segment hands and localize objects. By visualizing certain neuron activation of our network, we show that our proposed architecture naturally learns features that capture object attributes and hand-object configurations. Our extensive experiments on benchmark egocentric action datasets show that our deep architecture enables recognition rates that significantly outperform state-of-the-art techniques -- an average $6.6\%$ increase in accuracy over all datasets. Furthermore, by learning to recognize objects, actions and activities jointly, the performance of individual recognition tasks also increase by $30\%$ (actions) and $14\%$ (objects). We also include the results of extensive ablative analysis to highlight the importance of network design decisions.

Scoping out urban areas of tourist interest though geolocated social media data: Bucharest as a case study

Social media data has frequently sourced research on topics such as traveller planning or the factors that influence travel decisions. The literature on the location of tourist activities, however, is scarce. The studies in this line that do exist focus mainly on identifying points of interest and rarely on the urban areas that attract tourists. Specifically, as acknowledged in the literature, tourist attractions produce major imbalances with respect to adjacent urban areas. The present study aims to fill this research gap by addressing a twofold objective. The first was to design a methodology allowing to identify the preferred tourist areas based on concentrations of places and activities. The tourist area was delimited using Instasights heatmaps information and the areas of interest were identified by linking data from the location-based social network Foursquare to TripAdvisor’s database. The second objective was to delimit areas of interest based on users’ existing urban dynamics. The method provides a thorough understanding of functional diversity and the location of a city’s different functions. In this way, it contributes to a better understanding of the spatial distribution imbalances of tourist activities. Tourist areas of interest were revealed via the identification of users’ preferences and experiences. A novel methodology was thus created that can be used in the design of future tourism strategies or, indeed, in urban planning. The city of Bucharest, Romania, was taken as a case study to develop this exploratory research.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This research has been partially funded by the Valencian Conselleria de Innovación, Universidades, Ciencia y Sociedad Digital, Generalitat Valenciana and the European Social Fund (ACIF/2020/173); and by the University of Alicante—Vicerrectorado de Investigación (GRE 21-15)