a multimodal dataset for the analysis of movement qualities in karate martial art

A multimodal dataset is presented, which has been collected for analyzing and measuring the quality of movement performed during sport activities. Martial arts (namely karate) are taken as test-beds for investigation. Karate encompasses predefined sequences of movements ("katas") that can be carried out with different qualities, e.g., by performers at different skill levels (highly vs. poorly skilled). The experimental setup and method are described. The dataset is composed of motion capture (MoCap) data, synchronized with video and audio recordings, of several participants with different levels of experience. The raw MoCap data are independent of any particular post-processing algorithm and can be used for other research purposes. In the second part of the paper, a set of measures is proposed to evaluate a kata performance. They are based on the geometrical and kinematic features, such as posture correctness and synchronization between limbs. and were chosen according to karate experts' opinion

Analysis of movement quality in full-body physical activities

Full-body human movement is characterized by fine-grain expressive qualities that humans are easily capable of exhibiting and recognizing in others' movement. In sports (e.g., martial arts) and performing arts (e.g., dance), the same sequence of movements can be performed in a wide range of ways characterized by different qualities, often in terms of subtle (spatial and temporal) perturbations of the movement. Even a non-expert observer can distinguish between a top-level and average performance by a dancer or martial artist. The difference is not in the performed movements-the same in both cases-but in the \u201cquality\u201d of their performance. In this article, we present a computational framework aimed at an automated approximate measure of movement quality in full-body physical activities. Starting from motion capture data, the framework computes low-level (e.g., a limb velocity) and high-level (e.g., synchronization between different limbs) movement features. Then, this vector of features is integrated to compute a value aimed at providing a quantitative assessment of movement quality approximating the evaluation that an external expert observer would give of the same sequence of movements. Next, a system representing a concrete implementation of the framework is proposed. Karate is adopted as a testbed. We selected two different katas (i.e., detailed choreographies of movements in karate) characterized by different overall attitudes and expressions (aggressiveness, meditation), and we asked seven athletes, having various levels of experience and age, to perform them. Motion capture data were collected from the performances and were analyzed with the system. The results of the automated analysis were compared with the scores given by 14 karate experts who rated the same performances. Results show that the movement-quality scores computed by the system and the ratings given by the human observers are highly correlated (Pearson's correlations r = 0.84, p = 0.001 and r = 0.75, p = 0.005)

Towards a multimodal repository of expressive movement qualities in dance

In this paper, we present a new multimodal repository for the analysis of expressive movement qualities in dance. First, we discuss guidelines and methodology that we applied to create this repository. Next, the technical setup of recordings and the platform for capturing the synchronized audio-visual, physiological, and motion capture data are presented. The initial content of the repository consists of about 90 minutes of short dance performances movement sequences, and improvisations performed by four dancers, displaying three expressive qualities: Fluidity, Impulsivity, and Rigidity

Automated Analysis of Synchronization in Human Full-body Expressive Movement

The research presented in this thesis is focused on the creation of computational models for the study of human full-body movement in order to investigate human behavior and non-verbal communication. In particular, the research concerns the analysis of synchronization of expressive movements and gestures. Synchronization can be computed both on a single user (intra-personal), e.g., to measure the degree of coordination between the joints\u2019 velocities of a dancer, and on multiple users (inter-personal), e.g., to detect the level of coordination between multiple users in a group. The thesis, through a set of experiments and results, contributes to the investigation of both intra-personal and inter-personal synchronization applied to support the study of movement expressivity, and improve the state-of-art of the available methods by presenting a new algorithm to perform the analysis of synchronization

Multi-scale techniques for multi-dimensional data analysis

Large datasets of geometric data of various nature are becoming more and more available as sensors become cheaper and more widely used. Due to both their size and their noisy nature, special techniques must be employed to deal with them correctly. In order to efficiently handle this amount of data and to tackle the technical challenges they pose, we propose techniques that analyze a scalar signal by means of its critical points (i.e. maxima and minima), ranking them on a scale of importance, by which we can extrapolate important information of the input signal separating it from noise, thus dramatically reducing the complexity of the problem. In order to obtain a ranking of critical points we employ multi-scale techniques. The standard scale-space approach, however, is not sufficient when trying to track critical points across various scales. We start from an implementation of the scale-space which computes a linear interpolation between scales in order to make tracking of critical points easier. The linear interpolation of a process which is not itself linear, though, does not fulfill some theoretical properties of scale-space, thus making the tracking of critical points much harder. We propose an extension of this piecewiselinear scale-space implementation, which recovers the theoretical properties (e.g., to avoid the generation of new critical points as the scale increases) and keeps the tracking consistent. Next we combine the scale-space with another technique that comes from the topology theory: the classification of critical points based on their persistence value. While the scale-space applies a filtering in the frequency domain, by progressively smoothing the input signal with low-pass filters of increasing size, the computation of the persistence can be seen as a filtering applied in the amplitude domain, which progressively removes pairs of critical points based on their difference in amplitude. The two techniques, while being both relevant to the concept of scale, express different qualities of the critical points of the input signal; depending on the application domain we can use either of them, or, since they both have non-zero values only at critical points, they can be used together with a linear combination. The thesis will be structured as follows: In Chapter 1 we will present an overview on the problem of analyzing huge geometric datasets, focusing on the problem of dealing with their size and noise, and of reducing the problem to a subset of relevant samples. The Chapter 2 will contain a study of the state of the art in scale-space algorithms, followed by a more in-depth analysis of the virtually continuous framework used as base technique will be presented. In its last part, we will propose methods to extend these techniques in order to satisfy the axioms present in the continuous version of the scale-space and to have a stronger and more reliable tracking of critical points across scales, and the extraction of the persistence of critical points of a signal as a variant to the standard scale-space approach; we will show the differences between the two and discuss how to combine them. The Chapter 3 will introduce an ever growing source of data, the motion capture systems; we will motivate its importance by discussing the many applications in which it has been used for the past two decades. We will briefly summarize the different systems existing and then we will focus on a particular one, discussing its peculiarities and its output data. In Chapter 4, we will discuss the problem of studying intra-personal synchronization computed on data coming from such motion-capture systems. We will show how multi-scale approaches can be used to identify relevant instants in the motion and how these instants can be used to precisely study synchronization between the different parts of the body from which they are extracted. We will apply these techniques to the problem of generating a classifier to discriminate between martial artists of different skills who have been recorded doing karate\u2019s movements. In Chapter 5 will present a work on the automatic detection of relevant points of the human face from 3D data. We will show that the Gaussian curvature of the 3D surface is a good feature to distinguish the so-called fiducial points, but also that multi-scale techniques must be used to extract only relevant points and get rid of the noise. In closing, Chapter 6 will discuss an ongoing work about motion segmentation; after an introduction about the meaning and different possibilities of motion segmentation we will present the data we work with, the approach used to identify segments and some preliminary tools and results

Visualization to compare karate motion captures

Multi-dimensional time series from motion capture (MoCap) provide a rich source of data for human motion analysis, yet they are difficult to process and compare. We address MoCap data related to Karate katas, containing predefined sequences of movements, executed independently by several subjects with different timing and speed. We propose a combination of signal processing and data visualization techniques to analyze the misalignment between data from different subjects. We present a web app that implements this proposal, providing a visual comparison of time series, based on Dynamic Time Warping.XVII Workshop de Computación Gráfica.Red de Universidades con Carreras en Informátic

Exploring the impact of machine learning on dance performance: a systematic review

In recent years, the intersection of machine learning and dance has garnered increasing attention as a means to enhance the aesthetics and creativity of dance performance. Machine learning techniques, such as human-pose detection, have been utilized to analyze body movement and generate visual models in realtime, offering new approaches to traditional art form. However, despite the growing interest in this field, there is a lack of comprehensive research that evaluates the impact of machine learning on dance from multiple perspectives. This systematic review addresses this gap by examining the impact of machine learning on four key aspects of dance performance: choreographic creation support; dataset or network collection and training; improved techniques of human-pose detection; perception and new visual representations of dance movement. This exploration reveals that machine learning contributes both positively and negatively to artistic development, introducing a sense of novelty and heightened interest in the performance while concurrently prompting ethical concerns regarding artistic ownership and authorship. Through a nuanced analysis of machine learning's effects on key aspects of dance, this study aims to provide valuable insights into how these technologies can enhance the artistic practice in dance while signaling areas that demand further research in this rapidly evolving field

Evaluation of human movement qualities: A methodology based on transferable-utility games on graphs.

Abstract A novel computational method for the analysis of expressive full-body movement qualities is introduced, which exploits concepts and tools from graph theory and game theory. The human skeletal structure is modeled as an undirected graph, where the joints are the vertices and the edge set contains both physical and nonphysical links. Physical links correspond to connections between adjacent physical body joints (e.g., the forearm, which connects the elbow to the wrist). Nonphysical links act as \u201cbridges\u201d between parts of the body not directly connected by the skeletal structure, but sharing very similar feature values. The edge weights depend on features obtained by using Motion Capture data. Then, a mathematical game is constructed over the graph structure, where the vertices represent the players and the edges represent communication channels between them. Hence, the body movement is modeled in terms of a game built on the graph structure. Since the vertices and the edges contribute to the overall quality of the movement, the adopted game-theoretical model is of cooperative nature. A game-theoretical concept, called Shapley value, is exploited as a centrality index to estimate the contribution of each vertex to a shared goal (e.g., to the way a particular movement quality is transferred among the vertices). The proposed method is applied to a data set of Motion Capture data of subjects performing expressive movements, recorded in the framework of the H2020-ICT-2015 EU Project WhoLoDance, Project no. 688865. Results are presented: development of novel method, contribution to the scientific community with a new data corpus, application the discussed method to 100 movement recordings and creation of database archive of stimuli for further use in research studies in the framework of the WhoLoDance Project

Wearable Sensors and Smart Devices to Monitor Rehabilitation Parameters and Sports Performance: An Overview

A quantitative evaluation of kinetic parameters, the joint’s range of motion, heart rate, and breathing rate, can be employed in sports performance tracking and rehabilitation monitoring following injuries or surgical operations. However, many of the current detection systems are expensive and designed for clinical use, requiring the presence of a physician and medical staff to assist users in the device’s positioning and measurements. The goal of wearable sensors is to overcome the limitations of current devices, enabling the acquisition of a user’s vital signs directly from the body in an accurate and non–invasive way. In sports activities, wearable sensors allow athletes to monitor performance and body movements objectively, going beyond the coach’s subjective evaluation limits. The main goal of this review paper is to provide a comprehensive overview of wearable technologies and sensing systems to detect and monitor the physiological parameters of patients during post–operative rehabilitation and athletes’ training, and to present evidence that supports the efficacy of this technology for healthcare applications. First, a classification of the human physiological parameters acquired from the human body by sensors attached to sensitive skin locations or worn as a part of garments is introduced, carrying important feedback on the user’s health status. Then, a detailed description of the electromechanical transduction mechanisms allows a comparison of the technologies used in wearable applications to monitor sports and rehabilitation activities. This paves the way for an analysis of wearable technologies, providing a comprehensive comparison of the current state of the art of available sensors and systems. Comparative and statistical analyses are provided to point out useful insights for defining the best technologies and solutions for monitoring body movements. Lastly, the presented review is compared with similar ones reported in the literature to highlight its strengths and novelties