Noise-in, Bias-out: Balanced and Real-time MoCap Solving

Real-time optical Motion Capture (MoCap) systems have not benefited from the advances in modern data-driven modeling. In this work we apply machine learning to solve noisy unstructured marker estimates in real-time and deliver robust marker-based MoCap even when using sparse affordable sensors. To achieve this we focus on a number of challenges related to model training, namely the sourcing of training data and their long-tailed distribution. Leveraging representation learning we design a technique for imbalanced regression that requires no additional data or labels and improves the performance of our model in rare and challenging poses. By relying on a unified representation, we show that training such a model is not bound to high-end MoCap training data acquisition, and exploit the advances in marker-less MoCap to acquire the necessary data. Finally, we take a step towards richer and affordable MoCap by adapting a body model-based inverse kinematics solution to account for measurement and inference uncertainty, further improving performance and robustness. Project page: https://moverseai.github.io/noise-tailComment: Project page: https://moverseai.github.io/noise-tai

Low-cost accurate skeleton tracking based on fusion of kinect and wearable inertial sensors

In this paper, we present a novel multi-sensor fusion method to build a human skeleton. We propose to fuse the joint po- sition information obtained from the popular Kinect sensor with more precise estimation of body segment orientations provided by a small number of wearable inertial sensors. The use of inertial sensors can help to address many of the well known limitations of the Kinect sensor. The precise calcu- lation of joint angles potentially allows the quantification of movement errors in technique training, thus facilitating the use of the low-cost Kinect sensor for accurate biomechani- cal purposes e.g. the improved human skeleton could be used in visual feedback-guided motor learning, for example. We compare our system to the gold standard Vicon optical mo- tion capture system, proving that the fused skeleton achieves a very high level of accuracy

HeartHealth: A cardiovascular disease home-based rehabilitation system

The increasing pressure on medical institutions around the world requires health care professionals to be prescribing homebased exercise rehabilitation treatments to empower patients to self-monitor their rehabilitation journey. Home-based exercise rehabilitation has shown to be highly effective in treating conditions such as Cardiovascular Disease (CVD). However, adherence to home-based exercise rehabilitation remains low. Possible causes for this are that patients are not monitored, they cannot be confident that they are performing the exercise correctly or accurately and they receive no feedback. This paper proposes HeartHealth, a novel patient-centric gamified exercise rehabilitation platform that can help address the issue of adherence to these programmes. The key functionality is the ability to record the patient movements and compare them against the exercises that have been prescribed in order to return feedback to the patient and to the health care professional, as well. In order to synthesize a compact fully operational system able to work in real life scenarios, tools and services from FI-PPP projects, FIWARE1 and FI-STAR2, were exploited and a new FI-STAR component, Motion Evaluation Specific Enabler (SE), was designed and developed. The HeartHealth system brings together real-time cloud-based motion evaluation coupled with accurate low-cost motion capture, a personalised exercise rehabilitation programme and an intuitive and fun serious game interface, designed specifically with a Cardiac Rehabilitation population in mind

HeartHealth: new adventures in serious gaming

We present a novel, low-cost, interactive, exercise-based rehabilitation system. Our research involves the investigation and development of patient-centric, sensor-based rehabilitation games and surrounding technologies. HeartHealth is designed to provide a safe, personalised and fun exercise environment that could be deployed in any exercise based rehabilitation program. HeartHealth utilises a cloud-based patient information management system built on FIWARE Generic Enablers,and motion tracking coupled with our sophisticated motion comparison algorithms. Users can record customised exercises through a doctors interface and then play the rehabilitation game where they must perform a sequence of their exercises in order to complete the game scenario. Their exercises are monitored, recorded and compared by our Motion Evaluation software and real-time feedback is than given based on the users performance

A technology platform for enabling behavioural change as a “PATHway” towards better self-management of CVD

We describe a technology platform developed as part of a novel approach to technology-enabled exercise-based Cardiac Rehabilitation (CR), termed PATHway. We explain the overall concept and explain how technology can facilitate remote participation and better adherence to communitybased long-term Phase III CR. The demo will showcase the user experience of interacting with the PATHway system, including navigation and manual data entry, whilst also demonstrating real-time sensing and analysis of exercise movements and automatic adaptation of exercise based on physiological response

Interactive games for preservation and promotion of sporting movements

In this paper we describe two interactive applications for capturing the motion signatures associated with key skills of traditional sports and games. We first present the case for sport as an important example of intangible cultural heritage. We then explain that sport requires special consideration in terms of digitization for preservation as the key aspects to be digitized are the characteristic movement signatures of such sports. We explain that, given the nature of traditional sporting agencies, this requires low-cost motion capture technology. Furthermore we argue that in order to ensure ongoing preservation, this should be provided via fun interactive gaming scenarios that promote uptake of the sports, particularly among children. We then present two such games that we have developed and illustrate their performance

HUMAN4D: A human-centric multimodal dataset for motions and immersive media

We introduce HUMAN4D, a large and multimodal 4D dataset that contains a variety of human activities simultaneously captured by a professional marker-based MoCap, a volumetric capture and an audio recording system. By capturing 2 female and 2 male professional actors performing vari

SHREC 2018 - Protein Shape Retrieval

Proteins are macromolecules central to biological processes that display a dynamic and complex surface. They display multiple conformations differing by local (residue side-chain) or global (loop or domain) structural changes which can impact drastically their global and local shape. Since the structure of proteins is linked to their function and the disruption of their interactions can lead to a disease state, it is of major importance to characterize their shape. In the present work, we report the performance in enrichment of six shape-retrieval methods (3D-FusionNet, GSGW, HAPT, DEM, SIWKS and WKS) on a 2 267 protein structures dataset generated for this protein shape retrieval track of SHREC’18