429 research outputs found
Gait Analysis and Rehabilitation Using Web-Based Pose Estimation
Gait abnormalities are one of the most common health conditions in the elderly population, with almost one in three people over 60 experiencing symptoms that disrupt their movement [1]. These symptoms can cause disability [2] and present an increased fall risk [3] [4]. Detecting these abnormalities early is, therefore, crucial as it reduces the likelihood of injuries and accidents.
Current treatments for gait abnormalities depend on the condition, but many treatment plans commonly incorporate some form of physiotherapy. Clinicians typically deliver physiotherapy in the form of gait assessments and targeted exercises or therapies. Recent research has also shown that virtual reality (VR) treadmill walking, using motion capture technology, can be an effective method of treating certain gait abnormalities [5] [6] [7]. This thesis covers the development of a web-based VR treadmill walking system to make VR physiotherapy cheaper and more accessible. The system uses convolutional neural networks to assess the patient’s gait from an RGB webcam feed and provides them with live feedback on their body position within a VR environment. The system’s gait assessment capabilities are validated by comparing it to a gold standard – the OptiTrack motion capture system.
The results demonstrate that the system’s percentage error (ϵ˜%) was much less for temporal gait metrics (0.24 < ϵ˜< 12.40) than it was for spatial ones (70.90 < ϵ˜% < 79.72). Four out of five spatial metrics also had a “very strong correlation” (0.74 < r < 0.86) when compared to the OptiTrack’s metrics, meaning the accuracy could be increased using a gain factor. These findings establish the basis for a similar study with a larger sample size. They also raise the possibility that this system could analyse gait in the clinic and the home without specialist motion capture equipment or facilities
SwimmerNET: Underwater 2D Swimmer Pose Estimation Exploiting Fully Convolutional Neural Networks
Professional swimming coaches make use of videos to evaluate their athletes' performances. Specifically, the videos are manually analyzed in order to observe the movements of all parts of the swimmer's body during the exercise and to give indications for improving swimming technique. This operation is time-consuming, laborious and error prone. In recent years, alternative technologies have been introduced in the literature, but they still have severe limitations that make their correct and effective use impossible. In fact, the currently available techniques based on image analysis only apply to certain swimming styles; moreover, they are strongly influenced by disturbing elements (i.e., the presence of bubbles, splashes and reflections), resulting in poor measurement accuracy. The use of wearable sensors (accelerometers or photoplethysmographic sensors) or optical markers, although they can guarantee high reliability and accuracy, disturb the performance of the athletes, who tend to dislike these solutions. In this work we introduce swimmerNET, a new marker-less 2D swimmer pose estimation approach based on the combined use of computer vision algorithms and fully convolutional neural networks. By using a single 8 Mpixel wide-angle camera, the proposed system is able to estimate the pose of a swimmer during exercise while guaranteeing adequate measurement accuracy. The method has been successfully tested on several athletes (i.e., different physical characteristics and different swimming technique), obtaining an average error and a standard deviation (worst case scenario for the dataset analyzed) of approximately 1 mm and 10 mm, respectively
Augmented Human Inspired Phase Variable Using a Canonical Dynamical System
Accurately parameterizing human gait is highly important in the continued development of assistive robotics, including but not limited to lower limb prostheses and exoskeletons. Previous studies introduce the idea of time-invariant real-time gait parameterization via human-inspired phase variables. The phase represents the location or percent of the gait cycle the user has progressed through. This thesis proposes an alternative approach for determining the gait phase leveraging previous methods and a canonical dynamical system.
Human subject experiments demonstrate the ability to accurately produce a phase variable corresponding to the human gait progression for various walking configurations. Configurations include changes in incline and speed. Results show an augmented real-time approach capable of adapting to different walking conditions
Designing and developing a vision-based system to investigate the emotional effects of news on short sleep at noon: an experimental case study
Background: Sleep is a critical factor in maintaining good health, and its impact on various diseases has been recognized by scientists. Understanding sleep patterns and quality is crucial for investigating sleep-related disorders and their potential links to health conditions. The development of non-intrusive and contactless methods for analyzing sleep data is essential for accurate diagnosis and treatment. Methods: A novel system called the sleep visual analyzer (VSleep) was designed to analyze sleep movements and generate reports based on changes in body position angles. The system utilized camera data without requiring any physical contact with the body. A Python graphical user interface (GUI) section was developed to analyze body movements during sleep and present the data in an Excel format. To evaluate the effectiveness of the VSleep system, a case study was conducted. The participants' movements during daytime naps were recorded. The study also examined the impact of different types of news (positive, neutral, and negative) on sleep patterns. Results: The system successfully detected and recorded various angles formed by participants' bodies, providing detailed information about their sleep patterns. The results revealed distinct effects based on the news category, highlighting the potential impact of external factors on sleep quality and behaviors. Conclusions: The sleep visual analyzer (VSleep) demonstrated its efficacy in analyzing sleep-related data without the need for accessories. The VSleep system holds great potential for diagnosing and investigating sleep-related disorders. The proposed system is affordable, easy to use, portable, and a mobile application can be developed to perform the experiment and prepare the results
Design Patterns for Situated Visualization in Augmented Reality
Situated visualization has become an increasingly popular research area in
the visualization community, fueled by advancements in augmented reality (AR)
technology and immersive analytics. Visualizing data in spatial proximity to
their physical referents affords new design opportunities and considerations
not present in traditional visualization, which researchers are now beginning
to explore. However, the AR research community has an extensive history of
designing graphics that are displayed in highly physical contexts. In this
work, we leverage the richness of AR research and apply it to situated
visualization. We derive design patterns which summarize common approaches of
visualizing data in situ. The design patterns are based on a survey of 293
papers published in the AR and visualization communities, as well as our own
expertise. We discuss design dimensions that help to describe both our patterns
and previous work in the literature. This discussion is accompanied by several
guidelines which explain how to apply the patterns given the constraints
imposed by the real world. We conclude by discussing future research directions
that will help establish a complete understanding of the design of situated
visualization, including the role of interactivity, tasks, and workflows.Comment: To appear in IEEE VIS 202
EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild
We present EMDB, the Electromagnetic Database of Global 3D Human Pose and
Shape in the Wild. EMDB is a novel dataset that contains high-quality 3D SMPL
pose and shape parameters with global body and camera trajectories for
in-the-wild videos. We use body-worn, wireless electromagnetic (EM) sensors and
a hand-held iPhone to record a total of 58 minutes of motion data, distributed
over 81 indoor and outdoor sequences and 10 participants. Together with
accurate body poses and shapes, we also provide global camera poses and body
root trajectories. To construct EMDB, we propose a multi-stage optimization
procedure, which first fits SMPL to the 6-DoF EM measurements and then refines
the poses via image observations. To achieve high-quality results, we leverage
a neural implicit avatar model to reconstruct detailed human surface geometry
and appearance, which allows for improved alignment and smoothness via a dense
pixel-level objective. Our evaluations, conducted with a multi-view volumetric
capture system, indicate that EMDB has an expected accuracy of 2.3 cm
positional and 10.6 degrees angular error, surpassing the accuracy of previous
in-the-wild datasets. We evaluate existing state-of-the-art monocular RGB
methods for camera-relative and global pose estimation on EMDB. EMDB is
publicly available under https://ait.ethz.ch/emdbComment: Accepted to ICCV 202
Pathway to Future Symbiotic Creativity
This report presents a comprehensive view of our vision on the development
path of the human-machine symbiotic art creation. We propose a classification
of the creative system with a hierarchy of 5 classes, showing the pathway of
creativity evolving from a mimic-human artist (Turing Artists) to a Machine
artist in its own right. We begin with an overview of the limitations of the
Turing Artists then focus on the top two-level systems, Machine Artists,
emphasizing machine-human communication in art creation. In art creation, it is
necessary for machines to understand humans' mental states, including desires,
appreciation, and emotions, humans also need to understand machines' creative
capabilities and limitations. The rapid development of immersive environment
and further evolution into the new concept of metaverse enable symbiotic art
creation through unprecedented flexibility of bi-directional communication
between artists and art manifestation environments. By examining the latest
sensor and XR technologies, we illustrate the novel way for art data collection
to constitute the base of a new form of human-machine bidirectional
communication and understanding in art creation. Based on such communication
and understanding mechanisms, we propose a novel framework for building future
Machine artists, which comes with the philosophy that a human-compatible AI
system should be based on the "human-in-the-loop" principle rather than the
traditional "end-to-end" dogma. By proposing a new form of inverse
reinforcement learning model, we outline the platform design of machine
artists, demonstrate its functions and showcase some examples of technologies
we have developed. We also provide a systematic exposition of the ecosystem for
AI-based symbiotic art form and community with an economic model built on NFT
technology. Ethical issues for the development of machine artists are also
discussed
Monocular 3D Human Pose Markerless Systems for Gait Assessment
Gait analysis plays an important role in the fields of healthcare and sports sciences. Conventional gait analysis relies on costly equipment such as optical motion capture cameras and wearable sensors, some of which require trained assessors for data collection and processing. With the recent developments in computer vision and deep neural networks, using monocular RGB cameras for 3D human pose estimation has shown tremendous promise as a cost-effective and efficient solution for clinical gait analysis. In this paper, a markerless human pose technique is developed using motion captured by a consumer monocular camera (800 × 600 pixels and 30 FPS) for clinical gait analysis. The experimental results have shown that the proposed post-processing algorithm significantly improved the original human pose detection model (BlazePose)’s prediction performance compared to the gold-standard gait signals by 10.7% using the MoVi dataset. In addition, the predicted T2 score has an excellent correlation with ground truth (r = 0.99 and y = 0.94x + 0.01 regression line), which supports that our approach can be a potential alternative to the conventional marker-based solution to assist the clinical gait assessment
A multimodal dataset of real world mobility activities in Parkinson’s disease
Parkinson’s disease (PD) is a neurodegenerative disorder characterised by motor symptoms such as gait dysfunction and postural instability. Technological tools to continuously monitor outcomes could capture the hour-by-hour symptom fluctuations of PD. Development of such tools is hampered by the lack of labelled datasets from home settings. To this end, we propose REMAP (REal-world Mobility Activities in Parkinson’s disease), a human rater-labelled dataset collected in a home-like setting. It includes people with and without PD doing sit-to-stand transitions and turns in gait. These discrete activities are captured from periods of free-living (unobserved, unstructured) and during clinical assessments. The PD participants withheld their dopaminergic medications for a time (causing increased symptoms), so their activities are labelled as being “on” or “off” medications. Accelerometry from wrist-worn wearables and skeleton pose video data is included. We present an open dataset, where the data is coarsened to reduce re-identifiability, and a controlled dataset available on application which contains more refined data. A use-case for the data to estimate sit-to-stand speed and duration is illustrated
Accessibility of Health Data Representations for Older Adults: Challenges and Opportunities for Design
Health data of consumer off-the-shelf wearable devices is often conveyed to users through visual data representations and analyses. However, this is not always accessible to people with disabilities or older people due to low vision, cognitive impairments or literacy issues. Due to trade-offs between aesthetics predominance or information overload, real-time user feedback may not be conveyed easily from sensor devices through visual cues like graphs and texts. These difficulties may hinder critical data understanding. Additional auditory and tactile feedback can also provide immediate and accessible cues from these wearable devices, but it is necessary to understand existing data representation limitations initially. To avoid higher cognitive and visual overload, auditory and haptic cues can be designed to complement, replace or reinforce visual cues. In this paper, we outline the challenges in existing data representation and the necessary evidence to enhance the accessibility of health information from personal sensing devices used to monitor health parameters such as blood pressure, sleep, activity, heart rate and more. By creating innovative and inclusive user feedback, users will likely want to engage and interact with new devices and their own data
- …