3,260 research outputs found
Preterm Infants' Pose Estimation with Spatio-Temporal Features
Objective: Preterm infants' limb monitoring in neonatal intensive care units (NICUs) is of primary importance for assessing infants' health status and motor/cognitive development. Herein, we propose a new approach to preterm infants' limb pose estimation that features spatio-temporal information to detect and track limb joints from depth videos with high reliability. Methods: Limb-pose estimation is performed using a deep-learning framework consisting of a detection and a regression convolutional neural network (CNN) for rough and precise joint localization, respectively. The CNNs are implemented to encode connectivity in the temporal direction through 3D convolution. Assessment of the proposed framework is performed through a comprehensive study with sixteen depth videos acquired in the actual clinical practice from sixteen preterm infants (the babyPose dataset). Results: When applied to pose estimation, the median root mean square distance, computed among all limbs, between the estimated and the ground-truth pose was 9.06 pixels, overcoming approaches based on spatial features only (11.27 pixels). Conclusion: Results showed that the spatio-temporal features had a significant influence on the pose-estimation performance, especially in challenging cases (e.g., homogeneous image intensity). Significance: This article significantly enhances the state of art in automatic assessment of preterm infants' health status by introducing the use of spatio-temporal features for limb detection and tracking, and by being the first study to use depth videos acquired in the actual clinical practice for limb-pose estimation. The babyPose dataset has been released as the first annotated dataset for infants' pose estimation
Towards human-level performance on automatic pose estimation of infant spontaneous movements
Assessment of spontaneous movements can predict the long-term developmental
disorders in high-risk infants. In order to develop algorithms for automated
prediction of later disorders, highly precise localization of segments and
joints by infant pose estimation is required. Four types of convolutional
neural networks were trained and evaluated on a novel infant pose dataset,
covering the large variation in 1 424 videos from a clinical international
community. The localization performance of the networks was evaluated as the
deviation between the estimated keypoint positions and human expert
annotations. The computational efficiency was also assessed to determine the
feasibility of the neural networks in clinical practice. The best performing
neural network had a similar localization error to the inter-rater spread of
human expert annotations, while still operating efficiently. Overall, the
results of our study show that pose estimation of infant spontaneous movements
has a great potential to support research initiatives on early detection of
developmental disorders in children with perinatal brain injuries by
quantifying infant movements from video recordings with human-level
performance.Comment: Published in Computerized Medical Imaging and Graphics (CMIG
Markerless Human Motion Analysis
Measuring and understanding human motion is crucial in several domains,
ranging from neuroscience, to rehabilitation and sports biomechanics. Quantitative
information about human motion is fundamental to study how our
Central Nervous System controls and organizes movements to functionally
evaluate motor performance and deficits. In the last decades, the research in
this field has made considerable progress. State-of-the-art technologies that
provide useful and accurate quantitative measures rely on marker-based systems.
Unfortunately, markers are intrusive and their number and location must
be determined a priori. Also, marker-based systems require expensive laboratory
settings with several infrared cameras. This could modify the naturalness
of a subject\u2019s movements and induce discomfort. Last, but not less important,
they are computationally expensive in time and space. Recent advances on
markerless pose estimation based on computer vision and deep neural networks
are opening the possibility of adopting efficient video-based methods
for extracting movement information from RGB video data. In this contest,
this thesis presents original contributions to the following objectives: (i) the
implementation of a video-based markerless pipeline to quantitatively characterize
human motion; (ii) the assessment of its accuracy if compared with
a gold standard marker-based system; (iii) the application of the pipeline to
different domains in order to verify its versatility, with a special focus on the
characterization of the motion of preterm infants and on gait analysis. With
the proposed approach we highlight that, starting only from RGB videos and
leveraging computer vision and machine learning techniques, it is possible to
extract reliable information characterizing human motion comparable to that
obtained with gold standard marker-based systems
- …