2,716 research outputs found
CalibrationPhys: Self-supervised Video-based Heart and Respiratory Rate Measurements by Calibrating Between Multiple Cameras
Video-based heart and respiratory rate measurements using facial videos are
more useful and user-friendly than traditional contact-based sensors. However,
most of the current deep learning approaches require ground-truth pulse and
respiratory waves for model training, which are expensive to collect. In this
paper, we propose CalibrationPhys, a self-supervised video-based heart and
respiratory rate measurement method that calibrates between multiple cameras.
CalibrationPhys trains deep learning models without supervised labels by using
facial videos captured simultaneously by multiple cameras. Contrastive learning
is performed so that the pulse and respiratory waves predicted from the
synchronized videos using multiple cameras are positive and those from
different videos are negative. CalibrationPhys also improves the robustness of
the models by means of a data augmentation technique and successfully leverages
a pre-trained model for a particular camera. Experimental results utilizing two
datasets demonstrate that CalibrationPhys outperforms state-of-the-art heart
and respiratory rate measurement methods. Since we optimize camera-specific
models using only videos from multiple cameras, our approach makes it easy to
use arbitrary cameras for heart and respiratory rate measurements.Comment: Accepted to IEEE Journal of Biomedical and Health Informatics (J-BHI
Temporal Interpolation via Motion Field Prediction
Navigated 2D multi-slice dynamic Magnetic Resonance (MR) imaging enables high
contrast 4D MR imaging during free breathing and provides in-vivo observations
for treatment planning and guidance. Navigator slices are vital for
retrospective stacking of 2D data slices in this method. However, they also
prolong the acquisition sessions. Temporal interpolation of navigator slices an
be used to reduce the number of navigator acquisitions without degrading
specificity in stacking. In this work, we propose a convolutional neural
network (CNN) based method for temporal interpolation via motion field
prediction. The proposed formulation incorporates the prior knowledge that a
motion field underlies changes in the image intensities over time. Previous
approaches that interpolate directly in the intensity space are prone to
produce blurry images or even remove structures in the images. Our method
avoids such problems and faithfully preserves the information in the image.
Further, an important advantage of our formulation is that it provides an
unsupervised estimation of bi-directional motion fields. We show that these
motion fields can be used to halve the number of registrations required during
4D reconstruction, thus substantially reducing the reconstruction time.Comment: Submitted to 1st Conference on Medical Imaging with Deep Learning
(MIDL 2018), Amsterdam, The Netherland
Automatic Infant Respiration Estimation from Video: A Deep Flow-based Algorithm and a Novel Public Benchmark
Respiration is a critical vital sign for infants, and continuous respiratory
monitoring is particularly important for newborns. However, neonates are
sensitive and contact-based sensors present challenges in comfort, hygiene, and
skin health, especially for preterm babies. As a step toward fully automatic,
continuous, and contactless respiratory monitoring, we develop a deep-learning
method for estimating respiratory rate and waveform from plain video footage in
natural settings. Our automated infant respiration flow-based network
(AIRFlowNet) combines video-extracted optical flow input and spatiotemporal
convolutional processing tuned to the infant domain. We support our model with
the first public annotated infant respiration dataset with 125 videos
(AIR-125), drawn from eight infant subjects, set varied pose, lighting, and
camera conditions. We include manual respiration annotations and optimize
AIRFlowNet training on them using a novel spectral bandpass loss function. When
trained and tested on the AIR-125 infant data, our method significantly
outperforms other state-of-the-art methods in respiratory rate estimation,
achieving a mean absolute error of 2.9 breaths per minute, compared to
4.7--6.2 for other public models designed for adult subjects and more
uniform environments
The Role of Edge Robotics As-a-Service in Monitoring COVID-19 Infection
Deep learning technology has been widely used in edge computing. However,
pandemics like covid-19 require deep learning capabilities at mobile devices
(detect respiratory rate using mobile robotics or conduct CT scan using a
mobile scanner), which are severely constrained by the limited storage and
computation resources at the device level. To solve this problem, we propose a
three-tier architecture, including robot layers, edge layers, and cloud layers.
We adopt this architecture to design a non-contact respiratory monitoring
system to break down respiratory rate calculation tasks. Experimental results
of respiratory rate monitoring show that the proposed approach in this paper
significantly outperforms other approaches. It is supported by computation time
costs with 2.26 ms per frame, 27.48 ms per frame, 0.78 seconds for convolution
operation, similarity calculation, processing one-minute length respiratory
signals, respectively. And the computation time costs of our three-tier
architecture are less than that of edge+cloud architecture and cloud
architecture. Moreover, we use our three-tire architecture for CT image
diagnosis task decomposition. The evaluation of a CT image dataset of COVID-19
proves that our three-tire architecture is useful for resolving tasks on deep
learning networks by edge equipment. There are broad application scenarios in
smart hospitals in the future
Preterm Infants' Pose Estimation with Spatio-Temporal Features
Objective: Preterm infants' limb monitoring in neonatal intensive care units (NICUs) is of primary importance for assessing infants' health status and motor/cognitive development. Herein, we propose a new approach to preterm infants' limb pose estimation that features spatio-temporal information to detect and track limb joints from depth videos with high reliability. Methods: Limb-pose estimation is performed using a deep-learning framework consisting of a detection and a regression convolutional neural network (CNN) for rough and precise joint localization, respectively. The CNNs are implemented to encode connectivity in the temporal direction through 3D convolution. Assessment of the proposed framework is performed through a comprehensive study with sixteen depth videos acquired in the actual clinical practice from sixteen preterm infants (the babyPose dataset). Results: When applied to pose estimation, the median root mean square distance, computed among all limbs, between the estimated and the ground-truth pose was 9.06 pixels, overcoming approaches based on spatial features only (11.27 pixels). Conclusion: Results showed that the spatio-temporal features had a significant influence on the pose-estimation performance, especially in challenging cases (e.g., homogeneous image intensity). Significance: This article significantly enhances the state of art in automatic assessment of preterm infants' health status by introducing the use of spatio-temporal features for limb detection and tracking, and by being the first study to use depth videos acquired in the actual clinical practice for limb-pose estimation. The babyPose dataset has been released as the first annotated dataset for infants' pose estimation
Enhancing Compressed Sensing 4D Photoacoustic Tomography by Simultaneous Motion Estimation
A crucial limitation of current high-resolution 3D photoacoustic tomography
(PAT) devices that employ sequential scanning is their long acquisition time.
In previous work, we demonstrated how to use compressed sensing techniques to
improve upon this: images with good spatial resolution and contrast can be
obtained from suitably sub-sampled PAT data acquired by novel acoustic scanning
systems if sparsity-constrained image reconstruction techniques such as total
variation regularization are used. Now, we show how a further increase of image
quality can be achieved for imaging dynamic processes in living tissue (4D
PAT). The key idea is to exploit the additional temporal redundancy of the data
by coupling the previously used spatial image reconstruction models with
sparsity-constrained motion estimation models. While simulated data from a
two-dimensional numerical phantom will be used to illustrate the main
properties of this recently developed
joint-image-reconstruction-and-motion-estimation framework, measured data from
a dynamic experimental phantom will also be used to demonstrate their potential
for challenging, large-scale, real-world, three-dimensional scenarios. The
latter only becomes feasible if a carefully designed combination of tailored
optimization schemes is employed, which we describe and examine in more detail
- …