3,857 research outputs found
mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics
Hyperspectral imaging acquires data in both the spatial and frequency domains
to offer abundant physical or biological information. However, conventional
hyperspectral imaging has intrinsic limitations of bulky instruments, slow data
acquisition rate, and spatiospectral tradeoff. Here we introduce hyperspectral
learning for snapshot hyperspectral imaging in which sampled hyperspectral data
in a small subarea are incorporated into a learning algorithm to recover the
hypercube. Hyperspectral learning exploits the idea that a photograph is more
than merely a picture and contains detailed spectral information. A small
sampling of hyperspectral data enables spectrally informed learning to recover
a hypercube from an RGB image. Hyperspectral learning is capable of recovering
full spectroscopic resolution in the hypercube, comparable to high spectral
resolutions of scientific spectrometers. Hyperspectral learning also enables
ultrafast dynamic imaging, leveraging ultraslow video recording in an
off-the-shelf smartphone, given that a video comprises a time series of
multiple RGB images. To demonstrate its versatility, an experimental model of
vascular development is used to extract hemodynamic parameters via statistical
and deep-learning approaches. Subsequently, the hemodynamics of peripheral
microcirculation is assessed at an ultrafast temporal resolution up to a
millisecond, using a conventional smartphone camera. This spectrally informed
learning method is analogous to compressed sensing; however, it further allows
for reliable hypercube recovery and key feature extractions with a transparent
learning algorithm. This learning-powered snapshot hyperspectral imaging method
yields high spectral and temporal resolutions and eliminates the spatiospectral
tradeoff, offering simple hardware requirements and potential applications of
various machine-learning techniques.Comment: This paper will appear in PNAS Nexu
rPPG-Toolbox: Deep Remote PPG Toolbox
Camera-based physiological measurement is a fast growing field of computer
vision. Remote photoplethysmography (rPPG) utilizes imaging devices (e.g.,
cameras) to measure the peripheral blood volume pulse (BVP) via
photoplethysmography, and enables cardiac measurement via webcams and
smartphones. However, the task is non-trivial with important pre-processing,
modeling, and post-processing steps required to obtain state-of-the-art
results. Replication of results and benchmarking of new models is critical for
scientific progress; however, as with many other applications of deep learning,
reliable codebases are not easy to find or use. We present a comprehensive
toolbox, rPPG-Toolbox, that contains unsupervised and supervised rPPG models
with support for public benchmark datasets, data augmentation, and systematic
evaluation: \url{https://github.com/ubicomplab/rPPG-Toolbox
- …