13 research outputs found
Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily Living
Domain shifts, such as appearance changes, are a key challenge in real-world
applications of activity recognition models, which range from assistive
robotics and smart homes to driver observation in intelligent vehicles. For
example, while simulations are an excellent way of economical data collection,
a Synthetic-to-Real domain shift leads to a > 60% drop in accuracy when
recognizing activities of Daily Living (ADLs). We tackle this challenge and
introduce an activity domain generation framework which creates novel ADL
appearances (novel domains) from different existing activity modalities (source
domains) inferred from video training data. Our framework computes human poses,
heatmaps of body joints, and optical flow maps and uses them alongside the
original RGB videos to learn the essence of source domains in order to generate
completely new ADL domains. The model is optimized by maximizing the distance
between the existing source appearances and the generated novel appearances
while ensuring that the semantics of an activity is preserved through an
additional classification loss. While source data multimodality is an important
concept in this design, our setup does not rely on multi-sensor setups, (i.e.,
all source modalities are inferred from a single video only.) The newly created
activity domains are then integrated in the training of the ADL classification
networks, resulting in models far less susceptible to changes in data
distributions. Extensive experiments on the Synthetic-to-Real benchmark
Sims4Action demonstrate the potential of the domain generation paradigm for
cross-domain ADL recognition, setting new state-of-the-art results. Our code is
publicly available at https://github.com/Zrrr1997/syn2real_DGComment: 8 pages, 7 figures, to be published in IROS 202
ModSelect: Automatic Modality Selection for Synthetic-to-Real Domain Generalization
Modality selection is an important step when designing multimodal systems, especially in the case of cross-domain activity recognition as certain modalities are more robust to domain shift than others. However, selecting only the modalities which have a positive contribution requires a systematic approach. We tackle this problem by proposing an unsupervised modality selection method (ModSelect), which does not require any ground-truth labels. We determine the correlation between the predictions of multiple unimodal classifiers and the domain discrepancy between their embeddings. Then, we systematically compute modality selection thresholds, which select only modalities with a high correlation and low domain discrepancy. We show in our experiments that our method ModSelect chooses only modalities with positive contributions and consistently improves the performance on a Synthetic-to-Real domain adaptation benchmark, narrowing the domain gap
ModSelect: Automatic Modality Selection for Synthetic-to-Real Domain Generalization
Modality selection is an important step when designing multimodal systems,
especially in the case of cross-domain activity recognition as certain
modalities are more robust to domain shift than others. However, selecting only
the modalities which have a positive contribution requires a systematic
approach. We tackle this problem by proposing an unsupervised modality
selection method (ModSelect), which does not require any ground-truth labels.
We determine the correlation between the predictions of multiple unimodal
classifiers and the domain discrepancy between their embeddings. Then, we
systematically compute modality selection thresholds, which select only
modalities with a high correlation and low domain discrepancy. We show in our
experiments that our method ModSelect chooses only modalities with positive
contributions and consistently improves the performance on a Synthetic-to-Real
domain adaptation benchmark, narrowing the domain gap.Comment: 14 pages, 6 figures, Accepted at ECCV 2022 OOD worksho
Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environments
Deep learning-based models are at the forefront of most driver observation
benchmarks due to their remarkable accuracies but are also associated with high
computational costs. This is challenging, as resources are often limited in
real-world driving scenarios. This paper introduces a lightweight framework for
resource-efficient driver activity recognition. The framework enhances 3D
MobileNet, a neural architecture optimized for speed in video classification,
by incorporating knowledge distillation and model quantization to balance model
accuracy and computational efficiency. Knowledge distillation helps maintain
accuracy while reducing the model size by leveraging soft labels from a larger
teacher model (I3D), instead of relying solely on original ground truth data.
Model quantization significantly lowers memory and computation demands by using
lower precision integers for model weights and activations. Extensive testing
on a public dataset for in-vehicle monitoring during autonomous driving
demonstrates that this new framework achieves a threefold reduction in model
size and a 1.4-fold improvement in inference time, compared to an already
optimized architecture. The code for this study is available at
https://github.com/calvintanama/qd-driver-activity-reco.Comment: Accepted at IROS 202
MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision
Prior to the deep learning era, shape was commonly used to describe the
objects. Nowadays, state-of-the-art (SOTA) algorithms in medical imaging are
predominantly diverging from computer vision, where voxel grids, meshes, point
clouds, and implicit surface models are used. This is seen from numerous
shape-related publications in premier vision conferences as well as the growing
popularity of ShapeNet (about 51,300 models) and Princeton ModelNet (127,915
models). For the medical domain, we present a large collection of anatomical
shapes (e.g., bones, organs, vessels) and 3D models of surgical instrument,
called MedShapeNet, created to facilitate the translation of data-driven vision
algorithms to medical applications and to adapt SOTA vision algorithms to
medical problems. As a unique feature, we directly model the majority of shapes
on the imaging data of real patients. As of today, MedShapeNet includes 23
dataset with more than 100,000 shapes that are paired with annotations (ground
truth). Our data is freely accessible via a web interface and a Python
application programming interface (API) and can be used for discriminative,
reconstructive, and variational benchmarks as well as various applications in
virtual, augmented, or mixed reality, and 3D printing. Exemplary, we present
use cases in the fields of classification of brain tumors, facial and skull
reconstructions, multi-class anatomy completion, education, and 3D printing. In
future, we will extend the data and improve the interfaces. The project pages
are: https://medshapenet.ikim.nrw/ and
https://github.com/Jianningli/medshapenet-feedbackComment: 16 page
AutoPET Challenge 2023: Sliding Window-based Optimization of U-Net
Tumor segmentation in medical imaging is crucial and relies on precise
delineation. Fluorodeoxyglucose Positron-Emission Tomography (FDG-PET) is
widely used in clinical practice to detect metabolically active tumors.
However, FDG-PET scans may misinterpret irregular glucose consumption in
healthy or benign tissues as cancer. Combining PET with Computed Tomography
(CT) can enhance tumor segmentation by integrating metabolic and anatomic
information. FDG-PET/CT scans are pivotal for cancer staging and reassessment,
utilizing radiolabeled fluorodeoxyglucose to highlight metabolically active
regions. Accurately distinguishing tumor-specific uptake from physiological
uptake in normal tissues is a challenging aspect of precise tumor segmentation.
The AutoPET challenge addresses this by providing a dataset of 1014 FDG-PET/CT
studies, encouraging advancements in accurate tumor segmentation and analysis
within the FDG-PET/CT domain. Code:
https://github.com/matt3o/AutoPET2-Submission/Comment: 9 pages, 1 figure, MICCAI 2023 - AutoPET Challenge Submission Version
2: Added all results on the preliminary test se
Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction
Drones have become a common tool, which is utilized in many tasks such as
aerial photography, surveillance, and delivery. However, operating a drone
requires more and more interaction with the user. A natural and safe method for
Human-Drone Interaction (HDI) is using gestures. In this paper, we introduce an
HDI framework building upon skeleton-based pose estimation. Our framework
provides the functionality to control the movement of the drone with simple arm
gestures and to follow the user while keeping a safe distance. We also propose
a monocular distance estimation method, which is entirely based on image
features and does not require any additional depth sensors. To perform
comprehensive experiments and quantitative analysis, we create a customized
testing dataset. The experiments indicate that our HDI framework can achieve an
average of 93.5\% accuracy in the recognition of 11 common gestures. The code
is available at: https://github.com/Zrrr1997/Pose2Dron
Assessment of the acute impact of normobaric hypoxia as a part of an intermittent hypoxic training on heart rate variability
AbstractAimTo assess the dynamics of ANS by means of heart rate variability (HRV) during and after acute exposure to normobaric hypoxia, representing a single session of an intermittent hypoxic training protocol.Material and methodsTwenty four healthy males aged 28.0±7.2 (mean±SD) breathed hypoxic air (FIO2=12.3±1.5%) for one hour delivered via hypoxicator (AltiPro 8850 Summit+, Altitude Tech, Canada). Pulse oximetry and HRV were measured before, during and after the hypoxic exposure.ResultsAt the end of the hypoxic session all of the tested subjects had higher low frequency (lnLF) (6.9±1.1ms2 vs. 7.5±1.1ms2; p=0.042), LF/HF (1.5±0.8 vs. 3.3±2.8; p=0.007) and standard deviation 2 of the Poincaré plot (SD2) (92.8±140.0ms vs. 120.2±54.2ms; p=0.005) as well as increase in the Total power (7.7±1.1ms2 vs. 8.1±1.2ms2; p=0.032) and the Standard deviation of normal-to-normal interbeat intervals (SDNN) (57.3±31.0ms vs. 72.3±41.1ms; p=0.024) but lower Sample entropy (SampEn) (1.6±0.2 vs. 1.4±0.2; p=0.010). Immediately after the hypoxic exposure LF/HF lowered (3.3±2.8 vs. 2.2±1.8; p=0.001) but lnHF significantly increased (6.6±1.4ms2 vs. 7.1±1.3ms2; p=0.020).ConclusionAcute normobaric hypoxia as a part of a single session of an intermittent hypoxic training protocol leads to changes in the activity of the ANS. The sympathetic tone prevails during hypoxic exposure and parasympathetic tone increases immediately after the hypoxic factor is withdrawn
Evaluation of Acute Exogenous Hypoxia Impact on the Fraction of Exhaled Nitric Oxide in Healthy Males
Introduction: Exogenous hypoxia increases ventilation and contracts the pulmonary vessels. Whether those factors change the values of nitric oxide in exhaled air has not yet been evaluated. Objective: To examine the effect of exogenous normobaric hypoxia on the values of the fraction of nitric oxide in exhaled breath (FeNO). Subjects аnd Methods: Twenty healthy non-smoker males at mean age of 25.4 (SD = 3.7) were tested. The basal FeNO values were compared with those at 7 min. and 15 min. after introducing into the hypoxic environment (hypoxic tent), imitating atmospheric air with oxygen concentration corresponding to 3200 m above sea level. Exhaled breath temperature was measured at baseline and at 10-12 min. of the hypoxic exposition. Heart rate and oxygen saturation were registered by pulse-oximetry. Results: All the subjects had FeNO values in the reference range. The mean baseline value was 14.0 ± 3.2 ppb, and in hypoxic conditions - 15.5 ± 3.8 ppb (7 min.) and 15.3 ± 3.6 ppb (15 min.), respectively, as the elevation is statistically significant (p = 0.011 and p = 0.008). The values of exhaled breath temperature were 33.79 ± 1.55°С and 33.87 ± 1.83°С (p = 0.70) at baseline and in hypoxic conditions, respectively. Baseline oxygen saturation in all subjects was higher than that, measured in hypoxia (96.93 ± 1.29% vs. 94.27 ± 2.53%; p < 0.001). Conclusions: Exogenous hypoxia leads to an increase of FeNO values, but does not affect the exhaled breath temperature
The impact of dietary intervention on myokines: a narrative review
This review elucidates the role of dietary factors in influencing myokine secretion, complementing the recognized benefits of physical activity on metabolic health. While exercise is known to positively modulate myokines, contributing to their health-promoting effects, an exploration into how diet affects these critical cytokines remains sparse. Through analysis of the literature, this work identifies dietary patterns that markedly affect myokine levels, underscoring the complex relationship between nutrition and myokine activity. The investigation reveals that specific dietary approaches can profoundly alter myokine secretion, suggesting an essential avenue for combined dietary and exercise interventions to optimize health outcomes. These insights highlight the need for further detailed study into diet-induced myokine modulation