Cross-Modal Health State Estimation

Individuals create and consume more diverse data about themselves today than any time in history. Sources of this data include wearable devices, images, social media, geospatial information and more. A tremendous opportunity rests within cross-modal data analysis that leverages existing domain knowledge methods to understand and guide human health. Especially in chronic diseases, current medical practice uses a combination of sparse hospital based biological metrics (blood tests, expensive imaging, etc.) to understand the evolving health status of an individual. Future health systems must integrate data created at the individual level to better understand health status perpetually, especially in a cybernetic framework. In this work we fuse multiple user created and open source data streams along with established biomedical domain knowledge to give two types of quantitative state estimates of cardiovascular health. First, we use wearable devices to calculate cardiorespiratory fitness (CRF), a known quantitative leading predictor of heart disease which is not routinely collected in clinical settings. Second, we estimate inherent genetic traits, living environmental risks, circadian rhythm, and biological metrics from a diverse dataset. Our experimental results on 24 subjects demonstrate how multi-modal data can provide personalized health insight. Understanding the dynamic nature of health status will pave the way for better health based recommendation engines, better clinical decision making and positive lifestyle changes.Comment: Accepted to ACM Multimedia 2018 Conference - Brave New Ideas, Seoul, Korea, ACM ISBN 978-1-4503-5665-7/18/1

Real-time human ambulation, activity, and physiological monitoring:taxonomy of issues, techniques, applications, challenges and limitations

Automated methods of real-time, unobtrusive, human ambulation, activity, and wellness monitoring and data analysis using various algorithmic techniques have been subjects of intense research. The general aim is to devise effective means of addressing the demands of assisted living, rehabilitation, and clinical observation and assessment through sensor-based monitoring. The research studies have resulted in a large amount of literature. This paper presents a holistic articulation of the research studies and offers comprehensive insights along four main axes: distribution of existing studies; monitoring device framework and sensor types; data collection, processing and analysis; and applications, limitations and challenges. The aim is to present a systematic and most complete study of literature in the area in order to identify research gaps and prioritize future research directions

A feasibility study of wearable activity monitors for pre-adolescent school-age children.

IntroductionUnderstanding physical activity is key in the fight against childhood obesity. The objective of this study was to examine the feasibility of using certain wearable devices to measure physical activity among children.MethodsA qualitative study was conducted with 25 children aged 7 to 10 years to assess acceptability and compliance of wearable activity devices in this age group. During March through August 2012, children participated in a 4-week study of 3 accelerometer models and a heart rate monitor. Children were asked to use a different device each week for 7 consecutive days. Children and their parents completed structured interviews after using each device; they also completed a final exit interview.ResultsThe wrist-worn Polar Active was the device most preferred by children and was associated with the highest level of compliance. Devices that are comfortable to wear, fit properly, have engaging features, and are waterproof increase feasibility and are associated with higher levels of compliance.ConclusionThe wrist-worn device was the most feasible for measuring physical activity among children aged 7 to 10 years. These findings will inform researchers in selecting tools for measuring children's physical activity

CaloriNet: From silhouettes to calorie estimation in private environments

We propose a novel deep fusion architecture, CaloriNet, for the online estimation of energy expenditure for free living monitoring in private environments, where RGB data is discarded and replaced by silhouettes. Our fused convolutional neural network architecture is trainable end-to-end, to estimate calorie expenditure, using temporal foreground silhouettes alongside accelerometer data. The network is trained and cross-validated on a publicly available dataset, SPHERE_RGBD + Inertial_calorie. Results show state-of-the-art minimum error on the estimation of energy expenditure (calories per minute), outperforming alternative, standard and single-modal techniques.Comment: 11 pages, 7 figure

Energy expenditure estimation using visual and inertial sensors

© The Institution of Engineering and Technology 2017. Deriving a person's energy expenditure accurately forms the foundation for tracking physical activity levels across many health and lifestyle monitoring tasks. In this study, the authors present a method for estimating calorific expenditure from combined visual and accelerometer sensors by way of an RGB-Depth camera and a wearable inertial sensor. The proposed individual-independent framework fuses information from both modalities which leads to improved estimates beyond the accuracy of single modality and manual metabolic equivalents of task (MET) lookup table based methods. For evaluation, the authors introduce a new dataset called SPHERE_RGBD + Inertial_calorie, for which visual and inertial data are simultaneously obtained with indirect calorimetry ground truth measurements based on gas exchange. Experiments show that the fusion of visual and inertial data reduces the estimation error by 8 and 18% compared with the use of visual only and inertial sensor only, respectively, and by 33% compared with a MET-based approach. The authors conclude from their results that the proposed approach is suitable for home monitoring in a controlled environment

Cardiorespiratory fitness estimation using wearable sensors: laboratory and free-living analysis of context-specific submaximal heart rates

In this work, we propose to use pattern recognition methods to determine submaximal heart rate (HR) during specific contexts, such as walking at a certain speed, using wearable sensors in free-living, and use context-specific HR to estimate cardiorespiratory fitness (CRF). CRF of 51 participants was assessed by a maximal exertion test (VO2max). Participants wore a combined accelerometer and HR monitor during a laboratory based simulation of activities of daily living and for two weeks in free-living. Anthropometrics, HR while lying down and walking at predefined speeds in laboratory settings were used to estimate CRF. Explained variance (R2) was 0.64 for anthropometrics, and increased up to 0.74 for context-specific HR (0.73 to 0.78 when including fat-free mass). Then, we developed activity recognition and walking speed estimation algorithms to determine the same contexts (i.e. lying down and walking) in free-living. Context-specific HR in free-living was highly correlated with laboratory measurements (Pearson's r = 0.71-0.75). R2 for CRF estimation was 0.65 when anthropometrics were used as predictors, and increased up to 0.77 when including free-living context-specific HR (i.e. HR while walking at 5.5 km/h). R2 varied between 0.73 and 0.80 when including fat-free mass among the predictors. RMSE was reduced from 354.7 ml/min to 281.0 ml/min by the inclusion of context-specific HR parameters (21% error reduction). We conclude that pattern recognition techniques can be used to contextualize HR in free-living and estimated CRF with accuracy comparable to what can be obtained with laboratory measurements of HR response to walking

Design and Application of Wireless Body Sensors

Hörmann T. Design and Application of Wireless Body Sensors. Bielefeld: Universität Bielefeld; 2019