CaloriNet: From silhouettes to calorie estimation in private environments

We propose a novel deep fusion architecture, CaloriNet, for the online estimation of energy expenditure for free living monitoring in private environments, where RGB data is discarded and replaced by silhouettes. Our fused convolutional neural network architecture is trainable end-to-end, to estimate calorie expenditure, using temporal foreground silhouettes alongside accelerometer data. The network is trained and cross-validated on a publicly available dataset, SPHERE_RGBD + Inertial_calorie. Results show state-of-the-art minimum error on the estimation of energy expenditure (calories per minute), outperforming alternative, standard and single-modal techniques.Comment: 11 pages, 7 figure

A Comparative Study on the Suitability of Smartphones and IMU for Mobile, Unsupervised Energy Expenditure Calculi

The metabolic equivalent of task (MET) is currently the most used indicator for measuring the energy expenditure (EE) of a physical activity (PA) and has become an important measure for determining and supervising a person’s state of health. The use of new devices which are capable of measuring inertial movements by means of built-in accelerometers enable the PA to be measured objectively on the basis of the reckoning of “counts”. These devices are also known as inertial measurement units (IMUs) and each count is an aggregated value indicating the intensity of a movement and can be used in conjunction with other parameters to determine the MET rate of a particular physical activity and thus it’s associated EE. Various types of inertial devices currently exist that enable count calculus and physical activity to be monitored. The advent of mobile devices, such as smartphones, with empowered computation capabilities and integrated inertial sensors, has enabled EE to be measure in a distributed, ubiquitous and natural way, thereby overcoming the reluctance of users and practitioners associated with in-lab studies. From the point of view of the process analysis and infrastructure needed to manage data from inertial devices, there are also various differences in count computing: extra devices are required, out-of-device processing, etc. This paper presents a study to discover whether the estimation of energy expenditure is dependent on the accelerometer of the device used in measurements and to discover the suitability of each device for performing certain physical activities. In order to achieve this objective, we have conducted several experiments with different subjects on the basis of the performance of various daily activities with different smartphones and IMUs.This research work was partially supported by the project ‘Sistema Ergonómico Integral para la evaluación de la locomoción como predictor de la calidad de vida relacionada con la salud en Mayores (Ergoloc)’, funded by the Spanish Ministry of Economy and Competitiveness under the project DEP2012-40069; Facultad de Educación, Economía y Tecnología de Ceuta under the “Contrato-programa” of research for the period 2013–2015 and by the Programa de Fortalecimiento de I+D+i de la Universidad de Granada 2014-15. The Ministry of Education, Culture and Sports of Spain supported the work of Orantes-González, E. (ref. FPU13/00162). The authors would also like to acknowledge contribution from COST Action IC1303

Fall Prediction and Prevention Systems: Recent Trends, Challenges, and Future Research Directions.

Fall prediction is a multifaceted problem that involves complex interactions between physiological, behavioral, and environmental factors. Existing fall detection and prediction systems mainly focus on physiological factors such as gait, vision, and cognition, and do not address the multifactorial nature of falls. In addition, these systems lack efficient user interfaces and feedback for preventing future falls. Recent advances in internet of things (IoT) and mobile technologies offer ample opportunities for integrating contextual information about patient behavior and environment along with physiological health data for predicting falls. This article reviews the state-of-the-art in fall detection and prediction systems. It also describes the challenges, limitations, and future directions in the design and implementation of effective fall prediction and prevention systems

Mobile monitoring application to support sustainable behavioural change towards healthy lifestyle

We describe the development of body area networks (BANs) incorporating sensors and other devices to provide intelligent mobile services in healthcare and well-being. The first BAN applications were designed to simply transmit biosignals and display them remotely. Further developments include analysis and interpretation of biosignals in the light of context data. By including feedback loops, BAN telemonitoring was also augmented with teletreatment services. Recent developments include incorporation of clinical decision support by applying techniques from artificial intelligence. These developments represent a movement towards smart healthcare, making health BAN applications more intelligent by incorporating feedback, context awareness, personalization, and decision support.\ud The element of decision support was first introduced into the BAN health and well-being applications in the Food Valley Eating Advisor (FOVEA) project. Obesity and overweight represent a growing threat to health and well-being in modern society. Physical inactivity has been shown to contribute significantly to morbidity and mortality rates, and this is now a global trend bringing huge costs in terms of human suffering and reduction in life expectancy as well as uncontrolled growth in demand on healthcare services. Part of the solution is to foster healthier lifestyle. A major challenge however is that exercise and dietary programs may work for the individual in the short term, but adherence in the medium and long term is difficult to sustain, making weight management a continuing struggle for individuals and a growing problem for society, governments, and health services. Using ICT to support sustainable behavioral change in relation to healthy exercise and diet is the goal of the FOVEA monitoring and feedback application. We strive to design and develop intelligent BAN-based applications that support motivation and adherence in the long term. We present this healthy lifestyle application and report results of an evaluation conducted by surveying professionals in related disciplines

Multisensor Data Fusion for Human Activities Classification and Fall Detection

Significant research exists on the use of wearable sensors in the context of assisted living for activities recognition and fall detection, whereas radar sensors have been studied only recently in this domain. This paper approaches the performance limitation of using individual sensors, especially for classification of similar activities, by implementing information fusion of features extracted from experimental data collected by different sensors, namely a tri-axial accelerometer, a micro-Doppler radar, and a depth camera. Preliminary results confirm that combining information from heterogeneous sensors improves the overall performance of the system. The classification accuracy attained by means of this fusion approach improves by 11.2% compared to radar-only use, and by 16.9% compared to the accelerometer. Furthermore, adding features extracted from a RGB-D Kinect sensor, the overall classification accuracy increases up to 91.3%

Real-time food intake classification and energy expenditure estimation on a mobile device

© 2015 IEEE.Assessment of food intake has a wide range of applications in public health and life-style related chronic disease management. In this paper, we propose a real-time food recognition platform combined with daily activity and energy expenditure estimation. In the proposed method, food recognition is based on hierarchical classification using multiple visual cues, supported by efficient software implementation suitable for realtime mobile device execution. A Fischer Vector representation together with a set of linear classifiers are used to categorize food intake. Daily energy expenditure estimation is achieved by using the built-in inertial motion sensors of the mobile device. The performance of the vision-based food recognition algorithm is compared to the current state-of-the-art, showing improved accuracy and high computational efficiency suitable for realtime feedback. Detailed user studies have also been performed to demonstrate the practical value of the software environment

Classification of sporting activities using smartphone accelerometers

In this paper we present a framework that allows for the automatic identification of sporting activities using commonly available smartphones. We extract discriminative informational features from smartphone accelerometers using the Discrete Wavelet Transform (DWT). Despite the poor quality of their accelerometers, smartphones were used as capture devices due to their prevalence in today’s society. Successful classification on this basis potentially makes the technology accessible to both elite and non-elite athletes. Extracted features are used to train different categories of classifiers. No one classifier family has a reportable direct advantage in activity classification problems to date; thus we examine classifiers from each of the most widely used classifier families. We investigate three classification approaches; a commonly used SVM-based approach, an optimized classification model and a fusion of classifiers. We also investigate the effect of changing several of the DWT input parameters, including mother wavelets, window lengths and DWT decomposition levels. During the course of this work we created a challenging sports activity analysis dataset, comprised of soccer and field-hockey activities. The average maximum F-measure accuracy of 87% was achieved using a fusion of classifiers, which was 6% better than a single classifier model and 23% better than a standard SVM approach

Context-awareness for mobile sensing: a survey and future directions

The evolution of smartphones together with increasing computational power have empowered developers to create innovative context-aware applications for recognizing user related social and cognitive activities in any situation and at any location. The existence and awareness of the context provides the capability of being conscious of physical environments or situations around mobile device users. This allows network services to respond proactively and intelligently based on such awareness. The key idea behind context-aware applications is to encourage users to collect, analyze and share local sensory knowledge in the purpose for a large scale community use by creating a smart network. The desired network is capable of making autonomous logical decisions to actuate environmental objects, and also assist individuals. However, many open challenges remain, which are mostly arisen due to the middleware services provided in mobile devices have limited resources in terms of power, memory and bandwidth. Thus, it becomes critically important to study how the drawbacks can be elaborated and resolved, and at the same time better understand the opportunities for the research community to contribute to the context-awareness. To this end, this paper surveys the literature over the period of 1991-2014 from the emerging concepts to applications of context-awareness in mobile platforms by providing up-to-date research and future research directions. Moreover, it points out the challenges faced in this regard and enlighten them by proposing possible solutions