The Usage of Statistical Learning Methods on Wearable Devices and a Case Study: Activity Recognition on Smartwatches

The aim of this study is to explore the usage of statistical learning methods on wearable devices and realize an experimental study for recognition of human activities by using smartwatch sensor data. To achieve this objective, mobile applications that run on smartwatch and smartphone were developed to gain training data and detect human activity momentarily; 500 pattern data were obtained with 4‐second intervals for each activity (walking, typing, stationary, running, standing, writing on board, brushing teeth, cleaning and writing). Created dataset was tested with five different statistical learning methods (Naive Bayes, k nearest neighbour (kNN), logistic regression, Bayesian network and multilayer perceptron) and their performances were compared

Monitoring student activities with smartwatches: On the academic performance enhancement

Motivated by the importance of studying the relationship between habits of students and their academic performance, daily activities of undergraduate participants have been tracked with smartwatches and smartphones. Smartwatches collect data together with an Android application that interacts with the users who provide the labeling of their own activities. The tracked activities include eating, running, sleeping, classroom-session, exam, job, homework, transportation, watching TV-Series, and reading. The collected data were stored in a server for activity recognition with supervised machine learning algorithms. The methodology for the concept proof includes the extraction of features with the discrete wavelet transform from gyroscope and accelerometer signals to improve the classification accuracy. The results of activity recognition with Random Forest were satisfactory (86.9%) and support the relationship between smartwatch sensor signals and daily-living activities of students which opens the possibility for developing future experiments with automatic activity-labeling, and so forth to facilitate activity pattern recognition to propose a recommendationsystem to enhance the academic performance of each student

Thought on Food: A Systematic Review of Current Approaches and Challenges for Food Intake Detection

Nowadays, individuals have very stressful lifestyles, affecting their nutritional habits. In the early stages of life, teenagers begin to exhibit bad habits and inadequate nutrition. Likewise, other people with dementia, Alzheimer’s disease, or other conditions may not take food or medicine regularly. Therefore, the ability to monitor could be beneficial for them and for the doctors that can analyze the patterns of eating habits and their correlation with overall health. Many sensors help accurately detect food intake episodes, including electrogastrography, cameras, microphones, and inertial sensors. Accurate detection may provide better control to enable healthy nutrition habits. This paper presents a systematic review of the use of technology for food intake detection, focusing on the different sensors and methodologies used. The search was performed with a Natural Language Processing (NLP) framework that helps screen irrelevant studies while following the PRISMA methodology. It automatically searched and filtered the research studies in different databases, including PubMed, Springer, ACM, IEEE Xplore, MDPI, and Elsevier. Then, the manual analysis selected 30 papers based on the results of the framework for further analysis, which support the interest in using sensors for food intake detection and nutrition assessment. The mainly used sensors are cameras, inertial, and acoustic sensors that handle the recognition of food intake episodes with artificial intelligence techniques. This research identifies the most used sensors and data processing methodologies to detect food intake.info:eu-repo/semantics/publishedVersio

Emotional self-regulation of individuals with autism spectrum disorders: smartwatches for monitoring and interaction

In this paper, we analyze the needs of individuals with Autism Spectrum Disorders (ASD) to have a pervasive, feasible and non-stigmatizing form of assistance in their emotional self-regulation, in order to ease certain behavioral issues that undermine their mental health throughout their life. We argue the potential of recent widespread wearables, and more specifically smartwatches, to achieve this goal. Then, a smartwatch system that implements a wide range of self-regulation strategies and infers outburst patterns from physiological signals and movement is presented, along with an authoring tool for smartphones that is to be used by caregivers or family members to create and edit these strategies, in an adaptive way. We conducted an intensive experiment with two individuals with ASD who showed varied, representative behavioral responses to their emotional dysregulation. Both users were able to employ effective, customized emotional self-regulation strategies by means of the system, recovering from the majority of mild stress episodes and temper tantrums experienced in the nine days of experiment in their classroomThis work has been partially funded by the projects “e-Training y e-Coaching para la integración socio—laboral” (TIN2013-44586-R) and “eMadrid-CM: Investigación y Desarrollo de Tecnologías Educativas en la Comunidad de Madrid” (S2013/ICE-2715). It has been also funded by Fundación Orange during the early stages of the project “Tic-Tac-TEA: Sistema de asistencia para la autorregulación emocional en momentos de crisis para personas con TEA mediante smartwatches

Group activity recognition and analysis: concept, model and service architecture

This thesis presents a new method for representing and recognizing human group activities using sensors, showing how real-world group activity scenarios can be automatically detected and understood by combining sensor data from smartphones, smartwatches, and Internet-of-Things devices. This research also introduces a generic service-oriented architecture for cloud-based group activity recognition

Computational Sleep Behaviour Analysis and Application

Sleep affects a person’s health and is, therefore, assessed if health problems arise. Sleep behaviour is monitored for abnormalities in order to determine if any treatments, such as medication or behavioural changes (modifications to sleep habits), are necessary. Assessments are typically done using two methods: polysomnography over short periods and four-week retrospective questionnaires. These standard methods, however, cannot measure current sleep status continuously and unsupervised over long periods of time in the same way home-based sleep behaviour assessment can. In this work, we investigate the ability of sleep behaviour assessment using IoT devices in a natural home environment, which potential has not been investigated fully, to enable early abnormality detection and facilitate self-management. We developed a framework that incorporates different facets and perspectives to introduce focus and support in sleep behaviour assessment. The framework considers users’ needs, various available technologies, and factors that influence sleep behaviours. Sleep analysis approaches are incorporated to increase the reliability of the system. This assessment is strengthened by utilising sleep stage detection and sleep position recognition. This includes, first, the extraction and integration of influence factors of sleep stage recognition methods to create a fine-grained personalised approach and, second, the detection of common but more complex sleep positions, including leg positions. The relations between medical conditions and sleep are assessed through interviews with doctors and users on various topics, including treatment satisfaction and technology acceptance. The findings from these interviews led to the investigation of sleep behaviour as a diagnostic indicator. Changes in sleep behaviour are assessed alongside medical knowledge using data mining techniques to extract information about disease development; the following diseases were of interest: sleep apnoea, hypertension, diabetes, and chronic kidney disease. The proposed framework is designed in a way that allows it to be integrated into existing smart home environments. We believe that our framework provides promising building blocks for reliable sleep behaviour assessment by incorporating newly developed sleep analysis approaches. These approaches include a modular layered sleep behaviour assessment framework, a sleep regularity algorithm, a user-dependent visualisation concept, a higher-granularity sleep position analysis approach, a fine-grained sleep stage detection approach, a personalised sleep parameter extraction process, in-depth understanding on sleep and chronic disease relations, and a sleep-wake behaviour-based chronic disease detection method.This work has been supported by the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 676157

Practical and Rich User Digitization

A long-standing vision in computer science has been to evolve computing devices into proactive assistants that enhance our productivity, health and wellness, and many other facets of our lives. User digitization is crucial in achieving this vision as it allows computers to intimately understand their users, capturing activity, pose, routine, and behavior. Today's consumer devices - like smartphones and smartwatches provide a glimpse of this potential, offering coarse digital representations of users with metrics such as step count, heart rate, and a handful of human activities like running and biking. Even these very low-dimensional representations are already bringing value to millions of people's lives, but there is significant potential for improvement. On the other end, professional, high-fidelity comprehensive user digitization systems exist. For example, motion capture suits and multi-camera rigs that digitize our full body and appearance, and scanning machines such as MRI capture our detailed anatomy. However, these carry significant user practicality burdens, such as financial, privacy, ergonomic, aesthetic, and instrumentation considerations, that preclude consumer use. In general, the higher the fidelity of capture, the lower the user's practicality. Most conventional approaches strike a balance between user practicality and digitization fidelity. My research aims to break this trend, developing sensing systems that increase user digitization fidelity to create new and powerful computing experiences while retaining or even improving user practicality and accessibility, allowing such technologies to have a societal impact. Armed with such knowledge, our future devices could offer longitudinal health tracking, more productive work environments, full body avatars in extended reality, and embodied telepresence experiences, to name just a few domains.Comment: PhD thesi

Sampling Strategies for Tackling Imbalanced Data in Human Activity Recognition

Human activity recognition (HAR) using wearable sensors is a topic that is being actively researched in machine learning. Smart, sensor-embedded devices, such as smartphones, fitness trackers, or smart watches that collect detailed data on movement, are widely available now. HAR may be applied in areas such as healthcare, physiotherapy, and fitness to assist users of these smart devices in their daily lives. However, one of the main challenges facing HAR, particularly when it is used in supervised learning, is how balanced data may be obtained for algorithm optimisation and testing. Because users engage in some activities more than others, e.g. walking more than running, HAR datasets are typically imbalanced. The lack of dataset representation from minority classes, therefore, hinders the ability of HAR classifiers to sufficiently capture new instances of those activities. Inspired by the concept of data fusion, this thesis will introduce three new hybrid sampling methods. Thus, the diversity of the synthesised samples will be enhanced by combining output from separate sampling methods into three hybrid approaches. The advantage of the hybrid method is that it provides diverse synthetic data that can increase the size of the training data from different sampling approaches. This leads to improvements in the generalisation of a learning activity recognition model. The first strategy, known as the (DBM), combines synthetic minority oversampling techniques (SMOTE) with Random_SMOTE, both of which are built around the k-nearest neighbours algorithm. The second technique, called the noise detection-based method (NDBM), combines Tomek links (SMOTE_Tomeklinks) and the modified synthetic minority oversampling technique (MSMOTE). The third approach, titled the cluster-based method (CBM), combines cluster-based synthetic oversampling (CBSO) and the proximity weighted synthetic oversampling technique (ProWSyn). The performance of the proposed hybrid methods is compared with existing methods using accelerometer data from three commonly used benchmark datasets. The results show that the DBM, NDBM and CBM can significantly reduce the impact of class imbalance and enhance F1 scores of the multilayer perceptron (MLP) by as much as 9 % to 20 % compared with their constituent sampling methods. Also, the Friedman statistical significance test was conducted to compare the effect of the different sampling methods. The test results confirm that the CBM is more effective than the other sampling approaches. This thesis also introduces a method based on the Wasserstein generative adversarial network (WGAN) for generating different types of data on human activity. The WGAN is more stable to train than a generative adversarial network (GAN) and this is due to the use of a stable metric, namely Wasserstein distance, to compare the similarity between the real data distribution with the generated data distribution. WGAN is a deep learning approach, and in contrast to the six existing sampling methods referred to previously, it can operate on raw sensor data as convolutional and recurrent layers can act as feature extractors. WGAN is used to generate raw sensor data to overcome the limitations of the traditional machine learning-based sampling methods that can only operate on extracted features. The synthetic data that is produced by WGAN is then used to oversample the imbalanced training data. This thesis demonstrates that this approach significantly enhances the learning ability of the convolutional neural network(CNN) by as much as 5 % to 6 % from imbalanced human activity datasets. This thesis concludes that the proposed sampling methods based on traditional machine learning are efficient when human activity training data is imbalanced and small. These methods are less complex to implement, require less human activity training data to produce synthetic data and fewer computational resources than the WGAN approach. The proposed WGAN method is effective at producing raw sensor data when a large quantity of human activity training data is available. Additionally, it is time-consuming to optimise the hyperparameters related to the WGAN architecture, which significantly impacts the performance of the method