Recognizing Human Activities Based on Wearable Inertial Measurements - Methods and Applications

On April 10 of 2015 Pekka Siirtola defended his PhD thesis, called “Recognizing Human Activities Based on Wearable Inertial Measurements - Methods and Applications” [1]. The thesis was supervised by Professor Juha Röning and pre-eximined by Associate Professors Ulf Johansson from University of Borås, Sweden, and Daniel Roggen from University of Sussex, United Kingdom. Pekka Siirtola successfully defended his thesis against his opponent Professor Barbara Hammer from University of Bielefeld, Germany. This publicly open defence was held in Auditorium TS101 at University of Oulu, Finland

Feature Relevance Analysis to Explain Concept Drift -- A Case Study in Human Activity Recognition

This article studies how to detect and explain concept drift. Human activity recognition is used as a case study together with a online batch learning situation where the quality of the labels used in the model updating process starts to decrease. Drift detection is based on identifying a set of features having the largest relevance difference between the drifting model and a model that is known to be accurate and monitoring how the relevance of these features changes over time. As a main result of this article, it is shown that feature relevance analysis cannot only be used to detect the concept drift but also to explain the reason for the drift when a limited number of typical reasons for the concept drift are predefined. To explain the reason for the concept drift, it is studied how these predefined reasons effect to feature relevance. In fact, it is shown that each of these has an unique effect to features relevance and these can be used to explain the reason for concept drift.Comment: Accepted to HASCA 2022 workshop in conjunction with UbiComp/ISWC202

Revisiting “Recognizing Human Activities User- Independently on Smartphones Based on Accelerometer Data” – What Has Happened Since 2012?

Our article “Recognizing human activities user-independently on smartphones based on accelerometer data” was published in the International Journal of Interactive Multimedia and Artificial Intelligence (IJIMAI) in 2012. In 2018, it was selected as the most outstanding article published in the 10 years of IJIMAI life. To celebrate the 10th anniversary of IJIMAI, in this article we will introduce what has happened in the field of human activity recognition and wearable sensor-based recognition since 2012, and especially, this article concentrates on introducing our work since 2012

Recognizing Human Activities Userindependently on Smartphones Based on Accelerometer Data

Real-time human activity recognition on a mobile phone is presented in this article. Unlike in most other studies, not only the data were collected using the accelerometers of a smartphone, but also models were implemented to the phone and the whole classification process (preprocessing, feature extraction and classification) was done on the device. The system is trained using phone orientation independent features to recognize five everyday activities: walking, running, cycling, driving a car and sitting/standing while the phone is in the pocket of the subject's trousers. Two classifiers were compared, knn (k nearest neighbors) and QDA (quadratic discriminant analysis). The models for real-time activity recognition were trained offline using a data set collected from eight subjects and these offline results were compared to real-time recognition rates, which are obtained by implementing models to mobile activity recognition application which currently supports two operating systems: Symbian^3 and Android. The results show that the presented method is light and, therefore, suitable for be used in real-time recognition. In addition, the recognition rates on the smartphones were encouraging, in fact, the recognition accuracies obtained are approximately as high as offline recognition rates. Also, the results show that the method presented is not an operating system dependent

Impacts of Data Synthesis: A Metric for Quantifiable Data Standards and Performances

Clinical data analysis could lead to breakthroughs. However, clinical data contain sensitive information about participants that could be utilized for unethical activities, such as blackmailing, identity theft, mass surveillance, or social engineering. Data anonymization is a standard step during data collection, before sharing, to overcome the risk of disclosure. However, conventional data anonymization techniques are not foolproof and also hinder the opportunity for personalized evaluations. Much research has been done for synthetic data generation using generative adversarial networks and many other machine learning methods; however, these methods are either not free to use or are limited in capacity. This study evaluates the performance of an emerging tool named synthpop, an R package producing synthetic data as an alternative approach for data anonymization. This paper establishes data standards derived from the original data set based on the utilities and quality of information and measures variations in the synthetic data set to evaluate the performance of the data synthesis process. The methods to assess the utility of the synthetic data set can be broadly divided into two approaches: general utility and specific utility. General utility assesses whether synthetic data have overall similarities in the statistical properties and multivariate relationships with the original data set. Simultaneously, the specific utility assesses the similarity of a fitted model’s performance on the synthetic data to its performance on the original data. The quality of information is assessed by comparing variations in entropy bits and mutual information to response variables within the original and synthetic data sets. The study reveals that synthetic data succeeded at all utility tests with a statistically non-significant difference and not only preserved the utilities but also preserved the complexity of the original data set according to the data standard established in this study. Therefore, synthpop fulfills all the necessities and unfolds a wide range of opportunities for the research community, including easy data sharing and information protection

Accelerometer vs. Electromyogram in Activity Recognition

In this study, information from wearable sensors is used to recognize human activities. Commonly the approaches are based on accelerometer data while in this study the potential of electromyogram (EMG) signals in activity recognition is studied. The electromyogram data is used in two different scenarios: 1) recognition of completely new activities in real life and 2) to recognize the individual activities. In this study, it was shown that in gym settings electromyogram signals clearly outperforms the accelerometer data in recognition of completely new sets of gym movements from streaming data even though the sensors would not be positioned directly to the muscles trained. Nevertheless, in recognition of individual activities the EMG itself does not provide enough information to recognize activities accurately

Recognizing human activities based on wearable inertial measurements:methods and applications

Abstract Inertial sensors are devices that measure movement, and therefore, when they are attached to a body, they can be used to measure human movements. In this thesis, data from these sensors are studied to recognize human activities user-independently. This is possible if the following two hypotheses are valid: firstly, as human movements are dissimilar between activities, also inertial sensor data between activities is so different that this data can be used to recognize activities. Secondly, while movements and inertial data are dissimilar between activities, they are so similar when different persons are performing the same activity that they can be recognized as the same activity. In this thesis, pattern recognition -based solutions are applied to inertial data to find these dissimilarities and similarities, and therefore, to build models to recognize activities user-independently. Activity recognition within this thesis is studied in two contexts: daily activity recognition using mobile phones, and activity recognition in industrial context. Both of these contexts have special requirements and these are considered in the presented solutions. Mobile phones are optimal devices to measure daily activity: they include a wide range of useful sensors to detect activities, and people carry them with them most of the time. On the other hand, the usage of mobile phones in active recognition includes several challenges; for instance, a person can carry a phone in any orientation, and there are hundreds of smartphone models, and each of them have specific hardware and software. Moreover, as battery life is always as issue with smartphones, techniques to lighten the classification process are proposed. Industrial context is different from daily activity context: when daily activities are recognized, occasional misclassifications may disturb the user, but they do not cause any other type of harm. This is not the case when activities are recognized in industrial context and the purpose is to recognize if the assembly line worker has performed tasks correctly. In this case, false classifications may be much more harmful. Solutions to these challenges are presented in this thesis. The solutions introduced in this thesis are applied to activity recognition data sets. However, as the basic idea of the activity recognition problem is the same as in many other pattern recognition procedures, most of the solutions can be applied to any pattern recognition problem, especially to ones where time series data is studied.Tiivistelmä Liikettä mittaavista antureista, kuten kiihtyvyysantureista, saatavaa tietoa voidaan käyttää ihmisten liikkeiden mittaamiseen kiinnittämällä ne johonkin kohtaan ihmisen kehoa. Väitöskirjassani tavoitteena on opettaa tähän tietoon perustuvia käyttäjäriippumattomia malleja, joiden avulla voidaan tunnistaa ihmisten toimia, kuten käveleminen ja juokseminen. Näiden mallien toimivuus perustuu seuraavaan kahteen oletukseen: (1) koska henkilöiden liikkeet eri toimissa ovat erilaisia, myös niistä mitattava anturitieto on erilaista, (2) useamman henkilön liikkeet samassa toimessa ovat niin samanlaisia, että liikkeistä mitatun anturitiedon perusteella nämä liikkeet voidaan päätellä kuvaavan samaa toimea. Tässä väitöskirjassa käyttäjäriippumaton ihmisten toimien tunnistus perustuu hahmontunnistusmenetelmiin ja tunnistusta on sovellettu kahteen eri asiayhteyteen: arkitoimien tunnistamiseen älypuhelimella sekä toimintojen tunnistamiseen teollisessa ympäristössä. Molemmilla sovellusalueilla on omat erityisvaatimuksensa ja -haasteensa. Älypuhelimien liikettä mittaavien antureihin perustuva tunnistus on haastavaa esimerkiksi siksi, että puhelimen asento ja paikka voivat vaihdella. Se voi olla esimerkiksi laukussa tai taskussa, lisäksi se voi olla missä tahansa asennossa. Myös puhelimen akun rajallinen kesto luo omat haasteensa. Tämän vuoksi tunnistus tulisi tehdä mahdollisimman kevyesti ja vähän virtaa kuluttavalla tavalla. Teollisessa ympäristössä haasteet ovat toisenlaisia. Kun tarkoituksena on tunnistaa esimerkiksi työvaiheiden oikea suoritusjärjestys kokoamislinjastolla, yksikin virheellinen tunnistus voi aiheuttaa suuren vahingon. Teollisessa ympäristössä tavoitteena onkin tunnistaa toimet mahdollisimman tarkasti välittämättä siitä kuinka paljon virtaa ja tehoa tunnistus vaatii. Väitöskirjassani kerrotaan kuinka nämä erityisvaatimukset ja -haasteet voidaan ottaa huomioon suunniteltaessa malleja ihmisten toimien tunnistamiseen. Väitöskirjassani esiteltyjä uusia menetelmiä on sovellettu ihmisten toimien tunnistamiseen. Samoja menetelmiä voidaan kuitenkin käyttää monissa muissa hahmontunnistukseen liittyvissä ongelmissa, erityisesti sellaisissa, joissa analysoitava tieto on aikasarjamuotoista