Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone

Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of such datasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigate the robustness of common supervised classification approaches under instances of noisy data. We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85% average precision rate. Results obtained also show how a supervised approach trained using automatically generated labels achieved an 84% f-score (using Neural Networks and Random Forests); however, results also demonstrated how the presence of label noise could lower the f-score up to 64–74% depending on the classification approach (Nearest Centroid and Multi-Class Support Vector Machine)

SHPIA 2.0: An Easily Scalable, Low-Cost, Multi-purpose Smart Home Platform for Intelligent Applications

Sensors, electronic devices, and smart systems have invaded the market and our daily lives. As a result, their utility in smart home contexts to improve the quality of life, especially for the elderly and people with special needs, is getting stronger and stronger. Therefore, many systems based on smart applications and intelligent devices have been developed, for example, to monitor people’s environmental contexts, help in daily-life activities, and analyze their health status. However, most existing solutions have drawbacks related to accessibility and usability. They tend to be expensive and lack generality and interoperability. These solutions are not easily scalable and are typically designed for specific constrained scenarios. This paper tackles such drawbacks by presenting SHPIA 2.0, an easily scalable, low-cost, multi-purpose smart home platform for intelligent applications. It leverages low-cost Bluetooth Low Energy (BLE) devices featuring both BLE connected and BLE broadcast modes, to transform common objects of daily life into smart objects. Moreover, SHPIA 2.0 allows the col- lection and automatic labeling of different data types to provide indoor monitoring and assistance. Specifically, SHPIA 2.0 is designed to be adaptable to different home-based application scenarios, including human activity recognition, coaching systems, and occupancy detection and counting. The SHPIA platform is open source and freely available to the scientific community, fostering collaboration and innovation

Detecting Eating Episodes with an Ear-mounted Sensor

In this paper, we propose Auracle, a wearable earpiece that can automatically recognize eating behavior. More specifically, in free-living conditions, we can recognize when and for how long a person is eating. Using an off-the-shelf contact microphone placed behind the ear, Auracle captures the sound of a person chewing as it passes through the bone and tissue of the head. This audio data is then processed by a custom analog/digital circuit board. To ensure reliable (yet comfortable) contact between microphone and skin, all hardware components are incorporated into a 3D-printed behind-the-head framework. We collected field data with 14 participants for 32 hours in free-living conditions and additional eating data with 10 participants for 2 hours in a laboratory setting. We achieved accuracy exceeding 92.8% and F1 score exceeding 77.5% for eating detection. Moreover, Auracle successfully detected 20-24 eating episodes (depending on the metrics) out of 26 in free-living conditions. We demonstrate that our custom device could sense, process, and classify audio data in real time. Additionally, we estimateAuracle can last 28.1 hours with a 110 mAh battery while communicating its observations of eating behavior to a smartphone over Bluetooth

VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera

We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fully-convolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control---thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e. it works for outdoor scenes, community videos, and low quality commodity RGB cameras.Comment: Accepted to SIGGRAPH 201

Human Activity Annotation based on Active Learning

Human activity recognition algorithms have been increasingly sought due to their broad application, in areas such as healthcare, safety and sports. Current works focusing on human activity recognition are based majorly on Supervised Learning algorithms and have achieved promising results. However, high performance is achieved at the cost of a large amount of labelled data required to train and learn the model parameters, where a high volume of data will increase the algorithm’s performance and the classifier’s ability to generalise correctly into new, and previously unseen data. Commonly, the labelling process of ground truth data, which is required for supervised algorithms, must be done manually by the user, being tedious, time-consuming and difficult. On this account, we propose a Semi-Supervised Active Learning technique able to partly automate the labelling process and reduce considerably the labelling cost and the labelled data volume required to obtain a highly performing classifier. This is achieved through the selection of the most relevant samples for annotation and propagation of their label to similar samples. In order to accomplish this task, several sample selection strategies were tested in order to find the most valuable sample for labelling to be included in the classifier’s training set and create a representative set of the entire dataset. Followed by a semi-supervised stage, labelling with high confidence unlabelled samples, and augmenting the training set without any extra labelling effort from the user. Lastly, five stopping criteria were tested, optimising the trade-off between the classifier’s performance and the percentage of labelled data in its training set. Experimental results were performed on two different datasets with real data, allowing to validate the proposed method and compare it to literature methods, which were replicated. The developed model was able to reach similar accuracy values as supervised learning, with a reduction in the required labelled data of more than 89% for the two datasets, respectively