CAVIAR: Context-driven Active and Incremental Activity Recognition

Activity recognition on mobile device sensor data has been an active research area in mobile and pervasive computing for several years. While the majority of the proposed techniques are based on supervised learning, semi-supervised approaches are being considered to reduce the size of the training set required to initialize the model. These approaches usually apply self-training or active learning to incrementally refine the model, but their effectiveness seems to be limited to a restricted set of physical activities. We claim that the context which surrounds the user (e.g., time, location, proximity to transportation routes) combined with common knowledge about the relationship between context and human activities could be effective in significantly increasing the set of recognized activities including those that are difficult to discriminate only considering inertial sensors, and the highly context-dependent ones. In this paper, we propose CAVIAR, a novel hybrid semi-supervised and knowledge-based system for real-time activity recognition. Our method applies semantic reasoning on context-data to refine the predictions of an incremental classifier. The recognition model is continuously updated using active learning. Results on a real dataset obtained from 26 subjects show the effectiveness of our approach in increasing the recognition rate, extending the number of recognizable activities and, most importantly, reducing the number of queries triggered by active learning. In order to evaluate the impact of context reasoning, we also compare CAVIAR with a purely statistical version, considering features computed on context-data as part of the machine learning process

Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone

Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of such datasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigate the robustness of common supervised classification approaches under instances of noisy data. We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85% average precision rate. Results obtained also show how a supervised approach trained using automatically generated labels achieved an 84% f-score (using Neural Networks and Random Forests); however, results also demonstrated how the presence of label noise could lower the f-score up to 64–74% depending on the classification approach (Nearest Centroid and Multi-Class Support Vector Machine)

Image-based road type classification

The ability to automatically determine the road type from sensor data is of great significance for automatic annotation of routes and autonomous navigation of robots and vehicles. In this paper, we present a novel algorithm for content-based road type classification from images. The proposed method learns discriminative features from training data in an unsupervised manner, thus not requiring domain-specific feature engineering. This is an advantage over related road surface classification algorithms which are only able to make a distinction between pre-specified uniform terrains. In order to evaluate the proposed approach, we have constructed a challenging road image dataset of 20,000 samples from real-world road images in the paved and unpaved road classes. Experimental results on this dataset show that the proposed algorithm can achieve state-of-the-art performance in road type classification

Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities

The vast proliferation of sensor devices and Internet of Things enables the applications of sensor-based activity recognition. However, there exist substantial challenges that could influence the performance of the recognition system in practical scenarios. Recently, as deep learning has demonstrated its effectiveness in many areas, plenty of deep methods have been investigated to address the challenges in activity recognition. In this study, we present a survey of the state-of-the-art deep learning methods for sensor-based human activity recognition. We first introduce the multi-modality of the sensory data and provide information for public datasets that can be used for evaluation in different challenge tasks. We then propose a new taxonomy to structure the deep methods by challenges. Challenges and challenge-related deep methods are summarized and analyzed to form an overview of the current research progress. At the end of this work, we discuss the open issues and provide some insights for future directions

Human Activity Annotation based on Active Learning

Human activity recognition algorithms have been increasingly sought due to their broad application, in areas such as healthcare, safety and sports. Current works focusing on human activity recognition are based majorly on Supervised Learning algorithms and have achieved promising results. However, high performance is achieved at the cost of a large amount of labelled data required to train and learn the model parameters, where a high volume of data will increase the algorithm’s performance and the classifier’s ability to generalise correctly into new, and previously unseen data. Commonly, the labelling process of ground truth data, which is required for supervised algorithms, must be done manually by the user, being tedious, time-consuming and difficult. On this account, we propose a Semi-Supervised Active Learning technique able to partly automate the labelling process and reduce considerably the labelling cost and the labelled data volume required to obtain a highly performing classifier. This is achieved through the selection of the most relevant samples for annotation and propagation of their label to similar samples. In order to accomplish this task, several sample selection strategies were tested in order to find the most valuable sample for labelling to be included in the classifier’s training set and create a representative set of the entire dataset. Followed by a semi-supervised stage, labelling with high confidence unlabelled samples, and augmenting the training set without any extra labelling effort from the user. Lastly, five stopping criteria were tested, optimising the trade-off between the classifier’s performance and the percentage of labelled data in its training set. Experimental results were performed on two different datasets with real data, allowing to validate the proposed method and compare it to literature methods, which were replicated. The developed model was able to reach similar accuracy values as supervised learning, with a reduction in the required labelled data of more than 89% for the two datasets, respectively