Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence

Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models due to privacy, bandwidth limitations, and the prohibitive cost of annotations. Federated learning provides a compelling framework for learning models from decentralized data, but conventionally, it assumes the availability of labeled samples, whereas on-device data are generally either unlabeled or cannot be annotated readily through user interaction. To address these issues, we propose a self-supervised approach termed \textit{scalogram-signal correspondence learning} based on wavelet transform to learn useful representations from unlabeled sensor inputs, such as electroencephalography, blood volume pulse, accelerometer, and WiFi channel state information. Our auxiliary task requires a deep temporal neural network to determine if a given pair of a signal and its complementary viewpoint (i.e., a scalogram generated with a wavelet transform) align with each other or not through optimizing a contrastive objective. We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains. We demonstrate the effectiveness of representations learned from an unlabeled input collection on downstream tasks with training a linear classifier over pretrained network, usefulness in low-data regime, transfer learning, and cross-validation. Our methodology achieves competitive performance with fully-supervised networks, and it outperforms pre-training with autoencoders in both central and federated contexts. Notably, it improves the generalization in a semi-supervised setting as it reduces the volume of labeled data required through leveraging self-supervised learning.Comment: Accepted for publication at IEEE Internet of Things Journa

Cross-position Activity Recognition with Stratified Transfer Learning

Human activity recognition aims to recognize the activities of daily living by utilizing the sensors on different body parts. However, when the labeled data from a certain body position (i.e. target domain) is missing, how to leverage the data from other positions (i.e. source domain) to help learn the activity labels of this position? When there are several source domains available, it is often difficult to select the most similar source domain to the target domain. With the selected source domain, we need to perform accurate knowledge transfer between domains. Existing methods only learn the global distance between domains while ignoring the local property. In this paper, we propose a \textit{Stratified Transfer Learning} (STL) framework to perform both source domain selection and knowledge transfer. STL is based on our proposed \textit{Stratified} distance to capture the local property of domains. STL consists of two components: Stratified Domain Selection (STL-SDS) can select the most similar source domain to the target domain; Stratified Activity Transfer (STL-SAT) is able to perform accurate knowledge transfer. Extensive experiments on three public activity recognition datasets demonstrate the superiority of STL. Furthermore, we extensively investigate the performance of transfer learning across different degrees of similarities and activity levels between domains. We also discuss the potential applications of STL in other fields of pervasive computing for future research.Comment: Submit to Pervasive and Mobile Computing as an extension to PerCom 18 paper; First revision. arXiv admin note: substantial text overlap with arXiv:1801.0082

What Will You Do for the Rest of the Day?

Understanding and predicting human mobility is vital to a large number of applications, ranging from recommendations to safety and urban service planning. In some travel applications, the ability to accurately predict the user's future trajectory is vital for delivering high quality of service. The accurate prediction of detailed trajectories would empower location-based service providers with the ability to deliver more precise recommendations to users. Existing work on human mobility prediction has mainly focused on the prediction of the next location (or the set of locations) visited by the user, rather than on the prediction of the continuous trajectory (sequences of further locations and the corresponding arrival and departure times). Furthermore, existing approaches often return predicted locations as regions with coarse granularity rather than geographical coordinates, which limits the practicality of the prediction. In this paper, we introduce a novel trajectory prediction problem: given historical data and a user's initial trajectory in the morning, can we predict the user's full trajectory later in the day (e.g. the afternoon trajectory)? The predicted continuous trajectory includes the sequence of future locations, the stay times, and the departure times. We first conduct a comprehensive analysis about the relationship between morning trajectories and the corresponding afternoon trajectories, and found there is a positive correlation between them. Our proposed method combines similarity metrics over the extracted temporal sequences of locations to estimate similar informative segments across user trajectories. Our evaluation shows results on both labeled and geographical trajectories with a prediction error reduced by 10-35% in comparison to the baselines. This improvement has the potential to enable precise location services, raising usefulness to users to unprecedented levels. We also present empirical evaluations with Markov model and Long Short Term Memory (LSTM), a state-of-the-art Recurrent Neural Network model. Our proposed method is shown to be more effective when smaller number of samples are used and is exponentially more efficient than LSTM.</jats:p

Information gain-based metric for recognizing transitions in human activities

This paper aims to observe and recognize transition times, when human activities change. No generic method has been proposed for extracting transition times at different levels of activity granularity. Existing work in human behavior analysis and activity recognition has mainly used predefined sliding windows or fixed segments, either at low-level, such as standing or walking, or high-level, such as dining or commuting to work. We present an Information Gain-based Temporal Segmentation method (IGTS), an unsupervised segmentation technique, to find the transition times in human activities and daily routines, from heterogeneous sensor data. The proposed IGTS method is applicable for low-level activities, where each segment captures a single activity, such as walking, that is going to be recognized or predicted, and also for high-level activities. The heterogeneity of sensor data is dealt with a data transformation stage. The generic method has been thoroughly evaluated on a variety of labeled and unlabeled activity recognition and routine datasets from smartphones and device-free infrastructures. The experiment results demonstrate the robustness of the method, as all segments of low- and high-level activities can be captured from different datasets with minimum error and high computational efficiency