1,350 research outputs found
Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention
Deep neural networks, including recurrent networks, have been successfully
applied to human activity recognition. Unfortunately, the final representation
learned by recurrent networks might encode some noise (irrelevant signal
components, unimportant sensor modalities, etc.). Besides, it is difficult to
interpret the recurrent networks to gain insight into the models' behavior. To
address these issues, we propose two attention models for human activity
recognition: temporal attention and sensor attention. These two mechanisms
adaptively focus on important signals and sensor modalities. To further improve
the understandability and mean F1 score, we add continuity constraints,
considering that continuous sensor signals are more robust than discrete ones.
We evaluate the approaches on three datasets and obtain state-of-the-art
results. Furthermore, qualitative analysis shows that the attention learned by
the models agree well with human intuition.Comment: 8 pages. published in The International Symposium on Wearable
Computers (ISWC) 201
Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognition
The ubiquitous availability of smartphones and smartwatches with integrated
inertial measurement units (IMUs) enables straightforward capturing of human
activities. For specific applications of sensor based human activity
recognition (HAR), however, logistical challenges and burgeoning costs render
especially the ground truth annotation of such data a difficult endeavor,
resulting in limited scale and diversity of datasets. Transfer learning, i.e.,
leveraging publicly available labeled datasets to first learn useful
representations that can then be fine-tuned using limited amounts of labeled
data from a target domain, can alleviate some of the performance issues of
contemporary HAR systems. Yet they can fail when the differences between source
and target conditions are too large and/ or only few samples from a target
application domain are available, each of which are typical challenges in
real-world human activity recognition scenarios. In this paper, we present an
approach for economic use of publicly available labeled HAR datasets for
effective transfer learning. We introduce a novel transfer learning framework,
Cross-Domain HAR, which follows the teacher-student self-training paradigm to
more effectively recognize activities with very limited label information. It
bridges conceptual gaps between source and target domains, including sensor
locations and type of activities. Through our extensive experimental evaluation
on a range of benchmark datasets, we demonstrate the effectiveness of our
approach for practically relevant few shot activity recognition scenarios. We
also present a detailed analysis into how the individual components of our
framework affect downstream performance
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
The vast proliferation of sensor devices and Internet of Things enables the
applications of sensor-based activity recognition. However, there exist
substantial challenges that could influence the performance of the recognition
system in practical scenarios. Recently, as deep learning has demonstrated its
effectiveness in many areas, plenty of deep methods have been investigated to
address the challenges in activity recognition. In this study, we present a
survey of the state-of-the-art deep learning methods for sensor-based human
activity recognition. We first introduce the multi-modality of the sensory data
and provide information for public datasets that can be used for evaluation in
different challenge tasks. We then propose a new taxonomy to structure the deep
methods by challenges. Challenges and challenge-related deep methods are
summarized and analyzed to form an overview of the current research progress.
At the end of this work, we discuss the open issues and provide some insights
for future directions
Human behavior understanding for worker-centered intelligent manufacturing
“In a worker-centered intelligent manufacturing system, sensing and understanding of the worker’s behavior are the primary tasks, which are essential for automatic performance evaluation & optimization, intelligent training & assistance, and human-robot collaboration. In this study, a worker-centered training & assistant system is proposed for intelligent manufacturing, which is featured with self-awareness and active-guidance. To understand the hand behavior, a method is proposed for complex hand gesture recognition using Convolutional Neural Networks (CNN) with multiview augmentation and inference fusion, from depth images captured by Microsoft Kinect. To sense and understand the worker in a more comprehensive way, a multi-modal approach is proposed for worker activity recognition using Inertial Measurement Unit (IMU) signals obtained from a Myo armband and videos from a visual camera. To automatically learn the importance of different sensors, a novel attention-based approach is proposed to human activity recognition using multiple IMU sensors worn at different body locations. To deploy the developed algorithms to the factory floor, a real-time assembly operation recognition system is proposed with fog computing and transfer learning. The proposed worker-centered training & assistant system has been validated and demonstrated the feasibility and great potential for applying to the manufacturing industry for frontline workers. Our developed approaches have been evaluated: 1) the multi-view approach outperforms the state-of-the-arts on two public benchmark datasets, 2) the multi-modal approach achieves an accuracy of 97% on a worker activity dataset including 6 activities and achieves the best performance on a public dataset, 3) the attention-based method outperforms the state-of-the-art methods on five publicly available datasets, and 4) the developed transfer learning model achieves a real-time recognition accuracy of 95% on a dataset including 10 worker operations”--Abstract, page iv
TASKED: Transformer-based Adversarial learning for human activity recognition using wearable sensors via Self-KnowledgE Distillation
Wearable sensor-based human activity recognition (HAR) has emerged as a
principal research area and is utilized in a variety of applications. Recently,
deep learning-based methods have achieved significant improvement in the HAR
field with the development of human-computer interaction applications. However,
they are limited to operating in a local neighborhood in the process of a
standard convolution neural network, and correlations between different sensors
on body positions are ignored. In addition, they still face significant
challenging problems with performance degradation due to large gaps in the
distribution of training and test data, and behavioral differences between
subjects. In this work, we propose a novel Transformer-based Adversarial
learning framework for human activity recognition using wearable sensors via
Self-KnowledgE Distillation (TASKED), that accounts for individual sensor
orientations and spatial and temporal features. The proposed method is capable
of learning cross-domain embedding feature representations from multiple
subjects datasets using adversarial learning and the maximum mean discrepancy
(MMD) regularization to align the data distribution over multiple domains. In
the proposed method, we adopt the teacher-free self-knowledge distillation to
improve the stability of the training procedure and the performance of human
activity recognition. Experimental results show that TASKED not only
outperforms state-of-the-art methods on the four real-world public HAR datasets
(alone or combined) but also improves the subject generalization effectively.Comment: 17 pages, 5 figures, Submitted to Knowledge-Based Systems, Elsevier.
arXiv admin note: substantial text overlap with arXiv:2110.1216
Activity Classification Using Unsupervised Domain Transfer from Body Worn Sensors
Activity classification has become a vital feature of wearable health
tracking devices. As innovation in this field grows, wearable devices worn on
different parts of the body are emerging. To perform activity classification on
a new body location, labeled data corresponding to the new locations are
generally required, but this is expensive to acquire. In this work, we present
an innovative method to leverage an existing activity classifier, trained on
Inertial Measurement Unit (IMU) data from a reference body location (the source
domain), in order to perform activity classification on a new body location
(the target domain) in an unsupervised way, i.e. without the need for
classification labels at the new location. Specifically, given an IMU embedding
model trained to perform activity classification at the source domain, we train
an embedding model to perform activity classification at the target domain by
replicating the embeddings at the source domain. This is achieved using
simultaneous IMU measurements at the source and target domains. The replicated
embeddings at the target domain are used by a classification model that has
previously been trained on the source domain to perform activity classification
at the target domain. We have evaluated the proposed methods on three activity
classification datasets PAMAP2, MHealth, and Opportunity, yielding high F1
scores of 67.19%, 70.40% and 68.34%, respectively when the source domain is the
wrist and the target domain is the torso
Transfer Learning in Human Activity Recognition: A Survey
Sensor-based human activity recognition (HAR) has been an active research
area, owing to its applications in smart environments, assisted living,
fitness, healthcare, etc. Recently, deep learning based end-to-end training has
resulted in state-of-the-art performance in domains such as computer vision and
natural language, where large amounts of annotated data are available. However,
large quantities of annotated data are not available for sensor-based HAR.
Moreover, the real-world settings on which the HAR is performed differ in terms
of sensor modalities, classification tasks, and target users. To address this
problem, transfer learning has been employed extensively. In this survey, we
focus on these transfer learning methods in the application domains of smart
home and wearables-based HAR. In particular, we provide a problem-solution
perspective by categorizing and presenting the works in terms of their
contributions and the challenges they address. We also present an updated view
of the state-of-the-art for both application domains. Based on our analysis of
205 papers, we highlight the gaps in the literature and provide a roadmap for
addressing them. This survey provides a reference to the HAR community, by
summarizing the existing works and providing a promising research agenda.Comment: 40 pages, 5 figures, 7 table
- …