Unsupervised Human Activity Recognition through Two-stage Prompting with ChatGPT

Wearable sensor devices, which offer the advantage of recording daily objects used by a person while performing an activity, enable the feasibility of unsupervised Human Activity Recognition (HAR). Unfortunately, previous unsupervised approaches using the usage sequence of objects usually require a proper description of activities manually prepared by humans. Instead, we leverage the knowledge embedded in a Large Language Model (LLM) of ChatGPT. Because the sequence of objects robustly characterizes the activity identity, it is possible that ChatGPT already learned the association between activities and objects from existing contexts. However, previous prompt engineering for ChatGPT exhibits limited generalization ability when dealing with a list of words (i.e., sequence of objects) due to the similar weighting assigned to each word in the list. In this study, we propose a two-stage prompt engineering, which first guides ChatGPT to generate activity descriptions associated with objects while emphasizing important objects for distinguishing similar activities; then outputs activity classes and explanations for enhancing the contexts that are helpful for HAR. To the best of our knowledge, this is the first study that utilizes ChatGPT to recognize activities using objects in an unsupervised manner. We conducted our approach on three datasets and demonstrated the state-of-the-art performance.Comment: 4 page

Human behavior understanding for worker-centered intelligent manufacturing

“In a worker-centered intelligent manufacturing system, sensing and understanding of the worker’s behavior are the primary tasks, which are essential for automatic performance evaluation & optimization, intelligent training & assistance, and human-robot collaboration. In this study, a worker-centered training & assistant system is proposed for intelligent manufacturing, which is featured with self-awareness and active-guidance. To understand the hand behavior, a method is proposed for complex hand gesture recognition using Convolutional Neural Networks (CNN) with multiview augmentation and inference fusion, from depth images captured by Microsoft Kinect. To sense and understand the worker in a more comprehensive way, a multi-modal approach is proposed for worker activity recognition using Inertial Measurement Unit (IMU) signals obtained from a Myo armband and videos from a visual camera. To automatically learn the importance of different sensors, a novel attention-based approach is proposed to human activity recognition using multiple IMU sensors worn at different body locations. To deploy the developed algorithms to the factory floor, a real-time assembly operation recognition system is proposed with fog computing and transfer learning. The proposed worker-centered training & assistant system has been validated and demonstrated the feasibility and great potential for applying to the manufacturing industry for frontline workers. Our developed approaches have been evaluated: 1) the multi-view approach outperforms the state-of-the-arts on two public benchmark datasets, 2) the multi-modal approach achieves an accuracy of 97% on a worker activity dataset including 6 activities and achieves the best performance on a public dataset, 3) the attention-based method outperforms the state-of-the-art methods on five publicly available datasets, and 4) the developed transfer learning model achieves a real-time recognition accuracy of 95% on a dataset including 10 worker operations”--Abstract, page iv

Simple Yet Surprisingly Effective Training Strategies for LSTMs in Sensor-Based Human Activity Recognition

Human Activity Recognition (HAR) is one of the core research areas in mobile and wearable computing. With the application of deep learning (DL) techniques such as CNN, recognizing periodic or static activities (e.g, walking, lying, cycling, etc.) has become a well studied problem. What remains a major challenge though is the sporadic activity recognition (SAR) problem, where activities of interest tend to be non periodic, and occur less frequently when compared with the often large amount of irrelevant background activities. Recent works suggested that sequential DL models (such as LSTMs) have great potential for modeling nonperiodic behaviours, and in this paper we studied some LSTM training strategies for SAR. Specifically, we proposed two simple yet effective LSTM variants, namely delay model and inverse model, for two SAR scenarios (with and without time critical requirement). For time critical SAR, the delay model can effectively exploit predefined delay intervals (within tolerance) in form of contextual information for improved performance. For regular SAR task, the second proposed, inverse model can learn patterns from the time series in an inverse manner, which can be complementary to the forward model (i.e.,LSTM), and combining both can boost the performance. These two LSTM variants are very practical, and they can be deemed as training strategies without alteration of the LSTM fundamentals. We also studied some additional LSTM training strategies, which can further improve the accuracy. We evaluated our models on two SAR and one non-SAR datasets, and the promising results demonstrated the effectiveness of our approaches in HAR applications.Comment: 11 page

Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data

Protective behavior exhibited by people with chronic pain (CP) during physical activities is the key to understanding their physical and emotional states. Existing automatic protective behavior detection (PBD) methods rely on pre-segmentation of activities predefined by users. However, in real life, people perform activities casually. Therefore, where those activities present difficulties for people with chronic pain, technology-enabled support should be delivered continuously and automatically adapted to activity type and occurrence of protective behavior. Hence, to facilitate ubiquitous CP management, it becomes critical to enable accurate PBD over continuous data. In this paper, we propose to integrate human activity recognition (HAR) with PBD via a novel hierarchical HAR-PBD architecture comprising graph-convolution and long short-term memory (GC-LSTM) networks, and alleviate class imbalances using a class-balanced focal categorical-cross-entropy (CFCC) loss. Through in-depth evaluation of the approach using a CP patients' dataset, we show that the leveraging of HAR, GC-LSTM networks, and CFCC loss leads to clear increase in PBD performance against the baseline (macro F1 score of 0.81 vs. 0.66 and precision-recall area-under-the-curve (PR-AUC) of 0.60 vs. 0.44). We conclude by discussing possible use cases of the hierarchical architecture in CP management and beyond. We also discuss current limitations and ways forward.Comment: Submitted to PACM IMWU

Innovative intelligent sensors to objectively understand exercise interventions for older adults

The population of most western countries is ageing and, therefore, the ageing issue now matters more than ever. According to the reports of the United Nations in 2017, there were a total of 15.8 million (26.9%) people over 60 years of age in the United Kindom, and the numbers are projected to reach 23.5 million (31.5%) by 2050. Spending on medical treatment and healthcare for older adults accounts for two-fifths of the UK National Health Service (NHS) budget. Keeping older people healthy is a challenge. In general, exercise is believed to benefit both mental and physical health. Specifically, resistance band exercises are proven by many studies that they have potentially positive effects on both mental and physical health. However, treatment using resistance band exercise is usually done in unmonitored environments, such as at home or in a rehabilitation centre; therefore, the exercise cannot be measured and/or quantified accurately. Despite many years of research, the true effectiveness of resistance band exercises remains unclear. [Continues.]</div

Architecture and Applications of IoT Devices in Socially Relevant Fields

Number of IoT enabled devices are being tried and introduced every year and there is a healthy competition among researched and businesses to capitalize the space created by IoT, as these devices have a great market potential. Depending on the type of task involved and sensitive nature of data that the device handles, various IoT architectures, communication protocols and components are chosen and their performance is evaluated. This paper reviews such IoT enabled devices based on their architecture, communication protocols and functions in few key socially relevant fields like health care, farming, firefighting, women/individual safety/call for help/harm alert, home surveillance and mapping as these fields involve majority of the general public. It can be seen, to one's amazement, that already significant number of devices are being reported on these fields and their performance is promising. This paper also outlines the challenges involved in each of these fields that require solutions to make these devices reliableComment: 1

A pervasive body sensor network for monitoring post-operative recovery

Over the past decade, miniaturisation and cost reduction brought about by the semiconductor industry has led to computers smaller in size than a pin head, powerful enough to carry out the processing required, and affordable enough to be disposable. Similar technological advances in wireless communication, sensor design, and energy storage have resulted in the development of wireless “Body Sensor Network (BSN) platforms comprising of tiny integrated micro sensors with onboard processing and wireless data transfer capability, offering the prospect of pervasive and continuous home health monitoring. In surgery, the reduced trauma of minimally invasive interventions combined with initiatives to reduce length of hospital stay and a socioeconomic drive to reduce hospitalisation costs, have all resulted in a trend towards earlier discharge from hospital. There is now a real need for objective, pervasive, and continuous post-operative home recovery monitoring systems. Surgical recovery is a multi-faceted and dynamic process involving biological, physiological, functional, and psychological components. Functional recovery (physical independence, activities of daily living, and mobility) is recognised as a good global indicator of a patient’s post-operative course, but has traditionally been difficult to objectively quantify. This thesis outlines the development of a pervasive wireless BSN system to objectively monitor the functional recovery of post-operative patients at home. Biomechanical markers were identified as surrogate measures for activities of daily living and mobility impairment, and an ear-worn activity recognition (e-AR) sensor containing a three-axis accelerometer and a pulse oximeter was used to collect this data. A simulated home environment was created to test a Bayesian classifier framework with multivariate Gaussians to model activity classes. A real-time activity index was used to provide information on the intensity of activity being performed. Mobility impairment was simulated with bracing systems and a multiresolution wavelet analysis and margin-based feature selection framework was used to detect impaired mobility. The e-AR sensor was tested in a home environment before its clinical use in monitoring post-operative home recovery of real patients who have undergone surgery. Such a system may eventually form part of an objective pervasive home recovery monitoring system tailored to the needs of today’s post-operative patient.Open acces

Wearable Sensors Applied in Movement Analysis

Recent advances in electronics have led to sensors whose sizes and weights are such that they can be placed on living systems without impairing their natural motion and habits. They may be worn on the body as accessories or as part of the clothing and enable personalized mobile information processing. Wearable sensors open the way for a nonintrusive and continuous monitoring of body orientation, movements, and various physiological parameters during motor activities in real-life settings. Thus, they may become crucial tools not only for researchers, but also for clinicians, as they have the potential to improve diagnosis, better monitor disease development and thereby individualize treatment. Wearable sensors should obviously go unnoticed for the people wearing them and be intuitive in their installation. They should come with wireless connectivity and low-power consumption. Moreover, the electronics system should be self-calibrating and deliver correct information that is easy to interpret. Cross-platform interfaces that provide secure data storage and easy data analysis and visualization are needed.This book contains a selection of research papers presenting new results addressing the above challenges

Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges

Artificial General Intelligence (AGI), possessing the capacity to comprehend, learn, and execute tasks with human cognitive abilities, engenders significant anticipation and intrigue across scientific, commercial, and societal arenas. This fascination extends particularly to the Internet of Things (IoT), a landscape characterized by the interconnection of countless devices, sensors, and systems, collectively gathering and sharing data to enable intelligent decision-making and automation. This research embarks on an exploration of the opportunities and challenges towards achieving AGI in the context of the IoT. Specifically, it starts by outlining the fundamental principles of IoT and the critical role of Artificial Intelligence (AI) in IoT systems. Subsequently, it delves into AGI fundamentals, culminating in the formulation of a conceptual framework for AGI's seamless integration within IoT. The application spectrum for AGI-infused IoT is broad, encompassing domains ranging from smart grids, residential environments, manufacturing, and transportation to environmental monitoring, agriculture, healthcare, and education. However, adapting AGI to resource-constrained IoT settings necessitates dedicated research efforts. Furthermore, the paper addresses constraints imposed by limited computing resources, intricacies associated with large-scale IoT communication, as well as the critical concerns pertaining to security and privacy