Behaviour Profiling using Wearable Sensors for Pervasive Healthcare

In recent years, sensor technology has advanced in terms of hardware sophistication and miniaturisation. This has led to the incorporation of unobtrusive, low-power sensors into networks centred on human participants, called Body Sensor Networks. Amongst the most important applications of these networks is their use in healthcare and healthy living. The technology has the possibility of decreasing burden on the healthcare systems by providing care at home, enabling early detection of symptoms, monitoring recovery remotely, and avoiding serious chronic illnesses by promoting healthy living through objective feedback. In this thesis, machine learning and data mining techniques are developed to estimate medically relevant parameters from a participant‘s activity and behaviour parameters, derived from simple, body-worn sensors. The first abstraction from raw sensor data is the recognition and analysis of activity. Machine learning analysis is applied to a study of activity profiling to detect impaired limb and torso mobility. One of the advances in this thesis to activity recognition research is in the application of machine learning to the analysis of 'transitional activities': transient activity that occurs as people change their activity. A framework is proposed for the detection and analysis of transitional activities. To demonstrate the utility of transition analysis, we apply the algorithms to a study of participants undergoing and recovering from surgery. We demonstrate that it is possible to see meaningful changes in the transitional activity as the participants recover. Assuming long-term monitoring, we expect a large historical database of activity to quickly accumulate. We develop algorithms to mine temporal associations to activity patterns. This gives an outline of the user‘s routine. Methods for visual and quantitative analysis of routine using this summary data structure are proposed and validated. The activity and routine mining methodologies developed for specialised sensors are adapted to a smartphone application, enabling large-scale use. Validation of the algorithms is performed using datasets collected in laboratory settings, and free living scenarios. Finally, future research directions and potential improvements to the techniques developed in this thesis are outlined

Lightweight Power Aware and Scalable Movement Monitoring for Wearable Computers: A Mining and Recognition Technique At the Fingertip of Sensors

ABSTRACT Activity monitoring using Body Sensor Networks(BSN) has gained much attention from the scientific community due to its recreational and medical applications. Suggested techniques for activity monitoring face two major problem. First, systems have to be trained for the individual subjects due to the heterogeneity of the BSN data. While most solutions can address this problem on a small data set, they have no mechanics for automatic scaling of the solution as the data set increases. Second, the battery limitations of the BSN severely limit the maximum deployment time for the continuous monitoring. This problem is often solved by shifting some processing to the local sensor nodes to avoid a very heavy communication cost. However, little work has been done to optimize the sensing and processing cost of the action recognition. In this paper, we propose an action recognition approach based on the BSN repository. We show how the information of a large repository can be automatically used to customize the processing on sensor nodes based on a limited and automated training process. We also investigate the power cost of such a repository mining approach on the sensor nodes based on our implementation. To assess the power requirement, we define an energy model for data sensing and processing. We demonstrate the relationship between the activity recognition precision and the power consumption of the system during continuous action monitoring. We demonstrate the energy effectiveness of our approach with a classification accuracy constraint based on limited data repository

On the structure of natural human movement

Understanding of human motor control is central to neuroscience with strong implications in the fields of medicine, robotics and evolution. It is thus surprising that the vast majority of motor control studies have focussed on human movement in the laboratory while neglecting behaviour in natural environments. We developed an experimental paradigm to quantify human behaviour in high resolution over extended periods of time in ecologically relevant environments. This allows us to discover novel insights and contradictory evidence to well-established findings obtained in controlled laboratory conditions. Using our data, we map the statistics of natural human movement and their variability between people. The variability and complexity of the data recorded in these settings required us to develop new tools to extract meaningful information in an objective, data-driven fashion. Moving from descriptive statistics to structure, we identify stable structures of movement coordination, particularly within the arm-hand area. Combining our data with numerous published findings, we argue that current hypotheses that the brain simplifies motor control problems by dimensionality reduction are too reductionist. We propose an alternative hypothesis derived from sparse coding theory, a concept which has been successfully applied to the sensory system. To investigate this idea, we develop an algorithm for unsupervised identification of sparse structures in natural movement data. Our method outperforms state-of-the-art algorithms for accuracy and data-efficiency. Applying this method to hand data reveals a dictionary of \emph{sparse eigenmotions} (SEMs) which are well preserved across multiple subjects. These are highly efficient and invariant representation of natural movement, and suggest a potential higher-order grammatical structure or ``movement language''. Our findings make a number of testable predictions about neural coding of movement in the cortex. This has direct consequences for advancing research on dextrous prosthetics and robotics, and has profound implications for our understanding of how the brain controls our body.Open Acces

Multi-sensor fusion for human-robot interaction in crowded environments

For challenges associated with the ageing population, robot assistants are becoming a promising solution. Human-Robot Interaction (HRI) allows a robot to understand the intention of humans in an environment and react accordingly. This thesis proposes HRI techniques to facilitate the transition of robots from lab-based research to real-world environments. The HRI aspects addressed in this thesis are illustrated in the following scenario: an elderly person, engaged in conversation with friends, wishes to attract a robot's attention. This composite task consists of many problems. The robot must detect and track the subject in a crowded environment. To engage with the user, it must track their hand movement. Knowledge of the subject's gaze would ensure that the robot doesn't react to the wrong person. Understanding the subject's group participation would enable the robot to respect existing human-human interaction. Many existing solutions to these problems are too constrained for natural HRI in crowded environments. Some require initial calibration or static backgrounds. Others deal poorly with occlusions, illumination changes, or real-time operation requirements. This work proposes algorithms that fuse multiple sensors to remove these restrictions and increase the accuracy over the state-of-the-art. The main contributions of this thesis are: A hand and body detection method, with a probabilistic algorithm for their real-time association when multiple users and hands are detected in crowded environments; An RGB-D sensor-fusion hand tracker, which increases position and velocity accuracy by combining a depth-image based hand detector with Monte-Carlo updates using colour images; A sensor-fusion gaze estimation system, combining IR and depth cameras on a mobile robot to give better accuracy than traditional visual methods, without the constraints of traditional IR techniques; A group detection method, based on sociological concepts of static and dynamic interactions, which incorporates real-time gaze estimates to enhance detection accuracy.Open Acces

Robust Signal Processing Techniques for Wearable Inertial Measurement Unit (IMU) Sensors

Activity and gesture recognition using wearable motion sensors, also known as inertial measurement units (IMUs), provides important context for many ubiquitous sensing applications including healthcare monitoring, human computer interface and context-aware smart homes and offices. Such systems are gaining popularity due to their minimal cost and ability to provide sensing functionality at any time and place. However, several factors can affect the system performance such as sensor location and orientation displacement, activity and gesture inconsistency, movement speed variation and lack of tiny motion information. This research is focused on developing signal processing solutions to ensure the system robustness with respect to these factors. Firstly, for existing systems which have already been designed to work with certain sensor orientation/location, this research proposes opportunistic calibration algorithms leveraging camera information from the environment to ensure the system performs correctly despite location or orientation displacement of the sensors. The calibration algorithms do not require extra effort from the users and the calibration is done seamlessly when the users present in front of an environmental camera and perform arbitrary movements. Secondly, an orientation independent and speed independent approach is proposed and studied by exploring a novel orientation independent feature set and by intelligently selecting only the relevant and consistent portions of various activities and gestures. Thirdly, in order to address the challenge that the IMU is not able capture tiny motion which is important to some applications, a sensor fusion framework is proposed to fuse the complementary sensor modality in order to enhance the system performance and robustness. For example, American Sign Language has a large vocabulary of signs and a recognition system solely based on IMU sensors would not perform very well. In order to demonstrate the feasibility of sensor fusion techniques, a robust real-time American Sign Language recognition approach is developed using wrist worn IMU and surface electromyography (EMG) sensors

Robust Signal Processing Techniques for Wearable Inertial Measurement Unit (IMU) Sensors

Activity and gesture recognition using wearable motion sensors, also known as inertial measurement units (IMUs), provides important context for many ubiquitous sensing applications including healthcare monitoring, human computer interface and context-aware smart homes and offices. Such systems are gaining popularity due to their minimal cost and ability to provide sensing functionality at any time and place. However, several factors can affect the system performance such as sensor location and orientation displacement, activity and gesture inconsistency, movement speed variation and lack of tiny motion information. This research is focused on developing signal processing solutions to ensure the system robustness with respect to these factors. Firstly, for existing systems which have already been designed to work with certain sensor orientation/location, this research proposes opportunistic calibration algorithms leveraging camera information from the environment to ensure the system performs correctly despite location or orientation displacement of the sensors. The calibration algorithms do not require extra effort from the users and the calibration is done seamlessly when the users present in front of an environmental camera and perform arbitrary movements. Secondly, an orientation independent and speed independent approach is proposed and studied by exploring a novel orientation independent feature set and by intelligently selecting only the relevant and consistent portions of various activities and gestures. Thirdly, in order to address the challenge that the IMU is not able capture tiny motion which is important to some applications, a sensor fusion framework is proposed to fuse the complementary sensor modality in order to enhance the system performance and robustness. For example, American Sign Language has a large vocabulary of signs and a recognition system solely based on IMU sensors would not perform very well. In order to demonstrate the feasibility of sensor fusion techniques, a robust real-time American Sign Language recognition approach is developed using wrist worn IMU and surface electromyography (EMG) sensors

Grasp Classification with Weft Knit Data Glove using a Convolutional Neural Network

Grasp classification using data gloves can enable therapists to monitor patients efficiently by providing concise information about the activities performed by these patients. Although, classical machine learning algorithms have been applied in grasp classification, they require manual feature extraction to achieve high accuracy. In contrast, convolutional neural networks (CNNs) have outperformed popular machine learning algorithms in several classification scenarios because of their ability to extract features automatically from raw data. However, they have not been implemented on grasp classification using a data glove. In this study, we apply a CNN in grasp classification using a piezoresistive textile data glove knitted from conductive yarn and an elastomeric yarn. The data glove was used to collect data from five participants who grasped thirty objects each following Schlesinger’s taxonomy. We investigate a CNN’s performance in two scenarios where the validation objects are known and unknown. Our results show that a simple CNN architecture outperformed k-nn, Gaussian SVM, and Decision Tree algorithms in both scenarios in terms of the classification accuracy

Using Hidden Markov Models to Segment and Classify Wrist Motions Related to Eating Activities

Advances in body sensing and mobile health technology have created new opportunities for empowering people to take a more active role in managing their health. Measurements of dietary intake are commonly used for the study and treatment of obesity. However, the most widely used tools rely upon self-report and require considerable manual effort, leading to underreporting of consumption, non-compliance, and discontinued use over the long term. We are investigating the use of wrist-worn accelerometers and gyroscopes to automatically recognize eating gestures. In order to improve recognition accuracy, we studied the sequential ependency of actions during eating. In chapter 2 we first undertook the task of finding a set of wrist motion gestures which were small and descriptive enough to model the actions performed by an eater during consumption of a meal. We found a set of four actions: rest, utensiling, bite, and drink; any alternative gestures is referred as the other gesture. The stability of the definitions for gestures was evaluated using an inter-rater reliability test. Later, in chapter 3, 25 meals were hand labeled and used to study the existence of sequential dependence of the gestures. To study this, three types of classifiers were built: 1) a K-nearest neighbor classifier which uses no sequential context, 2) a hidden Markov model (HMM) which captures the sequential context of sub-gesture motions, and 3) HMMs that model inter-gesture sequential dependencies. We built first-order to sixth-order HMMs to evaluate the usefulness of increasing amounts of sequential dependence to aid recognition. The first two were our baseline algorithms. We found that the adding knowledge of the sequential dependence of gestures achieved an accuracy of 96.5%, which is an improvement of 20.7% and 12.2% over the KNN and sub-gesture HMM. Lastly, in chapter 4, we automatically segmented a continuous wrist motion signal and assessed its classification performance for each of the three classifiers. Again, the knowledge of sequential dependence enhances the recognition of gestures in unsegmented data, achieving 90% accuracy and improving 30.1% and 18.9% over the KNN and the sub-gesture HMM

Physiological and behavior monitoring systems for smart healthcare environments: a review

Healthcare optimization has become increasingly important in the current era, where numerous challenges are posed by population ageing phenomena and the demand for higher quality of the healthcare services. The implementation of Internet of Things (IoT) in the healthcare ecosystem has been one of the best solutions to address these challenges and therefore to prevent and diagnose possible health impairments in people. The remote monitoring of environmental parameters and how they can cause or mediate any disease, and the monitoring of human daily activities and physiological parameters are among the vast applications of IoT in healthcare, which has brought extensive attention of academia and industry. Assisted and smart tailored environments are possible with the implementation of such technologies that bring personal healthcare to any individual, while living in their preferred environments. In this paper we address several requirements for the development of such environments, namely the deployment of physiological signs monitoring systems, daily activity recognition techniques, as well as indoor air quality monitoring solutions. The machine learning methods that are most used in the literature for activity recognition and body motion analysis are also referred. Furthermore, the importance of physical and cognitive training of the elderly population through the implementation of exergames and immersive environments is also addressedinfo:eu-repo/semantics/publishedVersio