Probabilistic modelling and inference of human behaviour from mobile phone time series

With an estimated 4.1 billion subscribers around the world, the mobile phone offers a unique opportunity to sense and understand human behaviour from location, co-presence and communication data. While the benefit of modelling this unprecedented amount of data is widely recognised, a number of challenges impede the development of accurate behaviour models. In this thesis, we identify and address two modelling problems and show that their consideration improves the accuracy of behaviour inference. We first examine the modelling of long-range dependencies in human behaviour. Human behaviour models only take into account short-range dependencies in mobile phone time series. Using information theory, we quantify long-range dependencies in mobile phone time series for the first time, demonstrate that they exhibit periodic oscillations and introduce novel tools to analyse them. We further show that considering what the user did 24 hours earlier improves accuracy when predicting user behaviour five hours or longer in advance. The second problem that we address is the modelling of temporal variations in human behaviour. The time spent by a user on an activity varies from one day to the next. In order to recognise behaviour patterns despite temporal variations, we establish a methodological connection between human behaviour modelling and biological sequence alignment. This connection allows us to compare, cluster and model behaviour sequences and introduce novel features for behaviour recognition which improve its accuracy. The experiments presented in this thesis have been conducted on the largest publicly available mobile phone dataset labelled in an unsupervised fashion and are entirely repeatable. Furthermore, our techniques only require cellular data which can easily be recorded by today's mobile phones and could benefit a wide range of applications including life logging, health monitoring, customer profiling and large-scale surveillance

Inference Engine Based on a Hierarchical Structure for Detecting Everyday Activities within the Home

One of the key objectives of an ambient assisted living environment is to enable elderly people to lead a healthy and independent life. These assisted environments have the capability to capture and infer activities performed by individuals, which can be useful for providing assistance and tracking functional decline among the elderly community. This paper presents an activity recognition engine based on a hierarchal structure, which allows modelling, representation and recognition of ADLs, their associated tasks, objects, relationships and dependencies. The structure of this contextual information plays a vital role in conducting accurate ADL recognition. The recognition performance of the inference engine has been validated with a series of experiments based on object usage data collected within the home environment

An Investigation of Indoor Positioning Systems and their Applications

PhDActivities of Daily Living (ADL) are important indicators of both cognitive and physical well-being in healthy and ill humans. There is a range of methods to recognise ADLs, each with its own limitations. The focus of this research was on sensing location-driven activities, in which ADLs are derived from location sensed using Radio Frequency (RF, e.g., WiFi or BLE), Magnetic Field (MF) and light (e.g., Lidar) measurements in three different environments. This research discovered that different environments can have different constraints and requirements. It investigated how to improve the positioning accuracy and hence how to improve the ADL recognition accuracy. There are several challenges that need to be addressed in order to do this. First, RF location fingerprinting is affected by the heterogeneity smartphones and their orientation with respect to transmitters, increasing the location determination error. To solve this, a novel Received Signal Strength Indication (RSSI) ranking based location fingerprinting methods that use Kendall Tau Correlation Coefficient (KTCC) and Convolutional Neural Networks (CNN) are proposed to correlate a signal position to pre-defined Reference Points (RPs) or fingerprints, more accurately, The accuracy has increased by up to 25.8% when compared to using Euclidean Distance (ED) based Weighted K-Nearest Neighbours Algorithm (WKNN). Second, the use of MF measurements as fingerprints can overcome some additional RF fingerprinting challenges, as MF measurements are far more invariant to static and dynamic physical objects that affect RF transmissions. Hence, a novel fast path matching data algorithm for an MF sensor combined with an Inertial Measurement Unit (IMU) to determine direction was researched and developed. It can achieve an average of 1.72 m positioning accuracy when the user walks far fewer (5) steps. Third, a device-free or off-body novel location-driven ADL method based upon 2D Lidar was investigated. An innovative method for recognising daily activities using a Seq2Seq model to analyse location data from a low-cost rotating 2D Lidar is proposed. It provides an accuracy of 88% when recognising 17 targeted ADLs. These proposed methods in this thesis have been validated in real environments.Chinese Scholarship Counci

Learning Human Behaviour Patterns by Trajectory and Activity Recognition

The world’s population is ageing, increasing the awareness of neurological and behavioural impairments that may arise from the human ageing. These impairments can be manifested by cognitive conditions or mobility reduction. These conditions are difficult to be detected on time, relying only on the periodic medical appointments. Therefore, there is a lack of routine screening which demands the development of solutions to better assist and monitor human behaviour. The available technologies to monitor human behaviour are limited to indoors and require the installation of sensors around the user’s homes presenting high maintenance and installation costs. With the widespread use of smartphones, it is possible to take advantage of their sensing information to better assist the elderly population. This study investigates the question of what we can learn about human pattern behaviour from this rich and pervasive mobile sensing data. A deployment of a data collection over a period of 6 months was designed to measure three different human routines through human trajectory analysis and activity recognition comprising indoor and outdoor environment. A framework for modelling human behaviour was developed using human motion features, extracted in an unsupervised and supervised manner. The unsupervised feature extraction is able to measure mobility properties such as step length estimation, user points of interest or even locomotion activities inferred from an user-independent trained classifier. The supervised feature extraction was design to be user-dependent as each user may have specific behaviours that are common to his/her routine. The human patterns were modelled through probability density functions and clustering approaches. Using the human learned patterns, inferences about the current human behaviour were continuously quantified by an anomaly detection algorithm, where distance measurements were used to detect significant changes in behaviour. Experimental results demonstrate the effectiveness of the proposed framework that revealed an increase potential to learn behaviour patterns and detect anomalies

Inferring Activities of Daily Living of Home-Care Patients Through Wearable and Ambient Sensing

There is an increasing demand for remote healthcare systems for single person households as it facilitates independent living in a smart home setting. Much research effort has been invested to develop such systems to monitor and infer if the person is able to perform their routine activities on a daily basis. In this research study, two different methods have been proposed for recognizing activities of daily life (ADL) using wearable and ambient sensing respectively. The thesis presents a novel algorithm for near real-time recognition of low-level micro-activities and their associated zone of occurrence within the house by using just the wearable as the lone sensor data. This is achieved by gathering location information of the target person using a wearable beacon embedded with magnetometer and inertial sensors. A hybrid three-tier approach is adopted where the main intention is to map the location of a person performing an activity with pre-defined house landmarks and zones in the offline labeled database. Experimental results demonstrate that it is possible to achieve centimeter-level accuracy for recognition of micro-activities and a classification accuracy of 85% for trajectory prediction. Furthermore, addi-tional tests were carried out to assess whether increased antenna gain improves the ranking accuracy of the fingerprinting method adopted for location estimation. The thesis explores another method using ambient sensors for activity recognition by integrating stream reasoning, ontological modeling and probabilistic inference using Markov Logic Networks. The incoming sensor data stream is analyzed in real time by exploring semantic relationships, location context and temporal rea-soning between individual events using a stream-processing engine. Experimental analysis of the proposed method with two real-world datasets shows improvement in recognizing complex activities carried out in a smart home environment. An average F-measure score of 92.35% and 85.75% was achieved for recognition of interwoven activities using this method

Semantic interpretation of events in lifelogging

The topic of this thesis is lifelogging, the automatic, passive recording of a person’s daily activities and in particular, on performing a semantic analysis and enrichment of lifelogged data. Our work centers on visual lifelogged data, such as taken from wearable cameras. Such wearable cameras generate an archive of a person’s day taken from a first-person viewpoint but one of the problems with this is the sheer volume of information that can be generated. In order to make this potentially very large volume of information more manageable, our analysis of this data is based on segmenting each day’s lifelog data into discrete and non-overlapping events corresponding to activities in the wearer’s day. To manage lifelog data at an event level, we define a set of concepts using an ontology which is appropriate to the wearer, applying automatic detection of concepts to these events and then semantically enriching each of the detected lifelog events making them an index into the events. Once this enrichment is complete we can use the lifelog to support semantic search for everyday media management, as a memory aid, or as part of medical analysis on the activities of daily living (ADL), and so on. In the thesis, we address the problem of how to select the concepts to be used for indexing events and we propose a semantic, density- based algorithm to cope with concept selection issues for lifelogging. We then apply activity detection to classify everyday activities by employing the selected concepts as high-level semantic features. Finally, the activity is modeled by multi-context representations and enriched by Semantic Web technologies. The thesis includes an experimental evaluation using real data from users and shows the performance of our algorithms in capturing the semantics of everyday concepts and their efficacy in activity recognition and semantic enrichment

User Mobility Detection using Foot Force Sensors and Mobile Phone GPS.

PhDA user (or human) mobility context is defined as a type of user context that describes a type of whole body posture (e.g., standing versus sitting) and/or a type of travel or transportation mode (e.g., walking, cycling, travel by bus, etc). Such a context can be derived from low-level sensor data and spatial contexts, including location coordinates, 3D-orientation, direction (with respect to magnetic north), velocity and acceleration. Different value-added services can be adapted to users’ mobility contexts such as assessing how eco-friendly our travel is, and adapting travel information services such as maps to different transportation modes. Current sensor-based methods for user mobility detection have several key limitations: narrow range of recognition, coarse user mobility recognition capability, and low recognition accuracy. In this thesis, a new Foot-Force and GPS (FF+GPS) sensor method is proposed to overcome these challenges that leverages a set of wearable FF sensors in combination with mobile phone GPS. The novelty of this approach is that it provides a more comprehensive recognition capability in terms of reliably recognising various fine-grained human postures and transportation modes. In addition, by comparing the new FF+GPS method with both an accelerometer (ACC) method (62% accuracy) and an ACC+GPS based method (70% accuracy) as baseline methods, it obtains a higher accuracy (90%) with less computational complexity, when tested on a dataset obtained from ten individuals. In addition, the new FF+GPS method has been further extended and evaluated. More specifically, the trade-off between the computation and resources needed to support lower versus higher number of features and sensors has been investigated. The improved FF+GPS method reduced the number of classification features from 31 to 12, reduced the number of FF sensors from 8 to 4, and reduced the use of GPS in mobility activity recognition

Minimal Infrastructure Radio Frequency Home Localisation Systems

The ability to track the location of a subject in their home allows the provision of a number of location based services, such as remote activity monitoring, context sensitive prompts and detection of safety critical situations such as falls. Such pervasive monitoring functionality offers the potential for elders to live at home for longer periods of their lives with minimal human supervision. The focus of this thesis is on the investigation and development of a home roomlevel localisation technique which can be readily deployed in a realistic home environment with minimal hardware requirements. A conveniently deployed Bluetooth ® localisation platform is designed and experimentally validated throughout the thesis. The platform adopts the convenience of a mobile phone and the processing power of a remote location calculation computer. The use of Bluetooth ® also ensures the extensibility of the platform to other home health supervision scenarios such as wireless body sensor monitoring. Central contributions of this work include the comparison of probabilistic and nonprobabilistic classifiers for location prediction accuracy and the extension of probabilistic classifiers to a Hidden Markov Model Bayesian filtering framework. New location prediction performance metrics are developed and signicant performance improvements are demonstrated with the novel extension of Hidden Markov Models to higher-order Markov movement models. With the simple probabilistic classifiers, location is correctly predicted 80% of the time. This increases to 86% with the application of the Hidden Markov Models and 88% when high-order Hidden Markov Models are employed. Further novelty is exhibited in the derivation of a real-time Hidden Markov Model Viterbi decoding algorithm which presents all the advantages of the original algorithm, while producing location estimates in real-time. Significant contributions are also made to the field of human gait-recognition by applying Bayesian filtering to the task of motion detection from accelerometers which are already present in many mobile phones. Bayesian filtering is demonstrated to enable a 35% improvement in motion recognition rate and even enables a floor recognition rate of 68% using only accelerometers. The unique application of time-varying Hidden Markov Models demonstrates the effect of integrating these freely available motion predictions on long-term location predictions