An Unsupervised Approach for Automatic Activity Recognition based on Hidden Markov Model Regression

Using supervised machine learning approaches to recognize human activities from on-body wearable accelerometers generally requires a large amount of labelled data. When ground truth information is not available, too expensive, time consuming or difficult to collect, one has to rely on unsupervised approaches. This paper presents a new unsupervised approach for human activity recognition from raw acceleration data measured using inertial wearable sensors. The proposed method is based upon joint segmentation of multidimensional time series using a Hidden Markov Model (HMM) in a multiple regression context. The model is learned in an unsupervised framework using the Expectation-Maximization (EM) algorithm where no activity labels are needed. The proposed method takes into account the sequential appearance of the data. It is therefore adapted for the temporal acceleration data to accurately detect the activities. It allows both segmentation and classification of the human activities. Experimental results are provided to demonstrate the efficiency of the proposed approach with respect to standard supervised and unsupervised classification approache

Towards Smart Homes Using Low Level Sensory Data

Ubiquitous Life Care (u-Life care) is receiving attention because it provides high quality and low cost care services. To provide spontaneous and robust healthcare services, knowledge of a patient’s real-time daily life activities is required. Context information with real-time daily life activities can help to provide better services and to improve healthcare delivery. The performance and accuracy of existing life care systems is not reliable, even with a limited number of services. This paper presents a Human Activity Recognition Engine (HARE) that monitors human health as well as activities using heterogeneous sensor technology and processes these activities intelligently on a Cloud platform for providing improved care at low cost. We focus on activity recognition using video-based, wearable sensor-based, and location-based activity recognition engines and then use intelligent processing to analyze the context of the activities performed. The experimental results of all the components showed good accuracy against existing techniques. The system is deployed on Cloud for Alzheimer’s disease patients (as a case study) with four activity recognition engines to identify low level activity from the raw data captured by sensors. These are then manipulated using ontology to infer higher level activities and make decisions about a patient’s activity using patient profile information and customized rules

Sensor-based activity recognition with dynamically added context

An activity recognition system essentially processes raw sensor data and maps them into latent activity classes. Most of the previous systems are built with supervised learning techniques and pre-defined data sources, and result in static models. However, in realistic and dynamic environments, original data sources may fail and new data sources become available, a robust activity recognition system should be able to perform evolution automatically with dynamic sensor availability in dynamic environments. In this paper, we propose methods that automatically incorporate dynamically available data sources to adapt and refine the recognition system at run-time. The system is built upon ensemble classifiers which can automatically choose the features with the most discriminative power. Extensive experimental results with publicly available datasets demonstrate the effectiveness of our methods

Markov modelling on human activity recognition

Human Activity Recognition (HAR) is a research topic with a relevant interest in the machine learning community. Understanding the activities that a person is performing and the context where they perform them has a huge importance in multiple applications, including medical research, security or patient monitoring. The improvement of the smart-phones and inertial sensors technologies has lead to the implementation of activity recognition systems based on these devices, either by themselves or combining their information with other sensors. Since humans perform their daily activities sequentially in a specific order, there exist some temporal information in the physical activities that characterize the different human behaviour patterns. However, the most popular approach in HAR is to assume that the data is conditionally independent, segmenting the data in different windows and extracting the most relevant features from each segment. In this thesis we employ the temporal information explicitly, where the raw data provided by the wearable sensors is fed to the training models. Thus, we study how to perform a Markov modelling implementation of a long-term monitoring HAR system with wearable sensors, and we address the existing open problems arising while processing and training the data, combining different sensors and performing the long-term monitoring with battery powered devices. Employing directly the signals from the sensors to perform the recognition can lead to problems due to misplacements of the sensors on the body. We propose an orientation correction algorithm based on quaternions to process the signals and find a common frame reference for all of them independently on the position of the sensors or their orientation. This algorithm allows for a better activity recognition when feed to the classification algorithm when compared with similar approaches, and the quaternion transformations allow for a faster implementation. One of the most popular algorithms to model time series data are Hidden Markov Models (HMMs) and the training of the parameters of the model is performed using the Baum-Welch algorithm. However, this algorithm converges to local maxima and the multiple initializations needed to avoid them makes it computationally expensive for large datasets. We propose employing the theory of spectral learning to develop a discriminative HMM that avoids the problems of the Baum-Welch algorithm, outperforming it in both complexity and computational cost. When we implement a HAR system with several sensors, we need to consider how to perform the combination of the information provided by them. Data fusion can be performed either at signal level or at classification level. When performed at classification level, the usual approach is to combine the decisions of multiple classifiers on the body to obtain the performed activities. However, in the simple case with two classifiers, which can be a practical implementation of a HAR system, the combination reduces to selecting the most discriminative sensor, and no performance improvement is obtained against the single sensor implementation. In this thesis, we propose to employ the soft-outputs of the classifiers in the combination and we develop a method that considers the Markovian structure of the ground truth to capture the dynamics of the activities. We will show that this method improves the recognition of the activities with respect to other combination methods and with respect to the signal fusion case. Finally, in long-term monitoring HAR systems with wearable sensors we need to address the energy efficiency problem that is inherent to battery powered devices. The most common approach to improve the energy efficiency of such devices is to reduce the amount of data acquired by the wearable sensors. In that sense, we introduce a general framework for the energy efficiency of a system with multiple sensors under several energy restrictions. We propose a sensing strategy to optimize the temporal data acquisition based on computing the uncertainty of the activities given the data and adapt the acquisition actively. Furthermore, we develop a sensor selection algorithm based on Bayesian Experimental Design to obtain the best configuration of sensors that performs the activity recognition accurately, allowing for a further improvement on the energy efficiency by limiting the number of sensors employed in the acquisition.El reconocimiento de actividades humanas (HAR) es un tema de investigación con una gran relevancia para la comunidad de aprendizaje máquina. Comprender las actividades que una persona está realizando y el contexto en el que las realiza es de gran importancia en multitud de aplicaciones, entre las que se incluyen investigación médica, seguridad o monitorización de pacientes. La mejora en los smart-phones y en las tecnologías de sensores inerciales han dado lugar a la implementación de sistemas de reconocimiento de actividades basado en dichos dispositivos, ya sea por si mismos o combinándolos con otro tipo de sensores. Ya que los seres humanos realizan sus actividades diarias de manera secuencial en un orden específico, existe una cierta información temporal en las actividades físicas que caracterizan los diferentes patrones de comportamiento, Sin embargo, los algoritmos más comunes asumen que los datos son condicionalmente independientes, segmentándolos en diferentes ventanas y extrayendo las características más relevantes de cada segmento. En esta tesis utilizamos la información temporal de manera explícita, usando los datos crudos de los sensores como entrada de los modelos de entrenamiento. Por ello, analizamos como implementar modelos Markovianos para el reconocimiento de actividades en monitorizaciones de larga duración con sensores wearable, y tratamos los problemas existentes al procesar y entrenar los datos, al combinar diferentes sensores y al realizar adquisiciones de larga duración con dispositivos alimentados por baterías. Emplear directamente las señales de los sensores para realizar el reconocimiento de actividades puede dar lugar a problemas debido a la incorrecta colocación de los sensores en el cuerpo. Proponemos un algoritmo de corrección de la orientación basado en quaterniones para procesar las señales y encontrar un marco de referencia común independiente de la posición de los sensores y su orientación. Este algoritmo permite obtener un mejor reconocimiento de actividades al emplearlo en conjunto con un algoritmo de clasificación, cuando se compara con modelos similares. Además, la transformación de la orientación basada en quaterniones da lugar a una implementación más rápida. Uno de los algoritmos más populares para modelar series temporales son los modelos ocultos de Markov, donde los parámetros del modelo se entrenan usando el algoritmo de Baum-Welch. Sin embargo, este algoritmo converge en general a máximos locales, y las múltiples inicializaciones que se necesitan en su implementación lo convierten en un algoritmo de gran carga computacional cuando se emplea con bases de datos de un volumen considerable. Proponemos emplear la teoría de aprendizaje espectral para desarrollar un HMM discriminativo que evita los problemas del algoritmo de Baum-Welch, superándolo tanto en complejidad como en coste computacional. Cuando se implementa un sistema de reconocimiento de actividades con múltiples sensores, necesitamos considerar cómo realizar la combinación de la información que proporcionan. La fusión de los datos, se puede realizar tanto a nivel de señal como a nivel de clasificación. Cuando se realiza a nivel de clasificación, lo normal es combinar las decisiones de múltiples clasificadores colocados en el cuerpo para obtener las actividades que se están realizando. Sin embargo, en un caso simple donde únicamente se emplean dos sensores, que podría ser una implantación habitual de un sistema de reconocimiento de actividades, la combinación se reduce a seleccionar el sensor más discriminativo, y no se obtiene mejora con respecto a emplear un único sensor. En esta tesis proponemos emplear salidas blandas de los clasificadores para la combinación, desarrollando un modelo que considera la estructura Markoviana de los datos reales para capturar la dinámica de las actividades. Mostraremos como este método mejora el reconocimiento de actividades con respecto a otros métodos de combinación de clasificadores y con respecto a la fusión de los datos a nivel de señal. Por último, abordamos el problema de la eficiencia energética de dispositivos alimentados por baterías en sistemas de reconocimiento de actividades de larga duración. La aproximación más habitual para mejorar la eficiencia energética consiste en reducir el volumen de datos que adquieren los sensores. En ese sentido, introducimos un marco general para tratar el problema de la eficiencia energética en un sistema con múltiples sensores bajo ciertas restricciones de energética. Proponemos una estrategia de adquisición activa para optimizar el sistema temporal de recogida de datos, basándonos en la incertidumbre de las actividades dados los datos que conocemos. Además, desarrollamos un algoritmo de selección de sensores basado diseño experimental Bayesiano y así obtener la mejor configuración para realizar el reconocimiento de actividades limitando el número de sensores empleados y al mismo tiempo reduciendo su consumo energético.Programa Oficial de Doctorado en Multimedia y ComunicacionesPresidente: Luis Ignacio Santamaría Caballero.- Secretario: Pablo Martínez Olmos.- Vocal: Alberto Suárez Gonzále

Context-awareness for mobile sensing: a survey and future directions

The evolution of smartphones together with increasing computational power have empowered developers to create innovative context-aware applications for recognizing user related social and cognitive activities in any situation and at any location. The existence and awareness of the context provides the capability of being conscious of physical environments or situations around mobile device users. This allows network services to respond proactively and intelligently based on such awareness. The key idea behind context-aware applications is to encourage users to collect, analyze and share local sensory knowledge in the purpose for a large scale community use by creating a smart network. The desired network is capable of making autonomous logical decisions to actuate environmental objects, and also assist individuals. However, many open challenges remain, which are mostly arisen due to the middleware services provided in mobile devices have limited resources in terms of power, memory and bandwidth. Thus, it becomes critically important to study how the drawbacks can be elaborated and resolved, and at the same time better understand the opportunities for the research community to contribute to the context-awareness. To this end, this paper surveys the literature over the period of 1991-2014 from the emerging concepts to applications of context-awareness in mobile platforms by providing up-to-date research and future research directions. Moreover, it points out the challenges faced in this regard and enlighten them by proposing possible solutions

Physical Activity Classification with Conditional Random Fields

In this thesis we develop methods for classifying physical activity using accelerometer recordings. We cast this as a problem of classification in time series with moderate to high dimensional observations at each time point. Specifically, we observe a vector of summary statistics of the accelerometer signal at each point in time, and we wish to use these observations to estimate the type and intensity of physical activity the individual engaged in as it changes over time. Our methods are based on Conditional Random Fields, which allow us to capture temporal dependence in an individual’s physical activity type without requiring us to model the distribution of the observed features at each point in time. We develop three novel estimation strategies for Conditional Random Fields, evaluate their performance on classification tasks through simulation studies and demonstrate their use in applications with real physical activity data sets