A Probabilistic Approach to Handle Missing Data for Multi-Sensory Activity Recognition

Context and activity recognition in complex scenarios is prone to data loss due to disconnections, sensor failure, transmission problems, etc. This generally implies significant changes in the recognition performance. In the case of classifier fusion faulty sensors can be removed from the recognition chain to overcome this issue. Alternatively, we can try to compensate or impute data to replace the missing signals. In this paper we proposed a probabilistic method for imputation of missing data. The proposed method is based on conditional Gaussian distribution and has been previously applied in other fields, such as speech recognition and bioinformatics, but not in for activity recognition. Our method exploits the correlation among classifier outputs to infer missing values of decision profile from available values in a probabilistic manner. We assess the method performance using two datasets in a car manufacturing and in a daily activities scenario with three different configuration of sensors. Results show the advantages of the probabilistic estimation over other common methods such as removing and clustering. The method is also applicable in other classification problems which uses fusion methods to combine decisions of classifiers

Ensemble creation and reconfiguration for activity recognition: An information theoretic approach.

Technological advances in sensing and portable computing devices and wireless communication has lead to an increase in the number and variety of sensing enabled devices (e.g. smartphones or sensing garments). Pervasive computing and activity recognition systems should be able to take advantage of these sensors, even if they are not always available, or appear during the system operation. These sensors can be integrated into an ensemble where information from each sensor is then fused to obtain the system decisions. There is therefore a need for mechanisms to select which sensors should compose the ensemble, as well as techniques for dynamically reconfigure the ensemble so as to integrate new sensors. From the machine learning point of view, this approach corresponds to the combination of classifiers where measures of the accuracy and diversity of the ensemble are used to select the elements that may lead to the highest performance. Recent works have proposed measures of accuracy and diversity based on an information theoretical approach. In this paper we study the use of these measures for selecting ensembles in activity recognition based on body sensor networks. In addition to compare the obtained performance with traditional diversity measures (e.g., Q-, κ-statistics) we also present mechanisms to exploit these measures for the dynamic reconfiguration of the ensemble and detection of changes in the network (e.g. due to sensor noise or malfunction)

On-line anomaly detection and resilience in classifier ensembles

Detection of anomalies is a broad field of study, which is applied in different areas such as data monitoring, navigation, and pattern recognition. In this paper we propose two measures to detect anomalous behaviors in an ensemble of classifiers by monitoring their decisions; one based on Mahalanobis distance and another based on information theory. These approaches are useful when an ensemble of classifiers is used and a decision is made by ordinary classifier fusion methods, while each classifier is devoted to monitor part of the environment. Upon detection of anomalous classifiers we propose a strategy that attempts to minimize adverse effects of faulty classifiers by excluding them from the ensemble. We applied this method to an artificial dataset and sensor-based human activity datasets, with different sensor configurations and two types of noise (additive and rotational on inertial sensors). We compared our method with two other well-known approaches, generalized likelihood ratio (GLR) and One-Class Support Vector Machine (OCSVM), which detect anomalies at data/feature level. We found that our method is comparable with GLR and OCSVM. The advantages of our method compared to them is that it avoids monitoring raw data or features and only takes into account the decisions that are made by their classifiers, therefore it is independent of sensor modality and nature of anomaly. On the other hand, we found that OCSVM is very sensitive to the chosen parameters and furthermore in different types of anomalies it may react differently. In this paper we discuss the application domains which benefit from our method

Quantifying electrode reliability during Brain-Computer Interface operation

One of the problems of non-invasive Brain-Computer Interface (BCI) applications is the occurrence of anomalous (unexpected) signals that might degrade BCI performance. This situation might slip the operator’s attention since raw signals are not usually continuously visualized and monitored during BCI-actuated device operation. Anomalous data can for instance be the result of electrode misplacement, degrading impedance or loss of connectivity. Since this problem can develop at run-time, there is a need of a systematic approach to evaluate electrode reliability during online BCI operation. In this paper, we propose two metrics detecting how much each channel is deviating from its expected behavior. This quantifies electrode reliability at run-time which could be embedded into BCI data processing to increase performance. We manifest the effectiveness of these metrics in quantifying signal degradation by conducting three experiments: electrode swap, electrode manipulation and offline artificially degradation of P300 signals

A hybrid BCI based on the fusion of EEG and EMG activities

Hybrid Brain-Computer Interfaces (BCI) are representing a recent approach to develop practical BCIs. In such a system disabled users are able to use all their remaining functionalities as control possibilities in parallel with the BCI. Sometimes these people have residual activity of their muscles. Therefore, in the presented hybrid BCI framework we want to explore the parallel usage of electroencephalographic (EEG) and electromyographic (EMG) activity, whereby the control abilities of both channels are fused. Results showed that the participants could achieve a good control of their hybrid BCI independently of their level of muscular fatigue. Thereby the multimodal fusion approach of muscular and brain activity yielded better and more stable performance compared to the single conditions. Even in the case of an increasing muscular fatigue a good control (moderate and graceful degradation of the performance compared to the non-fatigued case) and a smooth handover could be achieved. Therefore, such systems allow the users a very reliable hybrid BCI control although they are getting more and more exhausted or fatigued during the day

The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition

There is a growing interest on using ambient and wearable sensors for human activity recognition, fostered by several application domains and wider availability of sensing technologies. This has triggered increasing attention on the development of robust machine learning techniques that exploits multimodal sensor setups. However, unlike other applications, there are no established benchmarking problems for this field. As a matter of fact, methods are usually tested on custom datasets acquired in very specific experimental setups. Furthermore, data is seldom shared between different groups. Our goal is to address this issue by introducing a versatile human activity dataset recorded in a sensor-rich environment. This database was the basis of an open challenge on activity recognition. We report here the outcome of this challenge, as well as baseline performance using different classification techniques. We expect this benchmarking database will motivate other researchers to replicate and outperform the presented results, thus contributing to further advances in the state-of-the-art of activity recognition methods

Benchmarking classification techniques using the Opportunity human activity dataset

Human activity recognition is a thriving research field. There are lots of studies in different sub-areas of activity recognition proposing different methods. However, unlike other applications, there is lack of established benchmarking problems for activity recognition. Typically, each research group tests and reports the performance of their algorithms on their own datasets using experimental setups specially conceived for that specific purpose. In this work, we introduce a versatile human activity dataset conceived to fill that void. We illustrate its use by presenting comparative results of different classification techniques, and discuss about several metrics that can be used to assess their performance. Being an initial benchmarking, we expect that the possibility to replicate and outperform the presented results will contribute to further advances in state-of-the-art methods