New applications of late fusion methods for EEG signal processing

[EN] Decision fusion consists in the combination of the outputs of multiple classifiers into a common decision that is more precise or stable. In most cases, however, only classical fusion techniques are considered. This work compares the performance of several state-of-the-art fusion methods on new applications of automatic stage classification of several neuropsychological tests. The tests were staged into three classes: stimulus display, retention interval, and subject response. The considered late fusion methods were: alpha integration; copulas; Dempster-Shafer combination; independent component analysis mixture models; and behavior knowledge space. Late fusion was able to improve the performance for the task, with alpha integration yielding the most stable result.This work was supported by Generalitat Valenciana under grant PROMETEO/2019/109 and Spanish Administration and European Union grant TEC2017-84743-P.Safont, G.; Salazar Afanador, A.; Vergara Domínguez, L. (2019). New applications of late fusion methods for EEG signal processing. IEEE. 617-621. https://doi.org/10.1109/CSCI49370.2019.00116S61762

Automatic object classification for surveillance videos.

PhDThe recent popularity of surveillance video systems, specially located in urban scenarios, demands the development of visual techniques for monitoring purposes. A primary step towards intelligent surveillance video systems consists on automatic object classification, which still remains an open research problem and the keystone for the development of more specific applications. Typically, object representation is based on the inherent visual features. However, psychological studies have demonstrated that human beings can routinely categorise objects according to their behaviour. The existing gap in the understanding between the features automatically extracted by a computer, such as appearance-based features, and the concepts unconsciously perceived by human beings but unattainable for machines, or the behaviour features, is most commonly known as semantic gap. Consequently, this thesis proposes to narrow the semantic gap and bring together machine and human understanding towards object classification. Thus, a Surveillance Media Management is proposed to automatically detect and classify objects by analysing the physical properties inherent in their appearance (machine understanding) and the behaviour patterns which require a higher level of understanding (human understanding). Finally, a probabilistic multimodal fusion algorithm bridges the gap performing an automatic classification considering both machine and human understanding. The performance of the proposed Surveillance Media Management framework has been thoroughly evaluated on outdoor surveillance datasets. The experiments conducted demonstrated that the combination of machine and human understanding substantially enhanced the object classification performance. Finally, the inclusion of human reasoning and understanding provides the essential information to bridge the semantic gap towards smart surveillance video systems

CONFIDENCE-BASED DECISION-MAKING SUPPORT FOR MULTI-SENSOR SYSTEMS

We live in a world where computer systems are omnipresent and are connected to more and more sensors. Ranging from small individual electronic assistants like smartphones to complex autonomous robots, from personal wearable health devices to professional eHealth frameworks, all these systems use the sensors’ data in order to make appropriate decisions according to the context they measure. However, in addition to complete failures leading to the lack of data delivery, these sensors can also send bad data due to influences from the environment which can sometimes be hard to detect by the computer system when checking each sensor individually. The computer system should be able to use its set of sensors as a whole in order to mitigate the influence of malfunctioning sensors, to overcome the absence of data coming from broken sensors, and to handle possible conflicting information coming from several sensors. In this thesis, we propose a computational model based on a two layer software architecture to overcome this challenge. In a first layer, classification algorithms will check for malfunctioning sensors and attribute a confidence value to each sensor. In the second layer, a rule-based proactive engine will then build a representation of the context of the system and use it along some empirical knowledge about the weaknesses of the different sensors to further tweak this confidence value. Furthermore, the system will then check for conflicting data between sensors. This can be done by having several sensors that measure the same parameters or by having multiple sensors that can be used together to calculate an estimation of a parameter given by another sensor. A confidence value will be calculated for this estimation as well, based on the confidence values of the related sensors. The successive design refinement steps of our model are shown over the course of three experiments. The first two experiments, located in the eHealth domain, have been used to better identify the challenges of such multi-sensor systems, while the third experiment, which consists of a virtual robot simulation, acts as a proof of concept for the semi-generic model proposed in this thesis

Integration of discriminative and generative models for activity recognition in smart homes

Activity recognition in smart homes enables the remote monitoring of elderly and patients. In healthcare systems, reliability of a recognition model is of high importance. Limited amount of training data and imbalanced number of activity instances result in over-fitting thus making recognition models inconsistent. In this paper, we propose an activity recognition approach that integrates the distance minimization (DM) and probability estimation (PE) approaches to improve the reliability of recognitions. DM uses distances of instances from the mean representation of each activity class for label assignment. DM is useful in avoiding decision biasing towards the activity class with majority instances; however, DM can result in over-fitting. PE on the other hand has good generalization abilities. PE measures the probability of correct assignments from the obtained distances, while it requires a large amount of data for training. We apply data oversampling to improve the representation of classes with less number of instances. Support vector machine (SVM) is applied to combine the outputs of both DM and PE, since SVM performs better with imbalanced data and further improves the generalization ability of the approach. The proposed approach is evaluated using five publicly available smart home datasets. The results demonstrate better performance of the proposed approach compared to the state-of-the-art activity recognition approaches

Non-intrusive load monitoring techniques for activity of daily living recognition

Esta tesis nace con la motivación de afrontar dos grandes problemas de nuestra era: la falta de recursos energéticos y el envejecimiento de la población. Respecto al primer problema, nace en la primera década de este siglo el concepto de Smart Grids con el objetivo de alcanzar la eficiencia energética. Numerosos países comienzan a realizar despliegues masivos de contadores inteligentes ("Smart Meters"), lo que despierta el interés de investigadores que comienzan a desarrollar nuevas técnicas para predecir la demanda. Así, los sistemas NILM (Non-Intrusive Load Monitoring) tratan de predecir el consumo individual de los dispositivos conectados a partir de un único sensor: el contador inteligente. Por otra parte, los grandes avances en la medicina moderna han permitido que nuestra esperanza de vida aumente considerablemente. No obstante, esta longevidad, junto con la baja fertilidad en los países desarrollados, tiene un efecto secundario: el envejecimiento de la población. Unos de los grandes avances es la incorporación de la tecnología en la vida cotidiana, lo que ayuda a los más mayores a llevar una vida independiente. El despliegue de una red de sensores dentro de la vivienda permite su monitorización y asistencia en las tareas cotidianas. Sin embargo, son intrusivos, no escalables y, en algunas ocasiones, de alto coste, por lo que no están preparados para hacer frente al incremento de la demanda de esta comunidad. Esta tesis doctoral nace de la motivación de afrontar estos problemas y tiene dos objetivos principales: lograr un modelo de monitorización sostenible para personas mayores y, a su vez, dar un valor añadido a los sistemas NILM que despierte el interés del usuario final. Con este objetivo, se presentan nuevas técnicas de monitorización basadas en NILM, aunando lo mejor de ambos campos. Esto supone un ahorro considerable de recursos en la monitorización, ya que únicamente se necesita un sensor: el contador inteligente; lo cual da escalabilidad a estos sistemas. Las contribuciones de esta tesis se dividen en dos bloques principales. En el primero se proponen nuevas técnicas NILM optimizadas para la detección de la actividad humana. Así, se desarrolla una propuesta basada en detección de eventos (conexiones de dispositivos) en tiempo real y su clasificación a un dispositivo. Con el objetivo de que pueda integrarse en contadores inteligentes. Cabe destacar que el clasificador se basa en modelos generalizados de dispositivos y no necesita conocimiento específico de la vivienda. El segundo bloque presenta tres nuevas técnicas de monitorización de personas mayores basadas en NILM. El objetivo es proporcionar una monitorización básica pero eficiente y altamente escalable, ahorrando en recursos. Los procesos Cox, log Gaussian Cox Processes (LGCP), monitorizan un único dispositivo si la rutina está estrechamente ligada a este. Así, se propone un sistema de alarmas si se detectan cambios en el comportamiento. LGCP tiene la ventaja de poder modelar periodicidades e incertidumbres propias del comportamiento humano. Cuando la rutina no depende de un único dispositivo, se proponen dos técnicas: una basada en gaussianas mixtas, Gaussian Mixture Models (GMM); y la otra basada en la Teoría de la Evidencia de Dempster-Shafer (DST). Ambas monitorizan y detectan deterioros en la actividad, causados por enfermedades como la demencia y el alzhéimer. Únicamente DST usa incertidumbres que simulan mejor el comportamiento humano y, por tanto, permite alarmas en caso de un repentino desvío. Finalmente, todas las propuestas han sido validadas mediante la evaluación de métricas y la obtención de resultados experimentales. Para ello, se han usado medidas de escenarios reales que han sido recopiladas en bases de datos. Los resultados obtenidos han sido satisfactorios, demostrando que este tipo de monitorización es posible y muy beneficioso para nuestra sociedad. Además, se ha dado a lugar nuevas propuestas que serán desarrolladas en el futuro. Códigos UNESCO: 120320 - sistemas de control medico, 332201 – distribución de la energía, 120701 – análisis de actividades, 120304 – inteligencia artificial, 120807 – plausibilidad, 221402 – patrones

From data acquisition to data fusion : a comprehensive review and a roadmap for the identification of activities of daily living using mobile devices

This paper focuses on the research on the state of the art for sensor fusion techniques, applied to the sensors embedded in mobile devices, as a means to help identify the mobile device user’s daily activities. Sensor data fusion techniques are used to consolidate the data collected from several sensors, increasing the reliability of the algorithms for the identification of the different activities. However, mobile devices have several constraints, e.g., low memory, low battery life and low processing power, and some data fusion techniques are not suited to this scenario. The main purpose of this paper is to present an overview of the state of the art to identify examples of sensor data fusion techniques that can be applied to the sensors available in mobile devices aiming to identify activities of daily living (ADLs)