Human Activity Recognition using a Semantic Ontology-Based Framework

In the last years, the extensive use of smart objects embedded in the physical world, in order to monitor and record physical or environmental conditions, has increased rapidly. In this scenario, heterogeneous devices are connected together into a network. Data generated from such system are usually stored in a database, which often shows a lack of semantic information and relationship among devices. Moreover, this set can be incomplete, unreliable, incorrect and noisy. So, it turns out to be important both the integration of information and the interoperability of applications. For this reason, ontologies are becoming widely used to describe the domain and achieve efficient interoperability of information system. An example of the described situation could be represented by Ambient Assisted Living context, which intends to enable older or disabled people to remain living independently longer in their own house. In this contest, human activity recognition plays a main role because it could be considered as starting point to facilitate assistance and care for elderly. Due to the nature of human behavior, it is necessary to manage the time and spatial restrictions. So, we propose a framework that implements a novel methodology based on the integration of an ontology for representing contextual knowledge and a Complex Event Processing engine for supporting timed reasoning. Moreover, it is an infrastructure where knowledge, organized in conceptual spaces (based on its meaning) can be semantically queried, discovered, and shared across applications. In our framework, benefits deriving from the implementation of a domain ontology are exploited into different levels of abstrac- tion. Thereafter, reasoning techniques represent a preprocessing method to prepare data for the final temporal analysis. The results, presented in this paper, have been obtained applying the methodology into AALISABETH, an Ambient Assisted Living project aimed to monitor the lifestyle of old people, not suffering from major chronic diseases or severe disabilities

Towards a cascading reasoning framework to support responsive ambient-intelligent healthcare interventions

In hospitals and smart nursing homes, ambient-intelligent care rooms are equipped with many sensors. They can monitor environmental and body parameters, and detect wearable devices of patients and nurses. Hence, they continuously produce data streams. This offers the opportunity to collect, integrate and interpret this data in a context-aware manner, with a focus on reactivity and autonomy. However, doing this in real time on huge data streams is a challenging task. In this context, cascading reasoning is an emerging research approach that exploits the trade-off between reasoning complexity and data velocity by constructing a processing hierarchy of reasoners. Therefore, a cascading reasoning framework is proposed in this paper. A generic architecture is presented allowing to create a pipeline of reasoning components hosted locally, in the edge of the network, and in the cloud. The architecture is implemented on a pervasive health use case, where medically diagnosed patients are constantly monitored, and alarming situations can be detected and reacted upon in a context-aware manner. A performance evaluation shows that the total system latency is mostly lower than 5 s, allowing for responsive intervention by a nurse in alarming situations. Using the evaluation results, the benefits of cascading reasoning for healthcare are analyzed

Software Architecture Trends and Promising Technology for Ambient Assisted Living Systems

Driven by the ongoing demographical, structural, and social changes in all modern, industrialized countries, there is a huge interest in IT-based equipment and services these days that enable independent living of people with specific needs. Despite of promising concepts, approaches and technology, those systems are still rather a vision than reality. In order to pave the way towards a common understanding of the problem and overall software solution approaches, this paper (i) characterizes the Ambient Assisted Living domain, (ii) briefly presents relevant software architecture trends, esp. applicable styles and patterns and (iii) discusses promising software technology already available to solve the problems

Semantic segmentation of real-time sensor data stream for complex activity recognition

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Data segmentation plays a critical role in performing human activity recognition in the ambient assistant living systems. It is particularly important for complex activity recognition when the events occur in short bursts with attributes of multiple sub-tasks. Although substantial efforts have been made in segmenting the real-time sensor data stream such as static/dynamic window sizing approaches, little has been explored to exploit object semantic for discerning sensor data into multiple threads of activity of daily living. This paper proposes a semantic-based approach for segmenting sensor data series using ontologies to perform terminology box and assertion box reasoning, along with logical rules to infer whether the incoming sensor event is related to a given sequences of the activity. The proposed approach is illustrated using a use-case scenario which conducts semantic segmentation of a real-time sensor data stream to recognise an elderly persons complex activities

Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

Context Aware Middleware Architectures: Survey and Challenges

Abstract: Context aware applications, which can adapt their behaviors to changing environments, are attracting more and more attention. To simplify the complexity of developing applications, context aware middleware, which introduces context awareness into the traditional middleware, is highlighted to provide a homogeneous interface involving generic context management solutions. This paper provides a survey of state-of-the-art context aware middleware architectures proposed during the period from 2009 through 2015. First, a preliminary background, such as the principles of context, context awareness, context modelling, and context reasoning, is provided for a comprehensive understanding of context aware middleware. On this basis, an overview of eleven carefully selected middleware architectures is presented and their main features explained. Then, thorough comparisons and analysis of the presented middleware architectures are performed based on technical parameters including architectural style, context abstraction, context reasoning, scalability, fault tolerance, interoperability, service discovery, storage, security & privacy, context awareness level, and cloud-based big data analytics. The analysis shows that there is actually no context aware middleware architecture that complies with all requirements. Finally, challenges are pointed out as open issues for future work

RESTful framework for collaborative internet of things based on IEC 61850

El contenido de los capítulos 2 y 3 está sujeto a confidencialidad 161 p.En 1991, Mark Weiser formuló el paradigma de Computación Ubicua definiendo el concepto de Entorno Inteligente como un espacio físico repleto de dispositivos, muy integrados en el entorno, y con capacidades de identificación, sensorización y actuación. Internet de las Cosas (IoT) expande el ámbito de localización de estos dispositivos y servicios ubicuos, representados como cosas, de un entorno local a internet como red global. Para la implementación de estos escenarios de aplicación, la colaboración entre las cosas es uno de los principales retos de investigación. El objetivo de esta colaboración es ser capaces de satisfacer necesidades globales mediante la combinación de servicios individuales. Esta Tesis propone una arquitectura colaborativa entre las cosas desplegadas en internet.Las tecnologías alrededor de los Servicios Web SOAP/XML, adecuadas para IoT, soportan aspectos claves para un sistema colaborativo como la publicación, descubrimiento, control y gestión de eventos de los dispositivos. Como alternativa, REST ha ganado terreno en este ámbito por ser considerada una opción más ligera, sencilla y natural para la comunicación en internet. Sin embargo, no existen protocolos para descubrimiento y gestión de eventos para recursos REST. Esta Tesis aborda dicha carencia proponiendo una especificación de estos protocolos para arquitecturas REST. Otro aspecto importante es la representación, a nivel de aplicación, de las cosas distribuidas. Entre las propuestas para la estandarización de los modelos de información y comunicación en este dominio que podrían aplicarse, de manera similar, a IoT, destaca el estándar IEC 61850. Sin embargo, los protocolos de comunicación definidos por el estándar no son adecuados para IoT. Esta Tesis analiza la idoneidad del IEC 61850 para escenarios IoT y propone un protocolo de comunicación REST para sus servicios.Por último, se trata la problemática asociada a la confiabilidad que debe proporcionar una arquitectura IoT para dominios de aplicación relacionados con la salud o sistemas de seguridad funcional (Safety)

Cognitive assisted living ambient system: a survey

The demographic change towards an aging population is creating a significant impact and introducing drastic challenges to our society. We therefore need to find ways to assist older people to stay independently and prevent social isolation of these population. Information and Communication Technologies (ICT) provide various solutions to help older adults to improve their quality of life, stay healthier, and live independently for a time. Ambient Assisted Living (AAL) is a field to investigate innovative technologies to provide assistance as well as healthcare and rehabilitation to impaired seniors. The paper provides a review of research background and technologies of AAL

Federation of AAL & AHA systems through semantically interoperable framework

[EN] Ambient Assisted Living (AAL) and Active and Healthy Ageing (AHA) immensely benefit from IoT application. The federation of IoT platforms can multiply the benefits obtained by the operation of those systems in an isolated way, as it enables important synergies (e.g., intelligent information sharing, system cooperation, service enhancement). This federation requires the enablement of interoperability between the IoT systems, which represents a major challenge, as systems typically follow very different standards, data formats, semantic models and manners of representing the information. We have provided a technical solution in the frame of ACTIVAGE, a project that aims to federate multiple heterogeneous IoT platforms and systems associated to clusters of AHA Smart Homes in 12 regions across Europe, with the goal to improve the AHA service provided and create the first European AHA ecosystem. Our technical solution allows the enablement of full semantic interoperability across heterogeneous platforms and it has been validated in a test scenario. It enables significant AHA service enhancement within the ACTIVAGE ecosystem, as native applications from one platform could be used indistinctly by all federated platforms. Our solution allows good scalability federating new platforms, with linear and relatively low effort.This research work has been partially funded by LSP H2020 ACTIVAGE project under Grant Agreement Nº 732679.González-Usach, R.; Julián, M.; Esteve Domingo, M.; Palau Salvador, CE. (2021). Federation of AAL & AHA systems through semantically interoperable framework. 1-6. https://doi.org/10.1109/ICCWorkshops50388.2021.94735031