Handling Real-World Context Awareness, Uncertainty and Vagueness in Real-Time Human Activity Tracking and Recognition with a Fuzzy Ontology-Based Hybrid Method

Human activity recognition is a key task in ambient intelligence applications to achieve proper ambient assisted living. There has been remarkable progress in this domain, but some challenges still remain to obtain robust methods. Our goal in this work is to provide a system that allows the modeling and recognition of a set of complex activities in real life scenarios involving interaction with the environment. The proposed framework is a hybrid model that comprises two main modules: a low level sub-activity recognizer, based on data-driven methods, and a high-level activity recognizer, implemented with a fuzzy ontology to include the semantic interpretation of actions performed by users. The fuzzy ontology is fed by the sub-activities recognized by the low level data-driven component and provides fuzzy ontological reasoning to recognize both the activities and their influence in the environment with semantics. An additional benefit of the approach is the ability to handle vagueness and uncertainty in the knowledge-based module, which substantially outperforms the treatment of incomplete and/or imprecise data with respect to classic crisp ontologies. We validate these advantages with the public CAD-120 dataset (Cornell Activity Dataset), achieving an accuracy of 90.1% and 91.07% for low-level and high-level activities, respectively. This entails an improvement over fully data-driven or ontology-based approaches.This work was funded by TUCS (Turku Centre for Computer Science), Finnish Cultural Foundation, Nokia Foundation, Google Anita Borg Scholarship, CEI BioTIC Project CEI2013-P-3, Contrato-Programa of Faculty of Education, Economy and Technology of Ceuta and Project TIN2012-30939 from National I+D Research Program (Spain). We also thank Fernando Bobillo for his support with FuzzyOWL and FuzzyDL tools

Interaction Analysis in Smart Work Environments through Fuzzy Temporal Logic

Interaction analysis is defined as the generation of situation descriptions from machine perception. World models created through machine perception are used by a reasoning engine based on fuzzy metric temporal logic and situation graph trees, with optional parameter learning and clustering as preprocessing, to deduce knowledge about the observed scene. The system is evaluated in a case study on automatic behavior report generation for staff training purposes in crisis response control rooms

A new fuzzy ontology development methodology (FODM) proposal

There is an upsurge in applying fuzzy ontologies to represent vague information in the knowledge representation field. Current research in the fuzzy ontologies paradigm mainly focuses on developing formalism languages to represent fuzzy ontologies, designing fuzzy ontology editors, and building fuzzy ontology applications in different domains. Less focus falls on establishing a formal methodological approach for building fuzzy ontologies. Existing fuzzy ontology development methodologies, such as the IKARUS-Onto methodology and Fuzzy Ontomethodology, provide formalized schedules for the conversion from crisp ontologies into fuzzy ones. However, a formal guidance on how to build fuzzy ontologies from scratch still lacks in current research. Therefore, this paper presents the first methodology, named FODM, for developing fuzzy ontologies from scratch. The proposed FODM can provide a very good guideline for formally constructing fuzzy ontologies in terms of completeness, comprehensiveness, generality, efficiency, and accuracy. To explain how the FODM works and demonstrate its usefulness, a fuzzy seabed characterization ontology is built based on the FODM and described step-by-step

Interaction Analysis in Smart Work Environments through Fuzzy Temporal Logic

Interaction analysis is defined as the generation of situation descriptions from machine perception. World models created through machine perception are used by a reasoning engine based on fuzzy metric temporal logic and situation graph trees, with optional parameter learning and clustering as preprocessing, to deduce knowledge about the observed scene. The system is evaluated in a case study on automatic behavior report generation for staff training purposes in crisis response control rooms

Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

From fuzzy to annotated semantic web languages

The aim of this chapter is to present a detailed, selfcontained and comprehensive account of the state of the art in representing and reasoning with fuzzy knowledge in Semantic Web Languages such as triple languages RDF/RDFS, conceptual languages of the OWL 2 family and rule languages. We further show how one may generalise them to so-called annotation domains, that cover also e.g. temporal and provenance extensions

Information Fusion Methodology for Enhancing Situation Awareness in Connected Cars Environment

This dissertation introduces novel approaches to develop a comprehensive model to address situation awareness in the Internet of Cars, called Attention Assist Framework (AAF). The proposed framework utilizes both Low-Level Data Fusion (LLDF), and High-Level Information Fusion (HLIF) to implement traffic entity, situation, and impact assessment, as well as decision making. The Internet of Cars is the convergence of the Internet of Things and Vehicular Ad-hoc Networks (VANETs). In fact, VANETs are the communication platforms that make possible the implementation of the Internet of Cars, and has become an integral part of this research field due to its major role to improve vehicle and road safety, traffic efficiency, and convenience as well as comfort to both drivers and passengers. Significant amount of VANETs research work has been focused on specific areas such as safety, routing, broadcasting, Quality of Service (QoS), and security. Among them, road safety issues are deemed one of the most challenging problems of VANETs. Specifically, lack of proper situational awareness of drivers has been shown to be the main cause of road accidents which makes it a major factor in road safety. The traffic entity assessment relies on a LLDF framework that is able to incorporate various multi-sensor data fusion approaches with means of communication links in VANETs. This is used to implement a cooperative localization approach through fusing common data fusion methods, such as Extended Kalman Filter (EKF) and Unscented Transform (UT), and vehicle-to-vehicle communication in VANETs. Furthermore, traffic situation assessment is based on a fuzzy extension to the Multi-Entity Bayesian Networks (MEBNs), which exploit the expressiveness of first-order logic for semantic relations, and the strength of the Fuzzy Bayesian Networks in handling uncertainty, while tackling the inherent vagueness in the soft data created by human entities. Finally, the impact assessment and decision making is realized through incorporating notions of game theory into Fuzzy-MEBNs, and introducing Active Fuzzy-MEBN (ATFY-MEBN), which is capable in hypothesizing future situations by assessing the impact of the current situation upon taking the actions indicated by an optimal strategy. In fact, such strategies are achieved through solving the games that are generated through a novel situation-specific normal form games generation algorithm that aims to create games based on the given context. In general, ATFY-MEBN presents the concepts of players and actions, and includes new game components, along with a 2-tier architecture, to efficiently model impact assessment and decision making. To demonstrate the capabilities of the proposed framework, a collision warning system simulator is developed, which evaluates the likelihood of a vehicle being in a near-collision situation using a wide variety of both local and global information sources available in the VANETs environment, and suggests an optimal action by assessing the impact of the current situation through generating and solving situation-specific games. Accordingly, first, the entities that highly influence the safety aspect, as well as both their casual and semantic relationships are identified. Next, an ATFY-MEBN-based model is presented, which allows for modeling these entities along with their relationships in specific contexts, assessing the current states of the situations of interest, predicting their future states, and finally suggesting optimal decision. Therefore, if the likelihood of being in a near-collision situation is determined to be high, and if the relevant situation-specific game is generated, then the impact of deciding on different combinations of actions that the game players take are calculated through a pre-fixed payoff function. Finally, the completed game is solved by finding its dominant strategy, that subsequently, results in proposing the optimal action to the driver. Our experimental results are divided into three main sections, through which we evaluate the capabilities of the traffic entity, situation, and impact assessment methods. Accordingly, the performance of the proposed cooperative localization approach is assessed by comparing its results with the ground truth solution and that of the other localization methods in various driving test cases. Moreover, two distinct single-vehicle and multi-vehicles categories of driving scenarios, as well as a novel hybrid MEBN inference, demonstrate the capabilities of the proposed traffic assessment model to efficiently achieve situation and threat assessment on the road. Finally, the impact assessment and decision making models are evaluated through two different scenarios of driving in highway and intersection that are formed with various number of player vehicles, and their actions

Datil: Learning Fuzzy Ontology Datatypes

International audienceReal-world applications using fuzzy ontologies are increasing in the last years, but the problem of fuzzy ontology learning has not received a lot of attention. While most of the previous approaches focus on the problem of learning fuzzy subclass axioms, we focus on learning fuzzy datatypes. In particular, we describe the Datil system, an implementation using unsupervised clustering algorithms to automatically obtain fuzzy datatypes from different input formats. We also illustrate the practical usefulness with an application: semantic lifestyle profiling