Using spatiotemporal patterns to qualitatively represent and manage dynamic situations of interest : a cognitive and integrative approach

Les situations spatio-temporelles dynamiques sont des situations qui évoluent dans l’espace et dans le temps. L’être humain peut identifier des configurations de situations dans son environnement et les utilise pour prendre des décisions. Ces configurations de situations peuvent aussi être appelées « situations d’intérêt » ou encore « patrons spatio-temporels ». En informatique, les situations sont obtenues par des systèmes d’acquisition de données souvent présents dans diverses industries grâce aux récents développements technologiques et qui génèrent des bases de données de plus en plus volumineuses. On relève un problème important dans la littérature lié au fait que les formalismes de représentation utilisés sont souvent incapables de représenter des phénomènes spatiotemporels dynamiques et complexes qui reflètent la réalité. De plus, ils ne prennent pas en considération l’appréhension cognitive (modèle mental) que l’humain peut avoir de son environnement. Ces facteurs rendent difficile la mise en œuvre de tels modèles par des agents logiciels. Dans cette thèse, nous proposons un nouveau modèle de représentation des situations d’intérêt s’appuyant sur la notion des patrons spatiotemporels. Notre approche utilise les graphes conceptuels pour offrir un aspect qualitatif au modèle de représentation. Le modèle se base sur les notions d’événement et d’état pour représenter des phénomènes spatiotemporels dynamiques. Il intègre la notion de contexte pour permettre aux agents logiciels de raisonner avec les instances de patrons détectés. Nous proposons aussi un outil de génération automatisée des relations qualitatives de proximité spatiale en utilisant un classificateur flou. Finalement, nous proposons une plateforme de gestion des patrons spatiotemporels pour faciliter l’intégration de notre modèle dans des applications industrielles réelles. Ainsi, les contributions principales de notre travail sont : Un formalisme de représentation qualitative des situations spatiotemporelles dynamiques en utilisant des graphes conceptuels. ; Une approche cognitive pour la définition des patrons spatio-temporels basée sur l’intégration de l’information contextuelle. ; Un outil de génération automatique des relations spatiales qualitatives de proximité basé sur les classificateurs neuronaux flous. ; Une plateforme de gestion et de détection des patrons spatiotemporels basée sur l’extension d’un moteur de traitement des événements complexes (Complex Event Processing).Dynamic spatiotemporal situations are situations that evolve in space and time. They are part of humans’ daily life. One can be interested in a configuration of situations occurred in the environment and can use it to make decisions. In the literature, such configurations are referred to as “situations of interests” or “spatiotemporal patterns”. In Computer Science, dynamic situations are generated by large scale data acquisition systems which are deployed everywhere thanks to recent technological advances. Spatiotemporal pattern representation is a research subject which gained a lot of attraction from two main research areas. In spatiotemporal analysis, various works extended query languages to represent patterns and to query them from voluminous databases. In Artificial Intelligence, predicate-based models represent spatiotemporal patterns and detect their instances using rule-based mechanisms. Both approaches suffer several shortcomings. For example, they do not allow for representing dynamic and complex spatiotemporal phenomena due to their limited expressiveness. Furthermore, they do not take into account the human’s mental model of the environment in their representation formalisms. This limits the potential of building agent-based solutions to reason about these patterns. In this thesis, we propose a novel approach to represent situations of interest using the concept of spatiotemporal patterns. We use Conceptual Graphs to offer a qualitative representation model of these patterns. Our model is based on the concepts of spatiotemporal events and states to represent dynamic spatiotemporal phenomena. It also incorporates contextual information in order to facilitate building the knowledge base of software agents. Besides, we propose an intelligent proximity tool based on a neuro-fuzzy classifier to support qualitative spatial relations in the pattern model. Finally, we propose a framework to manage spatiotemporal patterns in order to facilitate the integration of our pattern representation model to existing applications in the industry. The main contributions of this thesis are as follows: A qualitative approach to model dynamic spatiotemporal situations of interest using Conceptual Graphs. ; A cognitive approach to represent spatiotemporal patterns by integrating contextual information. ; An automated tool to generate qualitative spatial proximity relations based on a neuro-fuzzy classifier. ; A platform for detection and management of spatiotemporal patterns using an extension of a Complex Event Processing engine

Multimodal Content Delivery for Geo-services

This thesis describes a body of work carried out over several research projects in the area of multimodal interaction for location-based services. Research in this area has progressed from using simulated mobile environments to demonstrate the visual modality, to the ubiquitous delivery of rich media using multimodal interfaces (geo- services). To effectively deliver these services, research focused on innovative solutions to real-world problems in a number of disciplines including geo-location, mobile spatial interaction, location-based services, rich media interfaces and auditory user interfaces. My original contributions to knowledge are made in the areas of multimodal interaction underpinned by advances in geo-location technology and supported by the proliferation of mobile device technology into modern life. Accurate positioning is a known problem for location-based services, contributions in the area of mobile positioning demonstrate a hybrid positioning technology for mobile devices that uses terrestrial beacons to trilaterate position. Information overload is an active concern for location-based applications that struggle to manage large amounts of data, contributions in the area of egocentric visibility that filter data based on field-of-view demonstrate novel forms of multimodal input. One of the more pertinent characteristics of these applications is the delivery or output modality employed (auditory, visual or tactile). Further contributions in the area of multimodal content delivery are made, where multiple modalities are used to deliver information using graphical user interfaces, tactile interfaces and more notably auditory user interfaces. It is demonstrated how a combination of these interfaces can be used to synergistically deliver context sensitive rich media to users - in a responsive way - based on usage scenarios that consider the affordance of the device, the geographical position and bearing of the device and also the location of the device

Context classification for service robots

This dissertation presents a solution for environment sensing using sensor fusion techniques and a context/environment classification of the surroundings in a service robot, so it could change his behavior according to the different rea-soning outputs. As an example, if a robot knows he is outdoors, in a field environment, there can be a sandy ground, in which it should slow down. Contrariwise in indoor environments, that situation is statistically unlikely to happen (sandy ground). This simple assumption denotes the importance of context-aware in automated guided vehicles

Handling Real-World Context Awareness, Uncertainty and Vagueness in Real-Time Human Activity Tracking and Recognition with a Fuzzy Ontology-Based Hybrid Method

Human activity recognition is a key task in ambient intelligence applications to achieve proper ambient assisted living. There has been remarkable progress in this domain, but some challenges still remain to obtain robust methods. Our goal in this work is to provide a system that allows the modeling and recognition of a set of complex activities in real life scenarios involving interaction with the environment. The proposed framework is a hybrid model that comprises two main modules: a low level sub-activity recognizer, based on data-driven methods, and a high-level activity recognizer, implemented with a fuzzy ontology to include the semantic interpretation of actions performed by users. The fuzzy ontology is fed by the sub-activities recognized by the low level data-driven component and provides fuzzy ontological reasoning to recognize both the activities and their influence in the environment with semantics. An additional benefit of the approach is the ability to handle vagueness and uncertainty in the knowledge-based module, which substantially outperforms the treatment of incomplete and/or imprecise data with respect to classic crisp ontologies. We validate these advantages with the public CAD-120 dataset (Cornell Activity Dataset), achieving an accuracy of 90.1% and 91.07% for low-level and high-level activities, respectively. This entails an improvement over fully data-driven or ontology-based approaches.This work was funded by TUCS (Turku Centre for Computer Science), Finnish Cultural Foundation, Nokia Foundation, Google Anita Borg Scholarship, CEI BioTIC Project CEI2013-P-3, Contrato-Programa of Faculty of Education, Economy and Technology of Ceuta and Project TIN2012-30939 from National I+D Research Program (Spain). We also thank Fernando Bobillo for his support with FuzzyOWL and FuzzyDL tools

A framework for user-assisted knowledge acquisition from sensor data

The availability of a new generation of accurate, low-cost sensors to scientists has resulted in widespread deployment of these sensors in a variety of environments. Data generated by these devices are often in a raw, proprietary or unstructured format. As a result, it is difficult for scientists to analyse or query across various biological and physiological sensor data values. There exists both a physical-digital divide between sensor data with related real-world conditions, and a knowledge divide between the information needs of domain specialists. A key challenge is to bridge these divisions in order to allow scientists to make better decisions based on the sensed in- formation. The goal of this research is to show that low level data collection resources such as sensors can be used for high level query expressions and knowledge extraction, without the need for expensive human based operations. To achieve this goal, it was necessary to deliver a generic approach to enriching raw sensor data, providing information services to enable the end user to acquire knowledge from low-level sensor data and defining an integration framework for sensor data and related contextual information. As a result, key research questions of interpreting heterogeneous sensor data, enriching sensor data with contextual information, and integrating sensor data with related participant and environmental information, are tackled over the course of this dissertation

4Sensing - decentralized processing for participatory sensing data

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial para obtenção do grau de Mestre em Engenharia Informática.Participatory sensing is a new application paradigm, stemming from both technical and social drives, which is currently gaining momentum as a research domain. It leverages the growing adoption of mobile phones equipped with sensors, such as camera, GPS and accelerometer, enabling users to collect and aggregate data, covering a wide area without incurring in the costs associated with a large-scale sensor network. Related research in participatory sensing usually proposes an architecture based on a centralized back-end. Centralized solutions raise a set of issues. On one side, there is the implications of having a centralized repository hosting privacy sensitive information. On the other side, this centralized model has financial costs that can discourage grassroots initiatives. This dissertation focuses on the data management aspects of a decentralized infrastructure for the support of participatory sensing applications, leveraging the body of work on participatory sensing and related areas, such as wireless and internet-wide sensor networks, peer-to-peer data management and stream processing. It proposes a framework covering a common set of data management requirements - from data acquisition, to processing, storage and querying - with the goal of lowering the barrier for the development and deployment of applications. Alternative architectural approaches - RTree, QTree and NTree - are proposed and evaluated experimentally in the context of a case-study application - SpeedSense - supporting the monitoring and prediction of traffic conditions, through the collection of speed and location samples in an urban setting, using GPS equipped mobile phones