Anticipatory Mobile Computing: A Survey of the State of the Art and Research Challenges

Today's mobile phones are far from mere communication devices they were ten years ago. Equipped with sophisticated sensors and advanced computing hardware, phones can be used to infer users' location, activity, social setting and more. As devices become increasingly intelligent, their capabilities evolve beyond inferring context to predicting it, and then reasoning and acting upon the predicted context. This article provides an overview of the current state of the art in mobile sensing and context prediction paving the way for full-fledged anticipatory mobile computing. We present a survey of phenomena that mobile phones can infer and predict, and offer a description of machine learning techniques used for such predictions. We then discuss proactive decision making and decision delivery via the user-device feedback loop. Finally, we discuss the challenges and opportunities of anticipatory mobile computing.Comment: 29 pages, 5 figure

Exploring The Responsibilities Of Single-Inhabitant Smart Homes With Use Cases

DOI: 10.3233/AIS-2010-0076This paper makes a number of contributions to the field of requirements analysis for Smart Homes. It introduces Use Cases as a tool for exploring the responsibilities of Smart Homes and it proposes a modification of the conventional Use Case structure to suit the particular requirements of Smart Homes. It presents a taxonomy of Smart-Home-related Use Cases with seven categories. It draws on those Use Cases as raw material for developing questions and conclusions about the design of Smart Homes for single elderly inhabitants, and it introduces the SHMUC repository, a web-based repository of Use Cases related to Smart Homes that anyone can exploit and to which anyone may contribute

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Player agency in interactive narrative: audience, actor & author

The question motivating this review paper is, how can computer-based interactive narrative be used as a constructivist learn- ing activity? The paper proposes that player agency can be used to link interactive narrative to learner agency in constructivist theory, and to classify approaches to interactive narrative. The traditional question driving research in interactive narrative is, ‘how can an in- teractive narrative deal with a high degree of player agency, while maintaining a coherent and well-formed narrative?’ This question derives from an Aristotelian approach to interactive narrative that, as the question shows, is inherently antagonistic to player agency. Within this approach, player agency must be restricted and manip- ulated to maintain the narrative. Two alternative approaches based on Brecht’s Epic Theatre and Boal’s Theatre of the Oppressed are reviewed. If a Boalian approach to interactive narrative is taken the conflict between narrative and player agency dissolves. The question that emerges from this approach is quite different from the traditional question above, and presents a more useful approach to applying in- teractive narrative as a constructivist learning activity

Enabling Privacy and Trust in Edge AI Systems

Recent advances in mobile computing and the Internet of Things (IoT) enable the global integration of heterogeneous smart devices via wireless networks. A common characteristic across these modern day systems is their ability to collect and communicate streaming data, making machine learning (ML) appealing for processing, reasoning, and predicting about the environment. More recently, low network latency requirements have made offloading intelligence to the cloud undesirable. These novel requirements have led to the emergence of edge computing, an approach that brings computation closer to the device with low latency, high throughput, and enhanced reliability. Together, they enable ML-powered information processing and control pipelines spanning end devices, edge computing, and cloud environments. However, continuous collaboration between cloud, edge and device is susceptible to information leakage and loss, leading to insecure and unreliable operation. This raises an important question: how can we design, develop, and evaluate high-performing ML systems that are trustworthy and privacy-preserving in resource-constrained edge environments? In this thesis, I address this question by designing and implementing privacy-preserving and trustworthy ML systems for distributed applications. I first introduce a system that establishes trust in the explanations generated from a popular visualization technique, saliency maps, using counterfactual reasoning. Through the proposed evaluation system, I assess the degree to which hypothesized explanations correspond to the semantics of edge-based reinforcement learning environments. Second, I examine the privacy implications of personalized models in distributed mobile services by proposing time-series based model inversion attacks. To thwart such attacks, I present a distributed framework, Pelican, that learns and deploys transfer learning-based personalized ML models in a privacy preserving manner on resource-constrained mobile devices. Third, I investigate ML models that are deployed on local devices for inference and highlight the ease with which proprietary information embedded in these models can be exposed. For mitigating such attacks, I present a secure on-device application framework, SODA, which is supported by real-time adversarial detection. Finally, I present an end-to-end privacy-aware system for a real-world application to model group interaction behavior via mobility sensing. The proposed system, W4-Groups, distributes computation across device, edge, and cloud resources to strengthen its privacy and trustworthiness guarantees

Ami-deu : un cadre sémantique pour des applications adaptables dans des environnements intelligents

Cette thèse vise à étendre l’utilisation de l'Internet des objets (IdO) en facilitant le développement d’applications par des personnes non experts en développement logiciel. La thèse propose une nouvelle approche pour augmenter la sémantique des applications d’IdO et l’implication des experts du domaine dans le développement d’applications sensibles au contexte. Notre approche permet de gérer le contexte changeant de l’environnement et de générer des applications qui s’exécutent dans plusieurs environnements intelligents pour fournir des actions requises dans divers contextes. Notre approche est mise en œuvre dans un cadriciel (AmI-DEU) qui inclut les composants pour le développement d’applications IdO. AmI-DEU intègre les services d’environnement, favorise l’interaction de l’utilisateur et fournit les moyens de représenter le domaine d’application, le profil de l’utilisateur et les intentions de l’utilisateur. Le cadriciel permet la définition d’applications IoT avec une intention d’activité autodécrite qui contient les connaissances requises pour réaliser l’activité. Ensuite, le cadriciel génère Intention as a Context (IaaC), qui comprend une intention d’activité autodécrite avec des connaissances colligées à évaluer pour une meilleure adaptation dans des environnements intelligents. La sémantique de l’AmI-DEU est basée sur celle du ContextAA (Context-Aware Agents) – une plateforme pour fournir une connaissance du contexte dans plusieurs environnements. Le cadriciel effectue une compilation des connaissances par des règles et l'appariement sémantique pour produire des applications IdO autonomes capables de s’exécuter en ContextAA. AmI- DEU inclut également un outil de développement visuel pour le développement et le déploiement rapide d'applications sur ContextAA. L'interface graphique d’AmI-DEU adopte la métaphore du flux avec des aides visuelles pour simplifier le développement d'applications en permettant des définitions de règles étape par étape. Dans le cadre de l’expérimentation, AmI-DEU comprend un banc d’essai pour le développement d’applications IdO. Les résultats expérimentaux montrent une optimisation sémantique potentielle des ressources pour les applications IoT dynamiques dans les maisons intelligentes et les villes intelligentes. Notre approche favorise l'adoption de la technologie pour améliorer le bienêtre et la qualité de vie des personnes. Cette thèse se termine par des orientations de recherche que le cadriciel AmI-DEU dévoile pour réaliser des environnements intelligents omniprésents fournissant des adaptations appropriées pour soutenir les intentions des personnes.Abstract: This thesis aims at expanding the use of the Internet of Things (IoT) by facilitating the development of applications by people who are not experts in software development. The thesis proposes a new approach to augment IoT applications’ semantics and domain expert involvement in context-aware application development. Our approach enables us to manage the changing environment context and generate applications that run in multiple smart environments to provide required actions in diverse settings. Our approach is implemented in a framework (AmI-DEU) that includes the components for IoT application development. AmI- DEU integrates environment services, promotes end-user interaction, and provides the means to represent the application domain, end-user profile, and end-user intentions. The framework enables the definition of IoT applications with a self-described activity intention that contains the required knowledge to achieve the activity. Then, the framework generates Intention as a Context (IaaC), which includes a self-described activity intention with compiled knowledge to be assessed for augmented adaptations in smart environments. AmI-DEU framework semantics adopts ContextAA (Context-Aware Agents) – a platform to provide context-awareness in multiple environments. The framework performs a knowledge compilation by rules and semantic matching to produce autonomic IoT applications to run in ContextAA. AmI-DEU also includes a visual tool for quick application development and deployment to ContextAA. The AmI-DEU GUI adopts the flow metaphor with visual aids to simplify developing applications by allowing step-by-step rule definitions. As part of the experimentation, AmI-DEU includes a testbed for IoT application development. Experimental results show a potential semantic optimization for dynamic IoT applications in smart homes and smart cities. Our approach promotes technology adoption to improve people’s well-being and quality of life. This thesis concludes with research directions that the AmI-DEU framework uncovers to achieve pervasive smart environments providing suitable adaptations to support people’s intentions

Integrating Emotion Recognition Tools for Developing Emotionally Intelligent Agents

Emotionally responsive agents that can simulate emotional intelligence increase the acceptance of users towards them, as the feeling of empathy reduces negative perceptual feedback. This has fostered research on emotional intelligence during last decades, and nowadays numerous cloud and local tools for automatic emotional recognition are available, even for inexperienced users. These tools however usually focus on the recognition of discrete emotions sensed from one communication channel, even though multimodal approaches have been shown to have advantages over unimodal approaches. Therefore, the objective of this paper is to show our approach for multimodal emotion recognition using Kalman filters for the fusion of available discrete emotion recognition tools. The proposed system has been modularly developed based on an evolutionary approach so to be integrated in our digital ecosystems, and new emotional recognition sources can be easily integrated. Obtained results show improvements over unimodal tools when recognizing naturally displayed emotions