A review of the role of sensors in mobile context-aware recommendation systems

Recommendation systems are specialized in offering suggestions about specific items of different types (e.g., books, movies, restaurants, and hotels) that could be interesting for the user. They have attracted considerable research attention due to their benefits and also their commercial interest. Particularly, in recent years, the concept of context-aware recommendation system has appeared to emphasize the importance of considering the context of the situations in which the user is involved in order to provide more accurate recommendations. The detection of the context requires the use of sensors of different types, which measure different context variables. Despite the relevant role played by sensors in the development of context-aware recommendation systems, sensors and recommendation approaches are two fields usually studied independently. In this paper, we provide a survey on the use of sensors for recommendation systems. Our contribution can be seen from a double perspective. On the one hand, we overview existing techniques used to detect context factors that could be relevant for recommendation. On the other hand, we illustrate the interest of sensors by considering different recommendation use cases and scenarios

Analysing Crowd Behaviours using Mobile Sensing

PhDResearchers have examined crowd behaviour in the past by employing a variety of methods including ethnographic studies, computer vision techniques and manual annotation-based data analysis. However, because of the resources to collect, process and analyse data, it remains difficult to obtain large data sets for study. Mobile phones offer easier means for data collection that is easy to analyse and can preserve the user’s privacy. The aim of this thesis is to identify and model different qualities of social interactions inside crowds using mobile sensing technology. This Ph.D. research makes three main contributions centred around the mobile sensing and crowd sensing area. Firstly, an open-source licensed mobile sensing framework is developed, named SensingKit, that is capable of collecting mobile sensor data from iOS and Android devices, supporting most sensors available in modern smartphones. The framework has been evaluated in a case study that investigates the pedestrian gait synchronisation phenomenon. Secondly, a novel algorithm based on graph theory is proposed capable of detecting stationary social interactions within crowds. It uses sensor data available in a modern smartphone device, such as the Bluetooth Smart (BLE) sensor, as an indication of user proximity, and accelerometer sensor, as an indication of each user’s motion state. Finally, a machine learning model is introduced that uses multi-modal mobile sensor data extracted from Bluetooth Smart, accelerometer and gyroscope sensors. The validation was performed using a relatively large dataset with 24 participants, where they were asked to socialise with each other for 45 minutes. By using supervised machine learning based on gradient-boosted trees, a performance increase of 26.7% was achieved over a proximity-based approach. Such model can be beneficial to the design and implementation of in-the-wild crowd behavioural analysis, design of influence strategies, and algorithms for crowd reconfiguration.UK Defence Science & Technology Laboratory (DSTL

Transmission Modeling with Smartphone-based Sensing

Infectious disease spread is difficult to accurately measure and model. Even for well-studied pathogens, uncertainties remain regarding the dynamics of mixing behavior and how to balance simulation-generated estimates with empirical data. Smartphone-based sensing data promises the availability of inferred proximate contacts, with which we can improve transmission models. This dissertation addresses the problem of informing transmission models with proximity contact data by breaking it down into three sub-questions. Firstly, can proximity contact data inform transmission models? To this question, an extended-Kalman-filter enhanced System Dynamics Susceptible-Infectious-Removed (EKF-SD-SIR) model demonstrated the filtering approach, as a framework, for informing Systems Dynamics models with proximity contact data. This combination results in recurrently-regrounded system status as empirical data arrive throughout disease transmission simulations---simultaneously considering empirical data accuracy, growing simulation error between measurements, and supporting estimation of changing model parameters. However, as revealed by this investigation, this filtering approach is limited by the quality and reliability of sensing-informed proximate contacts, which leads to the dissertation's second and third questions---investigating the impact of temporal and spatial resolution on sensing inferred proximity contact data for transmission models. GPS co-location and Bluetooth beaconing are two of those common measurement modalities to sense proximity contacts with different underlying technologies and tradeoffs. However, both measurement modalities have shortcomings and are prone to false positives or negatives when used to detect proximate contacts because unmeasured environmental influences bias the data. Will differences in sensing modalities impact transmission models informed by proximity contact data? The second part of this dissertation compares GPS- and Bluetooth-inferred proximate contacts by accessing their impact on simulated attack rates in corresponding proximate-contact-informed agent-based Susceptible-Exposed-Infectious-Recovered (ABM-SEIR) models of four distinct contagious diseases. Results show that the inferred proximate contacts resulting from these two measurement modalities are different and give rise to significantly different attack rates across multiple data collections and pathogens. While the advent of commodity mobile devices has eased the collection of proximity contact data, battery capacity and associated costs impose tradeoffs between the frequency and scanning duration used for proximate-contact detection. The choice of a balanced sensing regime involves specifying temporal resolutions and interpreting sensing data---depending on circumstances such as the characteristics of a particular pathogen, accompanying disease, and underlying population. How will the temporal resolution of sensing impact transmission models informed by proximity contact data? Furthermore, how will circumstances alter the impact of temporal resolution? The third part of this dissertation investigates the impacts of sensing regimes on findings from two sampling methods of sensing at widely varying inter-observation intervals by synthetically downsampling proximity contact data from five contact network studies---with each of these five studies measuring participant-participant contact every 5 minutes for durations of four or more weeks. The impact of downsampling is evaluated through ABM-SEIR simulations from both population- and individual-level for 12 distinct contagious diseases and associated variants of concern. Studies in this part find that for epidemiological models employing proximity contact data, both the observation paradigms and the inter-observation interval configured to collect proximity contact data exert impacts on the simulation results. Moreover, the impact is subject to the population characteristics and pathogen infectiousness reflective (such as the basic reproduction number, $R_0$). By comparing the performance of two sampling methods of sensing, we found that in most cases, periodically observing for a certain duration can collect proximity contact data that allows agent-based models to produce a reasonable estimation of the attack rate. However, higher-resolution data are preferred for modeling individual infection risk. Findings from this part of the dissertation represent a step towards providing the empirical basis for guidelines to inform data collection that is at once efficient and effective. This dissertation addresses the problem of informing transmission models with proximity contact data in three steps. Firstly, the demonstration of an EKF-SD-SIR model suggests that the filtering approach could improve System Dynamics transmission models by leveraging proximity contact data. In addition, experiments with the EKF-SD-SIR model also revealed that the filtering approach is constrained by the limited quality and reliability of sensing-data-inferred proximate contacts. The following two parts of this dissertation investigate spatial-temporal factors that could impact the quality and reliability of sensor-collected proximity contact data. In the second step, the impact of spatial resolution is illustrated by differences between two typical sensing modalities---Bluetooth beaconing versus GPS co-location. Experiments show that, in general, proximity contact data collected with Bluetooth beaconing lead to transmission models with results different from those driven by proximity contact data collected with GPS co-location. Awareness of the differences between sensing modalities can aid researchers in incorporating proximity contact data into transmission models. Finally, in the third step, the impact of temporal resolution is elucidated by investigating the differences between results of transmission models led by proximity contact data collected with varying observation frequencies. These differences led by varying observation frequencies are evaluated under circumstances with alternative assumptions regarding sampling method, disease/pathogen type, and the underlying population. Experiments show that the impact of sensing regimes is influenced by the type of diseases/pathogens and underlying population, while sampling once in a while can be a decent choice across all situations. This dissertation demonstrated the value of a filtering approach to enhance transmission models with sensor-collected proximity contact data, as well as explored spatial-temporal factors that will impact the accuracy and reliability of sensor-collected proximity contact data. Furthermore, this dissertation suggested guidance for future sensor-based proximity contact data collection and highlighted needs and opportunities for further research on sensing-inferred proximity contact data for transmission models

Multisensor Data Fusion in Pervasive Artificial Intelligence Systems

Intelligent systems designed to manage smart environments exploit numerous sensing and actuating devices, pervasively deployed so as to remain invisible to users and subtly learn their preferences and satisfy their needs. Nowadays, such systems are constantly evolving and becoming ever more complex, so it is increasingly difficult to develop them successfully. A possible solution to this problem might lie in delegating certain decisions to the machines themselves, making them more autonomous and able to self-configure and self-manage. This work presents a multi-tier architecture for a complete pervasive system capable of understanding the state of the surrounding environment, as well as using this knowledge to decide what actions should be performed to provide the best possible environmental conditions for end-users, in line with the Ambient Intelligence (AmI) paradigm. To achieve such high-level goals, the system has to effectively merge and analyze heterogeneous data collected by multiple sensors, pervasively deployed in a smart environment. To this end, the proposed system includes a context-aware, self-optimizing, adaptive module for sensor data fusion. Contextual information is leveraged in the fusion process, so as to increase the accuracy of inference and hence decision making in a dynamically changing environment. Additionally, two self-optimization modules are responsible for dynamically determining the subset of sensors to use, finding an optimal trade-off to minimize energy consumption and maximize sensing accuracy. The effectiveness of the proposed approach is demonstrated with the application scenario of user activity recognition in an AmI system managing a smart home environment. In order to increase the resilience of the system to highly uncertain and unreliable information, the architecture is enriched by a filtering module to pre-process raw data coming from lower levels, before feeding them to the data fusion and reasoning modules in the higher levels.Intelligent systems designed to manage smart environments exploit numerous sensing and actuating devices, pervasively deployed so as to remain invisible to users and subtly learn their preferences and satisfy their needs. Nowadays, such systems are constantly evolving and becoming ever more complex, so it is increasingly difficult to develop them successfully. A possible solution to this problem might lie in delegating certain decisions to the machines themselves, making them more autonomous and able to self-configure and self-manage. This work presents a multi-tier architecture for a complete pervasive system capable of understanding the state of the surrounding environment, as well as using this knowledge to decide what actions should be performed to provide the best possible environmental conditions for end-users, in line with the Ambient Intelligence (AmI) paradigm. To achieve such high-level goals, the system has to effectively merge and analyze heterogeneous data collected by multiple sensors, pervasively deployed in a smart environment. To this end, the proposed system includes a context-aware, self-optimizing, adaptive module for sensor data fusion. Contextual information is leveraged in the fusion process, so as to increase the accuracy of inference and hence decision making in a dynamically changing environment. Additionally, two self-optimization modules are responsible for dynamically determining the subset of sensors to use, finding an optimal trade-off to minimize energy consumption and maximize sensing accuracy. The effectiveness of the proposed approach is demonstrated with the application scenario of user activity recognition in an AmI system managing a smart home environment. In order to increase the resilience of the system to highly uncertain and unreliable information, the architecture is enriched by a filtering module to pre-process raw data coming from lower levels, before feeding them to the data fusion and reasoning modules in the higher levels

Mobile-based online data mining : outdoor activity recognition

One of the unique features of mobile applications is the context awareness. The mobility and power afforded by smartphones allow users to interact more directly and constantly with the external world more than ever before. The emerging capabilities of smartphones are fueling a rise in the use of mobile phones as input devices for a great range of application fields; one of these fields is the activity recognition. In pervasive computing, activity recognition has a significant weight because it can be applied to many real-life, human-centric problems. This important role allows providing services to various application domains ranging from real-time traffic monitoring to fitness monitoring, social networking, marketing and healthcare. However, one of the major problems that can shatter any mobile-based activity recognition model is the limited battery life. It represents a big hurdle for the quality and the continuity of the service. Indeed, excessive power consumption may become a major obstacle to broader acceptance context-aware mobile applications, no matter how useful the proposed service may be. We present during this thesis a novel unsupervised battery-aware approach to online recognize users’ outdoor activities without depleting the mobile resources. We succeed in associating the places visited by individuals during their movements to meaningful human activities. Our approach includes novel models that incrementally cluster users’ movements into different types of activities without any massive use of historical records. To optimize battery consumption, our approach behaves variably according to users’ behaviors and the remaining battery level. Moreover, we propose to learn users’ habits in order to reduce the activity recognition computation. Our innovative battery-friendly method combines activity recognition and prediction in order to recognize users’ activities accurately without draining the battery of their phones. We show that our approach reduces significantly the battery consumption while keeping the same high accuracy. Une des caractéristiques uniques des applications mobiles est la sensibilité au contexte. La mobilité et la puissance de calcul offertes par les smartphones permettent aux utilisateurs d’interagir plus directement et en permanence avec le monde extérieur. Ces capacités émergentes ont pu alimenter plusieurs champs d’applications comme le domaine de la reconnaissance d’activités. Dans le domaine de l'informatique omniprésente, la reconnaissance des activités humaines reçoit une attention particulière grâce à son implication profonde dans plusieurs problématiques de vie quotidienne. Ainsi, ce domaine est devenu une pièce majeure qui fournit des services à un large éventail de domaines comme la surveillance du trafic en temps réel, les réseaux sociaux, le marketing et la santé. Cependant, l'un des principaux problèmes qui peuvent compromettre un modèle de reconnaissance d’activité sur les smartphones est la durée de vie limitée de la batterie. Ce handicap représente un grand obstacle pour la qualité et la continuité du service. En effet, la consommation d'énergie excessive peut devenir un obstacle majeur aux applications sensibles au contexte, peu importe à quel point ce service est utile. Nous présentons dans de cette thèse une nouvelle approche non supervisée qui permet la détection incrémentale des activités externes sans épuiser les ressources du téléphone. Nous parvenons à associer efficacement les lieux visités par des individus lors de leurs déplacements à des activités humaines significatives. Notre approche comprend de nouveaux modèles de classification en ligne des activités humaines sans une utilisation massive des données historiques. Pour optimiser la consommation de la batterie, notre approche se comporte de façon variable selon les comportements des utilisateurs et le niveau de la batterie restant. De plus, nous proposons d'apprendre les habitudes des utilisateurs afin de réduire la complexité de l’algorithme de reconnaissance d'activités. Pour se faire, notre méthode combine la reconnaissance d’activités et la prédiction des prochaines activités afin d’atteindre une consommation raisonnable des ressources du téléphone. Nous montrons que notre proposition réduit remarquablement la consommation de la batterie tout en gardant un taux de précision élevé

Selected Papers from the 5th International Electronic Conference on Sensors and Applications

This Special Issue comprises selected papers from the proceedings of the 5th International Electronic Conference on Sensors and Applications, held on 15–30 November 2018, on sciforum.net, an online platform for hosting scholarly e-conferences and discussion groups. In this 5th edition of the electronic conference, contributors were invited to provide papers and presentations from the field of sensors and applications at large, resulting in a wide variety of excellent submissions and topic areas. Papers which attracted the most interest on the web or that provided a particularly innovative contribution were selected for publication in this collection. These peer-reviewed papers are published with the aim of rapid and wide dissemination of research results, developments, and applications. We hope this conference series will grow rapidly in the future and become recognized as a new way and venue by which to (electronically) present new developments related to the field of sensors and their applications