Privacy-preserving human mobility and activity modelling

The exponential proliferation of digital trends and worldwide responses to the COVID-19 pandemic thrust the world into digitalization and interconnectedness, pushing increasingly new technologies/devices/applications into the market. More and more intimate data of users are collected for positive analysis purposes of improving living well-being but shared with/without the user's consent, emphasizing the importance of making human mobility and activity models inclusive, private, and fair. In this thesis, I develop and implement advanced methods/algorithms to model human mobility and activity in terms of temporal-context dynamics, multi-occupancy impacts, privacy protection, and fair analysis. The following research questions have been thoroughly investigated: i) whether the temporal information integrated into the deep learning networks can improve the prediction accuracy in both predicting the next activity and its timing; ii) how is the trade-off between cost and performance when optimizing the sensor network for multiple-occupancy smart homes; iii) whether the malicious purposes such as user re-identification in human mobility modelling could be mitigated by adversarial learning; iv) whether the fairness implications of mobility models and whether privacy-preserving techniques perform equally for different groups of users. To answer these research questions, I develop different architectures to model human activity and mobility. I first clarify the temporal-context dynamics in human activity modelling and achieve better prediction accuracy by appropriately using the temporal information. I then design a framework MoSen to simulate the interaction dynamics among residents and intelligent environments and generate an effective sensor network strategy. To relieve users' privacy concerns, I design Mo-PAE and show that the privacy of mobility traces attains decent protection at the marginal utility cost. Last but not least, I investigate the relations between fairness and privacy and conclude that while the privacy-aware model guarantees group fairness, it violates the individual fairness criteria.Open Acces

Machine learning for smart building applications: Review and taxonomy

© 2019 Association for Computing Machinery. The use of machine learning (ML) in smart building applications is reviewed in this article. We split existing solutions into two main classes: occupant-centric versus energy/devices-centric. The first class groups solutions that use ML for aspects related to the occupants, including (1) occupancy estimation and identification, (2) activity recognition, and (3) estimating preferences and behavior. The second class groups solutions that use ML to estimate aspects related either to energy or devices. They are divided into three categories: (1) energy profiling and demand estimation, (2) appliances profiling and fault detection, and (3) inference on sensors. Solutions in each category are presented, discussed, and compared; open perspectives and research trends are discussed as well. Compared to related state-of-the-art survey papers, the contribution herein is to provide a comprehensive and holistic review from the ML perspectives rather than architectural and technical aspects of existing building management systems. This is by considering all types of ML tools, buildings, and several categories of applications, and by structuring the taxonomy accordingly. The article ends with a summary discussion of the presented works, with focus on lessons learned, challenges, open and future directions of research in this field

The Sensor Network Workbench: Towards Functional Specification, Verification and Deployment of Constrained Distributed Systems

As the commoditization of sensing, actuation and communication hardware increases, so does the potential for dynamically tasked sense and respond networked systems (i.e., Sensor Networks or SNs) to replace existing disjoint and inflexible special-purpose deployments (closed-circuit security video, anti-theft sensors, etc.). While various solutions have emerged to many individual SN-centric challenges (e.g., power management, communication protocols, role assignment), perhaps the largest remaining obstacle to widespread SN deployment is that those who wish to deploy, utilize, and maintain a programmable Sensor Network lack the programming and systems expertise to do so. The contributions of this thesis centers on the design, development and deployment of the SN Workbench (snBench). snBench embodies an accessible, modular programming platform coupled with a flexible and extensible run-time system that, together, support the entire life-cycle of distributed sensory services. As it is impossible to find a one-size-fits-all programming interface, this work advocates the use of tiered layers of abstraction that enable a variety of high-level, domain specific languages to be compiled to a common (thin-waist) tasking language; this common tasking language is statically verified and can be subsequently re-translated, if needed, for execution on a wide variety of hardware platforms. snBench provides: (1) a common sensory tasking language (Instruction Set Architecture) powerful enough to express complex SN services, yet simple enough to be executed by highly constrained resources with soft, real-time constraints, (2) a prototype high-level language (and corresponding compiler) to illustrate the utility of the common tasking language and the tiered programming approach in this domain, (3) an execution environment and a run-time support infrastructure that abstract a collection of heterogeneous resources into a single virtual Sensor Network, tasked via this common tasking language, and (4) novel formal methods (i.e., static analysis techniques) that verify safety properties and infer implicit resource constraints to facilitate resource allocation for new services. This thesis presents these components in detail, as well as two specific case-studies: the use of snBench to integrate physical and wireless network security, and the use of snBench as the foundation for semester-long student projects in a graduate-level Software Engineering course

Positioning Commuters And Shoppers Through Sensing And Correlation

Positioning is a basic and important need in many scenarios of human daily activities. With position information, multifarious services could be vitalized to benefit all kinds of users, from individuals to organizations. Through positioning, people are able to obtain not only geo-location but also time related information. By aggregating position information from individuals, organizations could derive statistical knowledge about group behaviors, such as traffic, business, event, etc. Although enormous effort has been invested in positioning related academic and industrial work, there are still many holes to be filled. This dissertation proposes solutions to address the need of positioning in people’s daily life from two aspects: transportation and shopping. All the solutions are smart-device-based (e.g. smartphone, smartwatch), which could potentially benefit most users considering the prevalence of smart devices. In positioning relevant activities, the components and their movement information could be sensed by different entities from diverse perspectives. The mechanisms presented in this dissertation treat the information collected from one perspective as reference and match it against the data collected from other perspectives to acquire absolute or relative position, in spatial as well as temporal dimension. For transportation, both driver and passenger oriented solutions are proposed. To help drivers improve safety and ease the tension from driving, two correlated systems, OmniView [1] and DriverTalk [2], are provided. These systems infer the relative positions of the vehicles moving together by matching the appearance images of the vehicles seen by each other, which help drivers maintain safe distance from surrounding vehicles and also give them opportunities to precisely convey driving related messages to targeted peer drivers. To improve bus-riding experience for passengers of public transit systems, a system named RideSense [3] is developed. This system correlates the sensor traces collected by both passengers’ smart devices and reference devices in buses to position passengers’ bus-riding, spatially and temporally. With this system, passengers could be billed without any explicit interaction with conventional ticketing facilities in bus system, which makes the transportation system more efficient. For shopping activities, AutoLabel [4, 5] comes into play, which could position customers with regard to stores. AutoLabel constructs a mapping between WiFi vectors and semantic names of stores through correlating the text decorated inside stores with those on stores’ websites. Later, through WiFi scanning and a lookup in the mapping, customers’ smart devices could automatically recognize the semantic names of the stores they are in or nearby. Therefore, AutoLabel-enabled smart device serves as a bridge for the information flow between business owners and customers, which could benefit both sides

Vehicular Networks and Outdoor Pedestrian Localization

This thesis focuses on vehicular networks and outdoor pedestrian localization. In particular, it targets secure positioning in vehicular networks and pedestrian localization for safety services in outdoor environments. The former research topic must cope with three major challenges, concerning users’ privacy, computational costs of security and the system trust on user correctness. This thesis addresses those issues by proposing a new lightweight privacy-preserving framework for continuous tracking of vehicles. The proposed solution is evaluated in both dense and sparse vehicular settings through simulation and experiments in real-world testbeds. In addition, this thesis explores the benefit given by the use of low frequency bands for the transmission of control messages in vehicular networks. The latter topic is motivated by a significant number of traffic accidents with pedestrians distracted by their smartphones. This thesis proposes two different localization solutions specifically for pedestrian safety: a GPS-based approach and a shoe-mounted inertial sensor method. The GPS-based solution is more suitable for rural and suburban areas while it is not applicable in dense urban environments, due to large positioning errors. Instead the inertial sensor approach overcomes the limitations of previous technique in urban environments. Indeed, by exploiting accelerometer data, this architecture is able to precisely detect the transitions from safe to potentially unsafe walking locations without the need of any absolute positioning systems

Konvoluutioneuroverkot ja Gaussiset prosessit sensoridatan analysoimiseen

Different sensors are constantly collecting information about us and our surroundings, such as pollution levels or heart rates. This results in long sequences of noisy time series observations, often also referred to as signals. This thesis develops machine learning methods for analysing such sensor data. The motivation behind the work is based on three real-world applications. In one, the goal is to improve Wi-Fi networks and recognise devices causing interference from spectral data measured by a spectrum analyser. The second one uses ultrasound signals propagated through different paths to localise objects inside closed containers, such as fouling inside of industrial pipelines. In third, the goal is to model an engine of a car and its emissions. Machine learning builds models of complex systems based on a set of observations. We develop models that are designed for analysing time series data, and we build on existing work on two different models: convolutional neural networks (CNNs) and Gaussian processes (GPs). We show that CNNs are able to automatically recognise useful patterns both in 1D and 2D signal data, even when we use a chaotic cavity to scatter waves randomly in order to increase the acoustic aperture. We show how GPs can be used when the observations can be interpreted as integrals over some region, and how we can introduce a non-negativity constraint in such cases. We also show how Gaussian process state space models can be used to learn long- and short-term effects simultaneously by training the model with different resolutions of the data. The amount of data in our case studies is limited as the datasets have been collected manually using a limited amount of sensors. This adds additional challenges to modeling, and we have used different approaches to cope with limited data. GPs as a model are well suited for small data as they are able to naturally model uncertainties. We also show how a dataset can be collected so that it contains as much information as possible with the limited resources available in cases where we use GPs with integral observations. CNNs in general require large datasets, but we show how we can augment labeled data with unlabeled data by taking advantage of the continuity in sensor data.Erilaiset sensorit keräävät jatkuvasti dataa meistä ja ympäristöstämme, kuten ilmanlaadusta tai ihmisen sykkeestä. Tuloksena on pitkiä aikasarjahavaintoja, joita usein kutsutaan myös signaaleiksi. Tässä työssä kehitetään koneoppimismenetelmiä sensoridatan analysoimiseen. Motivaationa työssä on kolme erilaista käytännön sovellusta. Ensimmäisessä pyritään parantamaan Wi-Fi -verkkojen toimintaa tunnistamalla häiriötä aiheuttavia laitteita spektridatasta. Toisessa käytetään ultraääntä paikallistamaan kohteita suljettujen säiliöden sisällä. Kolmannessa mallinnetaan auton moottoria ja sen päästöjä. Koneoppiminen muodostaa malleja monimutkaisista järjestelmistä havaintojen pohjalta. Tässä työssä kehitetään malleja, jotka sopivat erityisesti aikasarjojen analysointiin. Nämä mallit perustuvat kahteen erilaiseen malliperheeseen: konvoluutioneuroverkkoihin ja Gaussisiin prosesseihin. Työssä kehitetään konvoluutioneuroverkkoja sekä yksi- että kaksiulotteisen signaalidatan analysointiin ja lisäksi osoitetaan, että niiden avulla voidaan tulkita myös signaaleja jotka on hajautettu satunnaisesti mittausalueen kasvattamiseksi. Työssä kehitetään Gaussisia prosesseja tapauksiin, joissa havainnot ovat integraaleja tuntemattoman funktion yli ja yleistetään menetelmä myös tilanteisiin joissa tuntemattoman funktion arvot ovat rajoitettuja, esimerkiksi ei-negativisia. Lisäksi esittelemme tavan, jolla gaussisia prosesseja hyödyntävät tila-avaruusmallit pystyvät oppimaan sekä pitkän että lyhyen aikavälin ilmiöitä käyttämällä opettamiseen datan eri resoluutioita. Työssä käsiteltävissä sovelluksissa datan määrä on verrattain pieni, sillä data on kerätty manuaalisesti vain pienellä määrällä sensoreita. Tässä työssä esitellään myös ratkaisuja pieniin datamääriin liittyviin haasteisiin. Näytämme, miten data voidaan kerätä niin, että se sisältää mahdollisimman paljon informaatiota pienistä resursseista huolimatta, tapauksissa, joissa havainnot vastaavat integraaleja alueiden yli. Konvoluutioneuroverkot tyypillisesti tarvitsevat opettamiseen paljon dataa, mutta työ esittelee miten opettamisessa voidaan täydentää luokiteltua dataa luokittelemattomalla datalla hyödyntämällä sensoridatan aikajatkuvuutta

Large-scale sensor-rich video management and delivery

Ph.DDOCTOR OF PHILOSOPH

Enabling Context-Awareness in Mobile Systems via Multi-Modal Sensing

<p>The inclusion of rich sensors on modern smartphones has changed mobile phones from simple communication devices to powerful human-centric sensing platforms. Similar trends are influencing other personal gadgets such as the tablets, cameras, and wearable devices like the Google glass. Together, these sensors can provide</p><p>a high-resolution view of the user's context, ranging from simple information like locations and activities, to high-level inferences about the users' intention, behavior, and social interactions. Understanding such context can help solving existing system-side</p><p>challenges and eventually enable a new world of real-life applications. </p><p>In this thesis, we propose to learn human behavior via multi-modal sensing. The intuition is that human behaviors leave footprints on different sensing dimensions - visual, acoustic, motion and in cyber space. By collaboratively analyzing these footprints, the system can obtain valuable insights about the user. We show that the</p><p>analysis results can lead to a series of applications including capturing life-logging videos, tagging user-generated photos and enabling new ways for human-object interactions. Moreover, the same intuition may potentially be applied to enhance existing</p><p>system-side functionalities - offloading, prefetching and compression.</p>Dissertatio

Recent Advances in Indoor Localization Systems and Technologies

Despite the enormous technical progress seen in the past few years, the maturity of indoor localization technologies has not yet reached the level of GNSS solutions. The 23 selected papers in this book present the recent advances and new developments in indoor localization systems and technologies, propose novel or improved methods with increased performance, provide insight into various aspects of quality control, and also introduce some unorthodox positioning methods