Exploiting Out-of-band Motion Sensor Data to De-anonymize Virtual Reality Users

Virtual Reality (VR) is an exciting new consumer technology which offers an immersive audio-visual experience to users through which they can navigate and interact with a digitally represented 3D space (i.e., a virtual world) using a headset device. By (visually) transporting users from the real or physical world to exciting and realistic virtual spaces, VR systems can enable true-to-life and more interactive versions of traditional applications such as gaming, remote conferencing, social networking and virtual tourism. However, as with any new consumer technology, VR applications also present significant user-privacy challenges. This paper studies a new type of privacy attack targeting VR users by connecting their activities visible in the virtual world (enabled by some VR application/service) to their physical state sensed in the real world. Specifically, this paper analyzes the feasibility of carrying out a de-anonymization or identification attack on VR users by correlating visually observed movements of users' avatars in the virtual world with some auxiliary data (e.g., motion sensor data from mobile/wearable devices held by users) representing their context/state in the physical world. To enable this attack, this paper proposes a novel framework which first employs a learning-based activity classification approach to translate the disparate visual movement data and motion sensor data into an activity-vector to ease comparison, followed by a filtering and identity ranking phase outputting an ordered list of potential identities corresponding to the target visual movement data. Extensive empirical evaluation of the proposed framework, under a comprehensive set of experimental settings, demonstrates the feasibility of such a de-anonymization attack

A Comprehensive Security Framework for Securing Sensors in Smart Devices and Applications

This doctoral dissertation introduces novel security frameworks to detect sensor-based threats on smart devices and applications in smart settings such as smart home, smart office, etc. First, we present a formal taxonomy and in-depth impact analysis of existing sensor-based threats to smart devices and applications based on attack characteristics, targeted components, and capabilities. Then, we design a novel context-aware intrusion detection system, 6thSense, to detect sensor-based threats in standalone smart devices (e.g., smartphone, smart watch, etc.). 6thSense considers user activity-sensor co-dependence in standalone smart devices to learn the ongoing user activity contexts and builds a context-aware model to distinguish malicious sensor activities from benign user behavior. Further, we develop a platform-independent context-aware security framework, Aegis, to detect the behavior of malicious sensors and devices in a connected smart environment (e.g., smart home, offices, etc.). Aegis observes the changing patterns of the states of smart sensors and devices for user activities in a smart environment and builds a contextual model to detect malicious activities considering sensor-device-user interactions and multi-platform correlation. Then, to limit unauthorized and malicious sensor and device access, we present, kratos, a multi-user multi-device-aware access control system for smart environment and devices. kratos introduces a formal policy language to understand diverse user demands in smart environment and implements a novel policy negotiation algorithm to automatically detect and resolve conflicting user demands and limit unauthorized access. For each contribution, this dissertation presents novel security mechanisms and techniques that can be implemented independently or collectively to secure sensors in real-life smart devices, systems, and applications. Moreover, each contribution is supported by several user and usability studies we performed to understand the needs of the users in terms of sensor security and access control in smart devices and improve the user experience in these real-time systems

Privacy-aware Security Applications in the Era of Internet of Things

In this dissertation, we introduce several novel privacy-aware security applications. We split these contributions into three main categories: First, to strengthen the current authentication mechanisms, we designed two novel privacy-aware alternative complementary authentication mechanisms, Continuous Authentication (CA) and Multi-factor Authentication (MFA). Our first system is Wearable-assisted Continuous Authentication (WACA), where we used the sensor data collected from a wrist-worn device to authenticate users continuously. Then, we improved WACA by integrating a noise-tolerant template matching technique called NTT-Sec to make it privacy-aware as the collected data can be sensitive. We also designed a novel, lightweight, Privacy-aware Continuous Authentication (PACA) protocol. PACA is easily applicable to other biometric authentication mechanisms when feature vectors are represented as fixed-length real-valued vectors. In addition to CA, we also introduced a privacy-aware multi-factor authentication method, called PINTA. In PINTA, we used fuzzy hashing and homomorphic encryption mechanisms to protect the users\u27 sensitive profiles while providing privacy-preserving authentication. For the second privacy-aware contribution, we designed a multi-stage privacy attack to smart home users using the wireless network traffic generated during the communication of the devices. The attack works even on the encrypted data as it is only using the metadata of the network traffic. Moreover, we also designed a novel solution based on the generation of spoofed traffic. Finally, we introduced two privacy-aware secure data exchange mechanisms, which allow sharing the data between multiple parties (e.g., companies, hospitals) while preserving the privacy of the individual in the dataset. These mechanisms were realized with the combination of Secure Multiparty Computation (SMC) and Differential Privacy (DP) techniques. In addition, we designed a policy language, called Curie Policy Language (CPL), to handle the conflicting relationships among parties. The novel methods, attacks, and countermeasures in this dissertation were verified with theoretical analysis and extensive experiments with real devices and users. We believe that the research in this dissertation has far-reaching implications on privacy-aware alternative complementary authentication methods, smart home user privacy research, as well as the privacy-aware and secure data exchange methods

Designing leakage-resilient password entry on head-mounted smart wearable glass devices

AXA Research Fun

An Approach to Software Development for Continuous Authentication of Smart Wearable Device Users

abstract: With the recent expansion in the use of wearable technology, a large number of users access personal data with these smart devices. The consumer market of wearables includes smartwatches, health and fitness bands, and gesture control armbands. These smart devices enable users to communicate with each other, control other devices, relax and work out more effectively. As part of their functionality, these devices store, transmit, and/or process sensitive user personal data, perhaps biological and location data, making them an abundant source of confidential user information. Thus, prevention of unauthorized access to wearables is necessary. In fact, it is important to effectively authenticate users to prevent intentional misuse or alteration of individual data. Current authentication methods for the legitimate users of smart wearable devices utilize passcodes, and graphical pattern based locks. These methods have the following problems: (1) passcodes can be stolen or copied, (2) they depend on conscious user inputs, which can be undesirable to a user, (3) they authenticate the user only at the beginning of the usage session, and (4) they do not consider user behavior or they do not adapt to evolving user behavior. In this thesis, an approach is presented for developing software for continuous authentication of the legitimate user of a smart wearable device. With this approach, the legitimate user of a smart wearable device can be authenticated based on the user's behavioral biometrics in the form of motion gestures extracted from the embedded sensors of the smart wearable device. The continuous authentication of this approach is accomplished by adapting the authentication to user's gesture pattern changes. This approach is demonstrated by using two comprehensive datasets generated by two research groups, and it is shown that this approach achieves better performance than existing methods.Dissertation/ThesisMasters Thesis Software Engineering 201

Practical and Rich User Digitization

A long-standing vision in computer science has been to evolve computing devices into proactive assistants that enhance our productivity, health and wellness, and many other facets of our lives. User digitization is crucial in achieving this vision as it allows computers to intimately understand their users, capturing activity, pose, routine, and behavior. Today's consumer devices - like smartphones and smartwatches provide a glimpse of this potential, offering coarse digital representations of users with metrics such as step count, heart rate, and a handful of human activities like running and biking. Even these very low-dimensional representations are already bringing value to millions of people's lives, but there is significant potential for improvement. On the other end, professional, high-fidelity comprehensive user digitization systems exist. For example, motion capture suits and multi-camera rigs that digitize our full body and appearance, and scanning machines such as MRI capture our detailed anatomy. However, these carry significant user practicality burdens, such as financial, privacy, ergonomic, aesthetic, and instrumentation considerations, that preclude consumer use. In general, the higher the fidelity of capture, the lower the user's practicality. Most conventional approaches strike a balance between user practicality and digitization fidelity. My research aims to break this trend, developing sensing systems that increase user digitization fidelity to create new and powerful computing experiences while retaining or even improving user practicality and accessibility, allowing such technologies to have a societal impact. Armed with such knowledge, our future devices could offer longitudinal health tracking, more productive work environments, full body avatars in extended reality, and embodied telepresence experiences, to name just a few domains.Comment: PhD thesi

Applying multimodal sensing to human location estimation

Mobile devices like smartphones and smartwatches are beginning to "stick" to the human body. Given that these devices are equipped with a variety of sensors, they are becoming a natural platform to understand various aspects of human behavior. This dissertation will focus on just one dimension of human behavior, namely "location". We will begin by discussing our research on localizing humans in indoor environments, a problem that requires precise tracking of human footsteps. We investigated the benefits of leveraging smartphone sensors (accelerometers, gyroscopes, magnetometers, etc.) into the indoor localization framework, which breaks away from pure radio frequency based localization (e.g., cellular, WiFi). Our research leveraged inherent properties of indoor environments to perform localization. We also designed additional solutions, where computer vision was integrated with sensor fusion to offer highly precise localization. We will close this thesis with micro-scale tracking of the human wrist and demonstrate how motion data processing is indeed a "double-edged sword", offering unprecedented utility on one hand while breaching privacy on the other

Edge Intelligence : Empowering Intelligence to the Edge of Network

Edge intelligence refers to a set of connected systems and devices for data collection, caching, processing, and analysis proximity to where data are captured based on artificial intelligence. Edge intelligence aims at enhancing data processing and protects the privacy and security of the data and users. Although recently emerged, spanning the period from 2011 to now, this field of research has shown explosive growth over the past five years. In this article, we present a thorough and comprehensive survey of the literature surrounding edge intelligence. We first identify four fundamental components of edge intelligence, i.e., edge caching, edge training, edge inference, and edge offloading based on theoretical and practical results pertaining to proposed and deployed systems. We then aim for a systematic classification of the state of the solutions by examining research results and observations for each of the four components and present a taxonomy that includes practical problems, adopted techniques, and application goals. For each category, we elaborate, compare, and analyze the literature from the perspectives of adopted techniques, objectives, performance, advantages and drawbacks, and so on. This article provides a comprehensive survey of edge intelligence and its application areas. In addition, we summarize the development of the emerging research fields and the current state of the art and discuss the important open issues and possible theoretical and technical directions.Peer reviewe

Machine Learning Algorithms for Privacy-preserving Behavioral Data Analytics

PhD thesisBehavioral patterns observed in data generated by mobile and wearable devices are used by many applications, such as wellness monitoring or service personalization. However, sensitive information may be inferred from these data when they are shared with cloud-based services. In this thesis, we propose machine learning algorithms for data transformations to allow the inference of information required for specific tasks while preventing the inference of privacy-sensitive information. Specifically, we focus on protecting the user’s privacy when sharing motion-sensor data and web-browsing histories. Firstly, for human activity recognition using data of wearable sensors, we introduce two algorithms for training deep neural networks to transform motion-sensor data, focusing on two objectives: (i) to prevent the inference of privacy-sensitive activities (e.g. smoking or drinking), and (ii) to protect user’s sensitive attributes (e.g. gender) and prevent the re-identification of user. We show how to combine these two algorithms and propose a compound architecture that protects both sensitive activities and attributes. Alongside the algorithmic contributions, we published a motion-sensor dataset for human activity recognition. Secondly, to prevent the identification of users using their web-browsing behavior, we introduce an algorithm for privacy-preserving collaborative training of contextual bandit algorithms. The proposed method improves the accuracy of personalized recommendation agents that run locally on the user’s devices. We propose an encoding algorithm for the user’s web-browsing data that preserves the required information for the personalization of the future contents while ensuring differential privacy for the participants in collaborative training. In addition, for processing multivariate sensor data, we show how to make neural network architectures adaptive to dynamic sampling rate and sensor selection. This allows handling situations in human activity recognition where the dimensions of input data can be varied at inference time. Specifically, we introduce a customized pooling layer for neural networks and propose a customized training procedure to generalize over a large number of feasible data dimensions. Using the proposed architectural improvement, we show how to convert existing non-adaptive deep neural networks into an adaptive network while keeping the same classification accuracy. We conclude this thesis by discussing open questions and the potential future directions for continuing research in this area