Effective Identity Management on Mobile Devices Using Multi-Sensor Measurements

Due to the dramatic increase in popularity of mobile devices in the past decade, sensitive user information is stored and accessed on these devices every day. Securing sensitive data stored and accessed from mobile devices, makes user-identity management a problem of paramount importance. The tension between security and usability renders the task of user-identity verification on mobile devices challenging. Meanwhile, an appropriate identity management approach is missing since most existing technologies for user-identity verification are either one-shot user verification or only work in restricted controlled environments. To solve the aforementioned problems, we investigated and sought approaches from the sensor data generated by human-mobile interactions. The data are collected from the on-board sensors, including voice data from microphone, acceleration data from accelerometer, angular acceleration data from gyroscope, magnetic force data from magnetometer, and multi-touch gesture input data from touchscreen. We studied the feasibility of extracting biometric and behaviour features from the on-board sensor data and how to efficiently employ the features extracted to perform user-identity verification on the smartphone device. Based on the experimental results of the single-sensor modalities, we further investigated how to integrate them with hardware such as fingerprint and Trust Zone to practically fulfill a usable identity management system for both local application and remote services control. User studies and on-device testing sessions were held for privacy and usability evaluation.Computer Science, Department o

Intelligent Techniques to Accelerate Everyday Text Communication

People with some form of speech- or motor-impairments usually use a high-tech augmentative and alternative communication (AAC) device to communicate with other people in writing or in face-to-face conversations. Their text entry rate on these devices is slow due to their motor abilities. Making good letter or word predictions can help accelerate the communication of such users. In this dissertation, we investigated several approaches to accelerate input for AAC users. First, considering that an AAC user is participating in a face-to-face conversation, we investigated whether performing speech recognition on the speaking-side can improve next word predictions. We compared the accuracy of three plausible microphone deployment options and the accuracy of two commercial speech recognition engines. We found that despite recognition word error rates of 7-16%, our ensemble of n-gram and recurrent neural network language models made predictions nearly as good as when they used the reference transcripts. In a user study with 160 participants, we also found that increasing number of prediction slots in a keyboard interface does not necessarily correlate to improved performance. Second, typing every character in a text message may require an AAC user more time or effort than strictly necessary. Skipping spaces or other characters may be able to speed input and reduce an AAC user\u27s physical input effort. We designed a recognizer optimized for expanding noisy abbreviated input where users often omitted spaces and mid-word vowels. We showed using neural language models for selecting conversational-style training text and for rescoring the recognizer\u27s n-best sentences improved accuracy. We found accurate abbreviated input was possible even if a third of characters was omitted. In a study where users had to dwell for a second on each key, we found sentence abbreviated input was competitive with a conventional keyboard with word predictions. Finally, AAC keyboards rely on language modeling to auto-correct noisy typing and to offer word predictions. While today language models can be trained on huge amounts of text, pre-trained models may fail to capture the unique writing style and vocabulary of individual users. We demonstrated improved performance compared to a unigram cache by adapting to a user\u27s text via language models based on prediction by partial match (PPM) and recurrent neural networks. Our best model ensemble increased keystroke savings by 9.6%

PowerSpy: Location Tracking using Mobile Device Power Analysis

Modern mobile platforms like Android enable applications to read aggregate power usage on the phone. This information is considered harmless and reading it requires no user permission or notification. We show that by simply reading the phone's aggregate power consumption over a period of a few minutes an application can learn information about the user's location. Aggregate phone power consumption data is extremely noisy due to the multitude of components and applications that simultaneously consume power. Nevertheless, by using machine learning algorithms we are able to successfully infer the phone's location. We discuss several ways in which this privacy leak can be remedied.Comment: Usenix Security 201

Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality

Virtual reality (VR) headsets are enabling a wide range of new opportunities for the user. For example, in the near future users may be able to visit virtual shopping malls and virtually join international conferences. These and many other scenarios pose new questions with regards to privacy and security, in particular authentication of users within the virtual environment. As a first step towards seamless VR authentication, this paper investigates the direct transfer of well-established concepts (PIN, Android unlock patterns) into VR. In a pilot study (N = 5) and a lab study (N = 25), we adapted existing mechanisms and evaluated their usability and security for VR. The results indicate that both PINs and patterns are well suited for authentication in VR. We found that the usability of both methods matched the performance known from the physical world. In addition, the private visual channel makes authentication harder to observe, indicating that authentication in VR using traditional concepts already achieves a good balance in the trade-off between usability and security. The paper contributes to a better understanding of authentication within VR environments, by providing the first investigation of established authentication methods within VR, and presents the base layer for the design of future authentication schemes, which are used in VR environments only

Privacy in Biometric Systems

Biometrics are physiological and/or behavioral characteristics of a person that have been used to provide an automatic proof of identity in a growing list of applications including crime/terrorism fighting, forensics, access and border control, securing e-/m-commerce transactions and service entitlements. In recent years, a great deal of research into a variety of new and traditional biometrics has widened the scope of investigations beyond improving accuracy into mechanisms that deal with serious concerns raised about the potential misuse of collected biometric data. Despite the long list of biometrics’ benefits, privacy concerns have become widely shared due to the fact that every time the biometric of a person is checked, a trace is left that could reveal personal and confidential information. In fact, biometric-based recognition has an inherent privacy problem as it relies on capturing, analyzing, and storing personal data about us as individuals. For example, biometric systems deal with data related to the way we look (face, iris), the way we walk (gait), the way we talk (speaker recognition), the way we write (handwriting), the way we type on a keyboard (keystroke), the way we read (eye movement), and many more. Privacy has become a serious concern for the public as biometric systems are increasingly deployed in many applications ranging from accessing our account on a Smartphone or computer to border control and national biometric cards on a very large scale. For example, the Unique Identification Authority of India (UIDAI) has issued 56 million biometric cards as of January 2014 [1], where each biometric card holds templates of the 10 fingers, the two irises and the face. An essential factor behind the growing popularity of biometrics in recent years is the fact that biometric sensors have become a lot cheaper as well as easier to install and handle. CCTV cameras are installed nearly everywhere and almost all Smartphones are equipped with a camera, microphone, fingerprint scanner, and probably very soon, an iris scanner