GazeTouchPIN: Protecting Sensitive Data on Mobile Devices Using Secure Multimodal Authentication

Although mobile devices provide access to a plethora of sensitive data, most users still only protect them with PINs or patterns, which are vulnerable to side-channel attacks (e.g., shoulder surfing). How-ever, prior research has shown that privacy-aware users are willing to take further steps to protect their private data. We propose GazeTouchPIN, a novel secure authentication scheme for mobile devices that combines gaze and touch input. Our multimodal approach complicates shoulder-surfing attacks by requiring attackers to ob-serve the screen as well as the user’s eyes to and the password. We evaluate the security and usability of GazeTouchPIN in two user studies (N=30). We found that while GazeTouchPIN requires longer entry times, privacy aware users would use it on-demand when feeling observed or when accessing sensitive data. The results show that successful shoulder surfing attack rate drops from 68% to 10.4%when using GazeTouchPIN

GTmoPass: Two-factor Authentication on Public Displays Using Gaze-touch Passwords and Personal Mobile Devices

As public displays continue to deliver increasingly private and personalized content, there is a need to ensure that only the legitimate users can access private information in sensitive contexts. While public displays can adopt similar authentication concepts like those used on public terminals (e.g., ATMs), authentication in public is subject to a number of risks. Namely, adversaries can uncover a user's password through (1) shoulder surfing, (2) thermal attacks, or (3) smudge attacks. To address this problem we propose GTmoPass, an authentication architecture that enables Multi-factor user authentication on public displays. The first factor is a knowledge-factor: we employ a shoulder-surfing resilient multimodal scheme that combines gaze and touch input for password entry. The second factor is a possession-factor: users utilize their personal mobile devices, on which they enter the password. Credentials are securely transmitted to a server via Bluetooth beacons. We describe the implementation of GTmoPass and report on an evaluation of its usability and security, which shows that although authentication using GTmoPass is slightly slower than traditional methods, it protects against the three aforementioned threats

EyeSpot: leveraging gaze to protect private text content on mobile devices from shoulder surfing

As mobile devices allow access to an increasing amount of private data, using them in public can potentially leak sensitive information through shoulder surfing. This includes personal private data (e.g., in chat conversations) and business-related content (e.g., in emails). Leaking the former might infringe on users’ privacy, while leaking the latter is considered a breach of the EU’s General Data Protection Regulation as of May 2018. This creates a need for systems that protect sensitive data in public. We introduce EyeSpot, a technique that displays content through a spot that follows the user’s gaze while hiding the rest of the screen from an observer’s view through overlaid masks. We explore different configurations for EyeSpot in a user study in terms of users’ reading speed, text comprehension, and perceived workload. While our system is a proof of concept, we identify crystallized masks as a promising design candidate for further evaluation with regard to the security of the system in a shoulder surfing scenario

GazeTouchPass: Multimodal Authentication Using Gaze and Touch on Mobile Devices

We propose a multimodal scheme, GazeTouchPass, that combines gaze and touch for shoulder-surfing resistant user authentication on mobile devices. GazeTouchPass allows passwords with multiple switches between input modalities during authentication. This requires attackers to simultaneously observe the device screen and the user's eyes to find the password. We evaluate the security and usability of GazeTouchPass in two user studies. Our findings show that GazeTouchPass is usable and significantly more secure than single-modal authentication against basic and even advanced shoulder-surfing attacks

Understanding face and eye visibility in front-facing cameras of smartphones used in the wild

Commodity mobile devices are now equipped with high-resolution front-facing cameras, allowing applications in biometrics (e.g., FaceID in the iPhone X), facial expression analysis, or gaze interaction. However, it is unknown how often users hold devices in a way that allows capturing their face or eyes, and how this impacts detection accuracy. We collected 25,726 in-the-wild photos, taken from the front-facing camera of smartphones as well as associated application usage logs. We found that the full face is visible about 29% of the time, and that in most cases the face is only partially visible. Furthermore, we identified an influence of users' current activity; for example, when watching videos, the eyes but not the entire face are visible 75% of the time in our dataset. We found that a state-of-the-art face detection algorithm performs poorly against photos taken from front-facing cameras. We discuss how these findings impact mobile applications that leverage face and eye detection, and derive practical implications to address state-of-the art's limitations

An Investigation of Power Saving and Privacy Protection on Smartphones

With the advancements in mobile technology, smartphones have become ubiquitous in people\u27s daily lives and have greatly facilitated users in many aspects. For a smartphone user, power saving and privacy protection are two important issues that matter and draw serious attentions from research communities. In this dissertation, we present our studies on some specific issues of power saving and privacy protection on a smartphone. Although IEEE 802.11 standards provide Power Save Mode (PSM) to help mobile devices conserve energy, PSM fails to bring expected benefits in many real scenarios. We define an energy conserving model to describe the general PSM traffic contention problem, and propose a solution called HPSM to address one specific case, in which multiple PSM clients associate to a single AP. In HPSM, we first use a basic sociological concept to define the richness of a PSM client based on the link resource it consumes. Then we separate these poor PSM clients from rich PSM clients in terms of link resource consumption, and favor the former to save power when they face PSM transmission contention. Our evaluations show that HPSM can help the poor PSM clients effectively save power while only slightly degrading the rich\u27s performance in comparison to the existing PSM solutions. Traditional user authentication methods using passcode or finger movement on smartphones are vulnerable to shoulder surfing attack, smudge attack, and keylogger attack. These attacks are able to infer a passcode based on the information collection of user\u27s finger movement or tapping input. as an alternative user authentication approach, eye tracking can reduce the risk of suffering those attacks effectively because no hand input is required. We propose a new eye tracking method for user authentication on a smartphone. It utilizes the smartphone\u27s front camera to capture a user\u27s eye movement trajectories which are used as the input of user authentication. No special hardware or calibration process is needed. We develop a prototype and evaluate its effectiveness on an android smartphone. Our evaluation results show that the proposed eye tracking technique achieves very high accuracy in user authentication. While LBS-based apps facilitate users in many application scenarios, they raise concerns on the breach of privacy related to location access. We perform the first measurement of this background action on the Google app market. Our investigation demonstrates that many popular apps conduct location access in background within short intervals. This enables these apps to collect a user\u27s location trace, from which the important personal information, Points of Interest (PoIs), can be recognized. We further extract a user\u27s movement pattern from the PoIs, and utilize it to measure the potential privacy breach. The measurement results also show that using the combination of movement pattern related metrics and the other PoI related metrics can help detect the privacy breach in an earlier manner than using either one of them alone. We then propose a preliminary solution to properly handle these location requests from background

The Role of Eye Gaze in Security and Privacy Applications: Survey and Future HCI Research Directions

For the past 20 years, researchers have investigated the use of eye tracking in security applications. We present a holistic view on gaze-based security applications. In particular, we canvassed the literature and classify the utility of gaze in security applications into a) authentication, b) privacy protection, and c) gaze monitoring during security critical tasks. This allows us to chart several research directions, most importantly 1) conducting field studies of implicit and explicit gaze-based authentication due to recent advances in eye tracking, 2) research on gaze-based privacy protection and gaze monitoring in security critical tasks which are under-investigated yet very promising areas, and 3) understanding the privacy implications of pervasive eye tracking. We discuss the most promising opportunities and most pressing challenges of eye tracking for security that will shape research in gaze-based security applications for the next decade