Is Anyone Looking? Mitigating Shoulder Surfing on Public Displays through Awareness and Protection

Displays are growing in size, and are increasingly deployed in semi-public and public areas. When people use these public displays to pursue personal work, they expose their activities and sensitive data to passers-by. In most cases, such shoulder-surfing by others is likely voyeuristic vs. a deliberate attempt to steal information. Even so, safeguards are needed. Our goal is to mitigate shoulder-surfing problems in such settings. Our method leverages notions of territoriality and proxemics, where we sense and take action based on the spatial relationships between the passerby, the user of the display, and the display itself. First, we provide participants with awareness of shoulder-surfing moments, which in turn helps both parties regulate their behaviours and mediate further social interactions. Second, we provide methods that protect information when shoulder-surfing is detected. Here, users can move or hide information through easy to perform explicit actions. Alternately, the system itself can mask information from the passerby’s view when it detects shoulder-surfing moments

The Challenges of Using an Existing Cross-Device Interaction Prototype for Supporting Actual Curation Practices

Volunteer-driven organisations curating historic documents, such as societies and charities, often work within a bring-your-own-device (BYOD) practice and their meetings are in varying situations. A recurring challenge is finding lightweight ways to enable them to share and collectively work with documents using their own devices while in situ. We are working on building novel interaction techniques and applications (prototyped with a custom developer toolkit) for supporting the curation of digital collections – for example, historic documents. We discuss the pros and cons of using an existing prototype system for this purpose and points to consider when taking a prototype from the lab into the wild

GTmoPass: Two-factor Authentication on Public Displays Using Gaze-touch Passwords and Personal Mobile Devices

As public displays continue to deliver increasingly private and personalized content, there is a need to ensure that only the legitimate users can access private information in sensitive contexts. While public displays can adopt similar authentication concepts like those used on public terminals (e.g., ATMs), authentication in public is subject to a number of risks. Namely, adversaries can uncover a user's password through (1) shoulder surfing, (2) thermal attacks, or (3) smudge attacks. To address this problem we propose GTmoPass, an authentication architecture that enables Multi-factor user authentication on public displays. The first factor is a knowledge-factor: we employ a shoulder-surfing resilient multimodal scheme that combines gaze and touch input for password entry. The second factor is a possession-factor: users utilize their personal mobile devices, on which they enter the password. Credentials are securely transmitted to a server via Bluetooth beacons. We describe the implementation of GTmoPass and report on an evaluation of its usability and security, which shows that although authentication using GTmoPass is slightly slower than traditional methods, it protects against the three aforementioned threats

EyeSpot: leveraging gaze to protect private text content on mobile devices from shoulder surfing

As mobile devices allow access to an increasing amount of private data, using them in public can potentially leak sensitive information through shoulder surfing. This includes personal private data (e.g., in chat conversations) and business-related content (e.g., in emails). Leaking the former might infringe on users’ privacy, while leaking the latter is considered a breach of the EU’s General Data Protection Regulation as of May 2018. This creates a need for systems that protect sensitive data in public. We introduce EyeSpot, a technique that displays content through a spot that follows the user’s gaze while hiding the rest of the screen from an observer’s view through overlaid masks. We explore different configurations for EyeSpot in a user study in terms of users’ reading speed, text comprehension, and perceived workload. While our system is a proof of concept, we identify crystallized masks as a promising design candidate for further evaluation with regard to the security of the system in a shoulder surfing scenario

Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as Touchpads

In this paper, we present a multi-user interaction interface for a large immersive space that supports simultaneous screen interactions by combining (1) user input via personal smartphones and Bluetooth microphones, (2) spatial tracking via an overhead array of Kinect sensors, and (3) WebSocket interfaces to a webpage running on the large screen. Users are automatically, dynamically assigned personal and shared screen sub-spaces based on their tracked location with respect to the screen, and use a webpage on their personal smartphone for touchpad-type input. We report user experiments using our interaction framework that involve image selection and placement tasks, with the ultimate goal of realizing display-wall environments as viable, interactive workspaces with natural multimodal interfaces.Comment: 8 pages with reference

Understanding Shoulder Surfing in the Wild: Stories from Users and Observers

Research has brought forth a variety of authentication systems to mitigate observation attacks. However, there is little work about shoulder surfing situations in the real world. We present the results of a user survey (N=174) in which we investigate actual stories about shoulder surfing on mobile devices from both users and observers. Our analysis indicates that shoulder surfing mainly occurs in an opportunistic, non-malicious way. It usually does not have serious consequences, but evokes negative feelings for both parties, resulting in a variety of coping strategies. Observed data was personal in most cases and ranged from information about interests and hobbies to login data and intimate details about third persons and relationships. Thus, our work contributes evidence for shoulder surfing in the real world and informs implications for the design of privacy protection mechanisms

Sens-Us: Designing Innovative Civic Technology for the Public Good

How can civic technology be designed to encourage more public engagement? What new methods of data collection and sharing can be used to engender a different relationship between citizens and the state? One approach has been to design physical systems that draw people in and which they can trust, leading them to give their views, opinions or other data. So far, they have been largely used to elicit feedback or votes for one or two questions about a given topic. Here, we describe a physical system, called Sens-Us, which was designed to ask a range of questions about personal and sensitive information, within the context of rethinking the UK Census. An in-the-wild study of its deployment in a city cultural center showed how a diversity of people approached, answered and compared the data that had been collected about themselves with others. We discuss the findings in relation to the pros and cons of using this kind of innovative technology when wanting to promote civic engagement or other forms of public engagement

LightTouch: Securely Connecting Wearables to Ambient Displays with User Intent

Wearables are small and have limited user interfaces, so they often wirelessly interface with a personal smartphone/computer to relay information from the wearable for display or other interactions. In this paper, we envision a new method, LightTouch, by which a wearable can establish a secure connection to an ambient display, such as a television or a computer monitor, while ensuring the user\u27s intention to connect to the display. LightTouch uses standard RF methods (like Bluetooth) for communicating the data to display, securely bootstrapped via the visible-light communication (the brightness channel) from the display to the low-cost, low-power, ambient light sensor of a wearable. A screen `touch\u27 gesture is adopted by users to ensure that the modulation of screen brightness can be securely captured by the ambient light sensor with minimized noise. Wireless coordination with the processor driving the display establishes a shared secret based on the brightness channel information. We further propose novel on-screen localization and correlation algorithms to improve security and reliability. Through experiments and a preliminary user study we demonstrate that LightTouch is compatible with current display and wearable designs, is easy to use (about 6 seconds to connect), is reliable (up to 98\% success connection ratio), and is secure against attacks

Situational Awareness and Public Wi-Fi Users\u27 Self-Protective Behaviors

Accessing public Wi-Fi networks can be as dangerous as it is convenient. People who access a public Wi-Fi network should engage in self-protective behaviors to keep their data safe from malicious actors on the same network as well as persons looking over their shoulder, literally and proverbially. Using two independent research designs, we examined under what circumstances were people more likely to access an unsecured Wi-Fi network and engage in risky behavior on these networks. Findings from the first study, based on survey data, reveal that people who are more situationally aware are less likely to access personal accounts on public Wi-Fi, and more likely to cover their screen to prevent others from viewing personal information. Additionally, findings show that people with higher computer proficiencies are less likely to engage with public Wi-Fi. For the second study, our research team designed and deployed honeypot Wi-Fi networks. We found that people are more likely to access these unsecured, rogue networks in establishments with fewer on-duty employees and that do not offer legitimate public Wi-Fi. Additionally, the number of on-duty employees is associated with an increase in physical security behaviors, such as concealing a screen. We conclude by discussing how these findings can aid in reducing susceptibility to online victimization

Increasing Passersby Engagement with Public Large Interactive Displays: A Study of Proxemics and Conation

This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in the Proceedings of the 2018 International Conference on Human Factors on Computing Systems on the ACM Digital Library at https://dx.doi.org/10.1145/3279778.3279789Prior research has shown that large interactive displays de- ployed in public spaces are often underutilized, or even un- noticed, phenomena connected to ‘interaction’ and ‘display blindness’, respectively. To better understand how designers can mitigate these issues, we conducted a field experiment that compared how different visual cues impacted engagement with a public display. The deployed interfaces were designed to progressively reveal more information about the display and entice interaction through the use of visual content designed to evoke direct or indirect conation (the mental faculty related to purpose or will to perform an action), and different ani- mation triggers (random or proxemic). Our results show that random triggers were more effective than proxemic triggers at overcoming display and interaction blindness. Our study of conation – the first we are aware of – found that “conceptual” visuals designed to evoke indirect conation were also useful in attracting people’s attention.Natural Sciences and Engineering Research Council of Canad