EyeSpot: leveraging gaze to protect private text content on mobile devices from shoulder surfing

As mobile devices allow access to an increasing amount of private data, using them in public can potentially leak sensitive information through shoulder surfing. This includes personal private data (e.g., in chat conversations) and business-related content (e.g., in emails). Leaking the former might infringe on users’ privacy, while leaking the latter is considered a breach of the EU’s General Data Protection Regulation as of May 2018. This creates a need for systems that protect sensitive data in public. We introduce EyeSpot, a technique that displays content through a spot that follows the user’s gaze while hiding the rest of the screen from an observer’s view through overlaid masks. We explore different configurations for EyeSpot in a user study in terms of users’ reading speed, text comprehension, and perceived workload. While our system is a proof of concept, we identify crystallized masks as a promising design candidate for further evaluation with regard to the security of the system in a shoulder surfing scenario

Proceedings of Cross-Surface 2015: Workshop on Interacting with Multi-Device Ecologies in the Wild

In this workshop, we reviewed and discussed opportunities, technical challenges and problems with cross-device interactions in real world interactive multi-surface and multi-device ecologies. We aim to bring together researchers and practitioners currently working on novel techniques for cross-surface interactions, focusing both on technical as well as interaction challenges for introducing these technologies into the wild, and highlighting opportunities for further research. The workshop will help to facilitate knowledge exchange on the inherent challenges of building robust and intuitive cross-surface interactions, identify application domains and enabling technologies for cross-surface interactions in the wild, and establish a research community to develop effective strategies for successful design of crossdevice interactions. Please find more details about the workshop, in the submitted proposal. The workshop was held in conjunction with the 2015 ACM International Conference on Interactive Tabletops and Surfaces, that took place from November 15 to 18 in Funchal in Madeira, Portugal

The Role of Eye Gaze in Security and Privacy Applications: Survey and Future HCI Research Directions

For the past 20 years, researchers have investigated the use of eye tracking in security applications. We present a holistic view on gaze-based security applications. In particular, we canvassed the literature and classify the utility of gaze in security applications into a) authentication, b) privacy protection, and c) gaze monitoring during security critical tasks. This allows us to chart several research directions, most importantly 1) conducting field studies of implicit and explicit gaze-based authentication due to recent advances in eye tracking, 2) research on gaze-based privacy protection and gaze monitoring in security critical tasks which are under-investigated yet very promising areas, and 3) understanding the privacy implications of pervasive eye tracking. We discuss the most promising opportunities and most pressing challenges of eye tracking for security that will shape research in gaze-based security applications for the next decade

Privacy Intelligence: A Survey on Image Sharing on Online Social Networks

Image sharing on online social networks (OSNs) has become an indispensable part of daily social activities, but it has also led to an increased risk of privacy invasion. The recent image leaks from popular OSN services and the abuse of personal photos using advanced algorithms (e.g. DeepFake) have prompted the public to rethink individual privacy needs when sharing images on OSNs. However, OSN image sharing itself is relatively complicated, and systems currently in place to manage privacy in practice are labor-intensive yet fail to provide personalized, accurate and flexible privacy protection. As a result, an more intelligent environment for privacy-friendly OSN image sharing is in demand. To fill the gap, we contribute a systematic survey of 'privacy intelligence' solutions that target modern privacy issues related to OSN image sharing. Specifically, we present a high-level analysis framework based on the entire lifecycle of OSN image sharing to address the various privacy issues and solutions facing this interdisciplinary field. The framework is divided into three main stages: local management, online management and social experience. At each stage, we identify typical sharing-related user behaviors, the privacy issues generated by those behaviors, and review representative intelligent solutions. The resulting analysis describes an intelligent privacy-enhancing chain for closed-loop privacy management. We also discuss the challenges and future directions existing at each stage, as well as in publicly available datasets.Comment: 32 pages, 9 figures. Under revie

Applications across Co-located Devices

We live surrounded by many computing devices. However, their presence has yet to be fully explored to create a richer ubiquitous computing environment. There is an opportunity to take better advantage of those devices by combining them into a unified user experience. To realize this vision, we studied and explored the use of a framework, which provides the tools and abstractions needed to develop applications that distribute UI components across co-located devices. The framework comprises the following components: authentication and authorization services; a broker to sync information across multiple application instances; background services that gather the capabilities of the devices; and a library to integrate web applications with the broker, determine which components to show based on UI requirements and device capabilities, and that provides custom elements to manage the distribution of the UI components and the multiple application states. Collaboration between users is supported by sharing application states. An indoor positioning solution had to be developed in order to determine when devices are close to each other to trigger the automatic redistribution of UI components. The research questions that we set out to respond are presented along with the contributions that have been produced. Those contributions include a framework for crossdevice applications, an indoor positioning solution for pervasive indoor environments, prototypes, end-user studies and developer focused evaluation. To contextualize our research, we studied previous research work about cross-device applications, proxemic interactions and indoor positioning systems. We presented four application prototypes. The first three were used to perform studies to evaluate the user experience. The last one was used to study the developer experience provided by the framework. The results were largely positive with users showing preference towards using multiple devices under some circumstances. Developers were also able to grasp the concepts provided by the framework relatively well.Vivemos rodeados de dispositivos computacionais. No entanto, ainda não tiramos partido da sua presença para criar ambientes de computação ubíqua mais ricos. Existe uma oportunidade de combiná-los para criar uma experiência de utilizador unificada. Para realizar esta visão, estudámos e explorámos a utilização de uma framework que forneça ferramentas e abstrações que permitam o desenvolvimento de aplicações que distribuem os componentes da interface do utilizador por dispositivos co-localizados. A framework é composta por: serviços de autenticação e autorização; broker que sincroniza informação entre várias instâncias da aplicação; serviços que reúnem as capacidades dos dispositivos; e uma biblioteca para integrar aplicações web com o broker, determinar as componentes a mostrar com base nos requisitos da interface e nas capacidades dos dispositivos, e que disponibiliza elementos para gerir a distribuição dos componentes da interface e dos estados de aplicação. A colaboração entre utilizadores é suportada através da partilha dos estados de aplicação. Foi necessário desenvolver um sistema de posicionamento em interiores para determinar quando é que os dispositivos estão perto uns dos outros para despoletar a redistribuição automática dos componentes da interface. As questões de investigação inicialmente colocadas são apresentadas juntamente com as contribuições que foram produzidas. Essas contribuições incluem uma framework para aplicações multi-dispositivo, uma solução de posicionamento em interiores para computação ubíqua, protótipos, estudos com utilizadores finais e avaliação com programadores. Para contextualizar a nossa investigação, estudámos trabalhos anteriores sobre aplicações multi-dispositivo, interação proxémica e sistemas de posicionamento em interiores. Apresentámos quatro aplicações protótipo. As primeiras três foram utilizadas para avaliar a experiência de utilização. A última foi utilizada para estudar a experiência de desenvolvimento com a framework. Os resultados foram geralmente positivos, com os utilizadores a preferirem utilizar múltiplos dispositivos em certas circunstâncias. Os programadores também foram capazes de compreender a framework relativamente bem

Privacy For Whom? A Multi-Stakeholder Exploration of Privacy Designs

Privacy is considered one of the fundamental human rights. Researchers have been investigating privacy issues in various domains, such as our physical privacy, data privacy, privacy as a legal right, and privacy designs. In the Human-Computer Interaction field, privacy researchers have been focusing on understanding people\u27s privacy concerns when they interact with computing systems, designing and building privacy-enhancing technologies to help people mitigate these concerns, and investigating how people\u27s privacy perceptions and the privacy designs influence people\u27s behaviors. Existing privacy research has been overwhelmingly focusing on the privacy needs of end-users, i.e., people who use a system or a product, such as Internet users and smartphone users. However, as our computing systems are becoming more and more complex, privacy issues within these systems have started to impact not only the end-users but also other stakeholders, and privacy-enhancing mechanisms designed for the end-users can also affect multiple stakeholders beyond the users. In this dissertation, I examine how different stakeholders perceive privacy-related issues and expect privacy designs to function across three application domains: online behavioral advertising, drones, and smart homes. I choose these three domains because they represent different multi-stakeholder environments with varying nature of complexity. In particular, these environments present the opportunities to study technology-mediated interpersonal relationships, i.e., the relationship between primary users (owners, end-users) and secondary users (bystanders), and to investigate how these relationships influence people\u27s privacy perceptions and their desired ways of privacy protection. Through a combination of qualitative, quantitative, and design methods, including interviews, surveys, participatory designs, and speculative designs, I present how multi-stakeholder considerations change our understandings of privacy and influence privacy designs. I draw design implications from the study results and guide future privacy designs to consider the needs of different stakeholders, e.g., cooperative mechanisms that aim to enhance the communication between primary and secondary users. In addition, this methodological approach allows researchers to directly and proactively engage with multiple stakeholders and explore their privacy perceptions and expected privacy designs. This is different from what has been commonly used in privacy literature and as such, points to a methodological contribution. Finally, this dissertation shows that when applying the theory of Contextual Integrity in a multi-stakeholder environment, there are hidden contextual factors that may alter the contextual informational norms. I present three examples from the study results and argue that it is necessary to carefully examine such factors in order to clearly identify the contextual norms. I propose a research agenda to explore best practices of applying the theory of Contextual Integrity in a multi-stakeholder environment

Proceedings of Cross-Surface 2016: Workshop on Challenges and Opportunities for 'Bring-Your-Own-Device' (BYOD) in the Wild

In this workshop, we reviewed and discussed challenges and opportunities for Human-Computer Interaction in relation to cross-surface interaction in the wild based on the bring-your-own-device (BYOD) practice. We brought together researchers and practitioners working on technical infrastructures for cross-surface computing, studies of cross-surface computing in particular domains as well as interaction challenges for introducing cross-surface computing in the wild, all with a particular focus on BYOD. Examples of application domains are: cultural institutions, work places, public libraries, schools and education. Please find more details about the workshop, in the submitted proposal [1]. The workshop was held in conjunction with the 2016 ACM Conference on Human Factors in Computing Systems (CHI), that took place from May 7 to 12 in San Jose, USA. [1] Steven Houben, Nicolai Marquardt, Jo Vermeulen, Johannes Schöning, Clemens Klokmose, Harald Reiterer, Henrik Korsgaard, and Mario Schreiner. 2016. Cross-Surface: Challenges and Opportunities for 'bring your own device' in the wild

Using natural user interfaces to support synchronous distributed collaborative work

Synchronous Distributed Collaborative Work (SDCW) occurs when group members work together at the same time from different places together to achieve a common goal. Effective SDCW requires good communication, continuous coordination and shared information among group members. SDCW is possible because of groupware, a class of computer software systems that supports group work. Shared-workspace groupware systems are systems that provide a common workspace that aims to replicate aspects of a physical workspace that is shared among group members in a co-located environment. Shared-workspace groupware systems have failed to provide the same degree of coordination and awareness among distributed group members that exists in co-located groups owing to unintuitive interaction techniques that these systems have incorporated. Natural User Interfaces (NUIs) focus on reusing natural human abilities such as touch, speech, gestures and proximity awareness to allow intuitive human-computer interaction. These interaction techniques could provide solutions to the existing issues of groupware systems by breaking down the barrier between people and technology created by the interaction techniques currently utilised. The aim of this research was to investigate how NUI interaction techniques could be used to effectively support SDCW. An architecture for such a shared-workspace groupware system was proposed and a prototype, called GroupAware, was designed and developed based on this architecture. GroupAware allows multiple users from distributed locations to simultaneously view and annotate text documents, and create graphic designs in a shared workspace. Documents are represented as visual objects that can be manipulated through touch gestures. Group coordination and awareness is maintained through document updates via immediate workspace synchronization, user action tracking via user labels and user availability identification via basic proxemic interaction. Members can effectively communicate via audio and video conferencing. A user study was conducted to evaluate GroupAware and determine whether NUI interaction techniques effectively supported SDCW. Ten groups of three members each participated in the study. High levels of performance, user satisfaction and collaboration demonstrated that GroupAware was an effective groupware system that was easy to learn and use, and effectively supported group work in terms of communication, coordination and information sharing. Participants gave highly positive comments about the system that further supported the results. The successful implementation of GroupAware and the positive results obtained from the user evaluation provides evidence that NUI interaction techniques can effectively support SDCW