Using the Android Tablet to develop a game platform for older adults

Tese de mestrado integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201

Software Usability

This volume delivers a collection of high-quality contributions to help broaden developers’ and non-developers’ minds alike when it comes to considering software usability. It presents novel research and experiences and disseminates new ideas accessible to people who might not be software makers but who are undoubtedly software users

How WEIRD is Usable Privacy and Security Research? (Extended Version)

In human factor fields such as human-computer interaction (HCI) and psychology, researchers have been concerned that participants mostly come from WEIRD (Western, Educated, Industrialized, Rich, and Democratic) countries. This WEIRD skew may hinder understanding of diverse populations and their cultural differences. The usable privacy and security (UPS) field has inherited many research methodologies from research on human factor fields. We conducted a literature review to understand the extent to which participant samples in UPS papers were from WEIRD countries and the characteristics of the methodologies and research topics in each user study recruiting Western or non-Western participants. We found that the skew toward WEIRD countries in UPS is greater than that in HCI. Geographic and linguistic barriers in the study methods and recruitment methods may cause researchers to conduct user studies locally. In addition, many papers did not report participant demographics, which could hinder the replication of the reported studies, leading to low reproducibility. To improve geographic diversity, we provide the suggestions including facilitate replication studies, address geographic and linguistic issues of study/recruitment methods, and facilitate research on the topics for non-WEIRD populations.Comment: This paper is the extended version of the paper presented at USENIX SECURITY 202

Risks and potentials of graphical and gesture-based authentication for touchscreen mobile devices

While a few years ago, mobile phones were mainly used for making phone calls and texting short messages, the functionality of mobile devices has massively grown. We are surfing the web, sending emails and we are checking our bank accounts on the go. As a consequence, these internet-enabled devices store a lot of potentially sensitive data and require enhanced protection. We argue that authentication often represents the only countermeasure to protect mobile devices from unwanted access. Knowledge-based concepts (e.g., PIN) are the most used authentication schemes on mobile devices. They serve as the main protection barrier for many users and represent the fallback solution whenever alternative mechanisms fail (e.g., fingerprint recognition). This thesis focuses on the risks and potentials of gesture-based authentication concepts that particularly exploit the touch feature of mobile devices. The contribution of our work is threefold. Firstly, the problem space of mobile authentication is explored. Secondly, the design space is systematically evaluated utilizing interactive prototypes. Finally, we provide generalized insights into the impact of specific design factors and present recommendations for the design and the evaluation of graphical gesture-based authentication mechanisms. The problem space exploration is based on four research projects that reveal important real-world issues of gesture-based authentication on mobile devices. The first part focuses on authentication behavior in the wild and shows that the mobile context makes great demands on the usability of authentication concepts. The second part explores usability features of established concepts and indicates that gesture-based approaches have several benefits in the mobile context. The third part focuses on observability and presents a prediction model for the vulnerability of a given grid-based gesture. Finally, the fourth part investigates the predictability of user-selected gesture-based secrets. The design space exploration is based on a design-oriented research approach and presents several practical solutions to existing real-world problems. The novel authentication mechanisms are implemented into working prototypes and evaluated in the lab and the field. In the first part, we discuss smudge attacks and present alternative authentication concepts that are significantly more secure against such attacks. The second part focuses on observation attacks. We illustrate how relative touch gestures can support eyes-free authentication and how they can be utilized to make traditional PIN-entry secure against observation attacks. The third part addresses the problem of predictable gesture choice and presents two concepts which nudge users to select a more diverse set of gestures. Finally, the results of the basic research and the design-oriented applied research are combined to discuss the interconnection of design space and problem space. We contribute by outlining crucial requirements for mobile authentication mechanisms and present empirically proven objectives for future designs. In addition, we illustrate a systematic goal-oriented development process and provide recommendations for the evaluation of authentication on mobile devices.Während Mobiltelefone vor einigen Jahren noch fast ausschließlich zum Telefonieren und zum SMS schreiben genutzt wurden, sind die Anwendungsmöglichkeiten von Mobilgeräten in den letzten Jahren erheblich gewachsen. Wir surfen unterwegs im Netz, senden E-Mails und überprüfen Bankkonten. In der Folge speichern moderne internetfähigen Mobilgeräte eine Vielfalt potenziell sensibler Daten und erfordern einen erhöhten Schutz. In diesem Zusammenhang stellen Authentifizierungsmethoden häufig die einzige Möglichkeit dar, um Mobilgeräte vor ungewolltem Zugriff zu schützen. Wissensbasierte Konzepte (bspw. PIN) sind die meistgenutzten Authentifizierungssysteme auf Mobilgeräten. Sie stellen für viele Nutzer den einzigen Schutzmechanismus dar und dienen als Ersatzlösung, wenn alternative Systeme (bspw. Fingerabdruckerkennung) versagen. Diese Dissertation befasst sich mit den Risiken und Potenzialen gestenbasierter Konzepte, welche insbesondere die Touch-Funktion moderner Mobilgeräte ausschöpfen. Der wissenschaftliche Beitrag dieser Arbeit ist vielschichtig. Zum einen wird der Problemraum mobiler Authentifizierung erforscht. Zum anderen wird der Gestaltungsraum anhand interaktiver Prototypen systematisch evaluiert. Schließlich stellen wir generelle Einsichten bezüglich des Einflusses bestimmter Gestaltungsaspekte dar und geben Empfehlungen für die Gestaltung und Bewertung grafischer gestenbasierter Authentifizierungsmechanismen. Die Untersuchung des Problemraums basiert auf vier Forschungsprojekten, welche praktische Probleme gestenbasierter Authentifizierung offenbaren. Der erste Teil befasst sich mit dem Authentifizierungsverhalten im Alltag und zeigt, dass der mobile Kontext hohe Ansprüche an die Benutzerfreundlichkeit eines Authentifizierungssystems stellt. Der zweite Teil beschäftigt sich mit der Benutzerfreundlichkeit etablierter Methoden und deutet darauf hin, dass gestenbasierte Konzepte vor allem im mobilen Bereich besondere Vorzüge bieten. Im dritten Teil untersuchen wir die Beobachtbarkeit gestenbasierter Eingabe und präsentieren ein Vorhersagemodell, welches die Angreifbarkeit einer gegebenen rasterbasierten Geste abschätzt. Schließlich beschäftigen wir uns mit der Erratbarkeit nutzerselektierter Gesten. Die Untersuchung des Gestaltungsraums basiert auf einem gestaltungsorientierten Forschungsansatz, welcher zu mehreren praxisgerechte Lösungen führt. Die neuartigen Authentifizierungskonzepte werden als interaktive Prototypen umgesetzt und in Labor- und Feldversuchen evaluiert. Im ersten Teil diskutieren wir Fettfingerattacken ("smudge attacks") und präsentieren alternative Authentifizierungskonzepte, welche effektiv vor diesen Angriffen schützen. Der zweite Teil beschäftigt sich mit Angriffen durch Beobachtung und verdeutlicht wie relative Gesten dazu genutzt werden können, um blickfreie Authentifizierung zu gewährleisten oder um PIN-Eingaben vor Beobachtung zu schützen. Der dritte Teil beschäftigt sich mit dem Problem der vorhersehbaren Gestenwahl und präsentiert zwei Konzepte, welche Nutzer dazu bringen verschiedenartige Gesten zu wählen. Die Ergebnisse der Grundlagenforschung und der gestaltungsorientierten angewandten Forschung werden schließlich verknüpft, um die Verzahnung von Gestaltungsraum und Problemraum zu diskutieren. Wir präsentieren wichtige Anforderungen für mobile Authentifizierungsmechanismen und erläutern empirisch nachgewiesene Zielvorgaben für zukünftige Konzepte. Zusätzlich zeigen wir einen zielgerichteten Entwicklungsprozess auf, welcher bei der Entwicklung neuartiger Konzepte helfen wird und geben Empfehlungen für die Evaluation mobiler Authentifizierungsmethoden

Digital Traces of the Mind::Using Smartphones to Capture Signals of Well-Being in Individuals

General context and questions Adolescents and young adults typically use their smartphone several hours a day. Although there are concerns about how such behaviour might affect their well-being, the popularity of these powerful devices also opens novel opportunities for monitoring well-being in daily life. If successful, monitoring well-being in daily life provides novel opportunities to develop future interventions that provide personalized support to individuals at the moment they require it (just-in-time adaptive interventions). Taking an interdisciplinary approach with insights from communication, computational, and psychological science, this dissertation investigated the relation between smartphone app use and well-being and developed machine learning models to estimate an individual’s well-being based on how they interact with their smartphone. To elucidate the relation between smartphone trace data and well-being and to contribute to the development of technologies for monitoring well-being in future clinical practice, this dissertation addressed two overarching questions:RQ1: Can we find empirical support for theoretically motivated relations between smartphone trace data and well-being in individuals? RQ2: Can we use smartphone trace data to monitor well-being in individuals?Aims The first aim of this dissertation was to quantify the relation between the collected smartphone trace data and momentary well-being at the sample level, but also for each individual, following recent conceptual insights and empirical findings in psychological, communication, and computational science. A strength of this personalized (or idiographic) approach is that it allows us to capture how individuals might differ in how smartphone app use is related to their well-being. Considering such interindividual differences is important to determine if some individuals might potentially benefit from spending more time on their smartphone apps whereas others do not or even experience adverse effects. The second aim of this dissertation was to develop models for monitoring well-being in daily life. The present work pursued this transdisciplinary aim by taking a machine learning approach and evaluating to what extent we might estimate an individual’s well-being based on their smartphone trace data. If such traces can be used for this purpose by helping to pinpoint when individuals are unwell, they might be a useful data source for developing future interventions that provide personalized support to individuals at the moment they require it (just-in-time adaptive interventions). With this aim, the dissertation follows current developments in psychoinformatics and psychiatry, where much research resources are invested in using smartphone traces and similar data (obtained with smartphone sensors and wearables) to develop technologies for detecting whether an individual is currently unwell or will be in the future. Data collection and analysis This work combined novel data collection techniques (digital phenotyping and experience sampling methodology) for measuring smartphone use and well-being in the daily lives of 247 student participants. For a period up to four months, a dedicated application installed on participants’ smartphones collected smartphone trace data. In the same time period, participants completed a brief smartphone-based well-being survey five times a day (for 30 days in the first month and 30 days in the fourth month; up to 300 assessments in total). At each measurement, this survey comprised questions about the participants’ momentary level of procrastination, stress, and fatigue, while sleep duration was measured in the morning. Taking a time-series and machine learning approach to analysing these data, I provide the following contributions: Chapter 2 investigates the person-specific relation between passively logged usage of different application types and momentary subjective procrastination, Chapter 3 develops machine learning methodology to estimate sleep duration using smartphone trace data, Chapter 4 combines machine learning and explainable artificial intelligence to discover smartphone-tracked digital markers of momentary subjective stress, Chapter 5 uses a personalized machine learning approach to evaluate if smartphone trace data contains behavioral signs of fatigue. Collectively, these empirical studies provide preliminary answers to the overarching questions of this dissertation.Summary of results With respect to the theoretically motivated relations between smartphone trace data and wellbeing (RQ1), we found that different patterns in smartphone trace data, from time spent on social network, messenger, video, and game applications to smartphone-tracked sleep proxies, are related to well-being in individuals. The strength and nature of this relation depends on the individual and app usage pattern under consideration. The relation between smartphone app use patterns and well-being is limited in most individuals, but relatively strong in a minority. Whereas some individuals might benefit from using specific app types, others might experience decreases in well-being when spending more time on these apps. With respect to the question whether we might use smartphone trace data to monitor well-being in individuals (RQ2), we found that smartphone trace data might be useful for this purpose in some individuals and to some extent. They appear most relevant in the context of sleep monitoring (Chapter 3) and have the potential to be included as one of several data sources for monitoring momentary procrastination (Chapter 2), stress (Chapter 4), and fatigue (Chapter 5) in daily life. Outlook Future interdisciplinary research is needed to investigate whether the relationship between smartphone use and well-being depends on the nature of the activities performed on these devices, the content they present, and the context in which they are used. Answering these questions is essential to unravel the complex puzzle of developing technologies for monitoring well-being in daily life.<br/

Enhancing interaction in mixed reality

With continuous technological innovation, we observe mixed reality emerging from research labs into the mainstream. The arrival of capable mixed reality devices transforms how we are entertained, consume information, and interact with computing systems, with the most recent being able to present synthesized stimuli to any of the human senses and substantially blur the boundaries between the real and virtual worlds. In order to build expressive and practical mixed reality experiences, designers, developers, and stakeholders need to understand and meet its upcoming challenges. This research contributes a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. We present the results of seven studies examining the challenges and opportunities of mixed reality experiences, the impact of modalities and interaction techniques on the user experience, and how to enhance the experiences. We begin with a study determining user attitudes towards mixed reality in domestic and educational environments, followed by six research probes that each investigate an aspect of reality or virtuality. In the first, a levitating steerable projector enables us to investigate how the real world can be enhanced without instrumenting the user. We show that the presentation of in-situ instructions for navigational tasks leads to a significantly higher ability to observe and recall real-world landmarks. With the second probe, we enhance the perception of reality by superimposing information usually not visible to the human eye. In amplifying the human vision, we enable users to perceive thermal radiation visually. Further, we examine the effect of substituting physical components with non-functional tangible proxies or entirely virtual representations. With the third research probe, we explore how to enhance virtuality to enable a user to input text on a physical keyboard while being immersed in the virtual world. Our prototype tracked the user’s hands and keyboard to enable generic text input. Our analysis of text entry performance showed the importance and effect of different hand representations. We then investigate how to touch virtuality by simulating generic haptic feedback for virtual reality and show how tactile feedback through quadcopters can significantly increase the sense of presence. Our final research probe investigates the usability and input space of smartphones within mixed reality environments, pairing the user’s smartphone as an input device with a secondary physical screen. Based on our learnings from these individual research probes, we developed a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. The taxonomy is based on the human sensory system and human capabilities of articulation. We showcased its versatility and set our research probes into perspective by organizing them inside the taxonomic space. The design guidelines are divided into user-centered and technology-centered. It is our hope that these will contribute to the bright future of mixed reality systems while emphasizing the new underlining interaction paradigm.Mixed Reality (vermischte Realitäten) gehen aufgrund kontinuierlicher technologischer Innovationen langsam von der reinen Forschung in den Massenmarkt über. Mit der Einführung von leistungsfähigen Mixed-Reality-Geräten verändert sich die Art und Weise, wie wir Unterhaltungsmedien und Informationen konsumieren und wie wir mit Computersystemen interagieren. Verschiedene existierende Geräte sind in der Lage, jeden der menschlichen Sinne mit synthetischen Reizen zu stimulieren. Hierdurch verschwimmt zunehmend die Grenze zwischen der realen und der virtuellen Welt. Um eindrucksstarke und praktische Mixed-Reality-Erfahrungen zu kreieren, müssen Designer und Entwicklerinnen die künftigen Herausforderungen und neuen Möglichkeiten verstehen. In dieser Dissertation präsentieren wir eine neue Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen. Wir stellen die Ergebnisse von sieben Studien vor, in denen die Herausforderungen und Chancen von Mixed-Reality-Erfahrungen, die Auswirkungen von Modalitäten und Interaktionstechniken auf die Benutzererfahrung und die Möglichkeiten zur Verbesserung dieser Erfahrungen untersucht werden. Wir beginnen mit einer Studie, in der die Haltung der nutzenden Person gegenüber Mixed Reality in häuslichen und Bildungsumgebungen analysiert wird. In sechs weiteren Fallstudien wird jeweils ein Aspekt der Realität oder Virtualität untersucht. In der ersten Fallstudie wird mithilfe eines schwebenden und steuerbaren Projektors untersucht, wie die Wahrnehmung der realen Welt erweitert werden kann, ohne dabei die Person mit Technologie auszustatten. Wir zeigen, dass die Darstellung von in-situ-Anweisungen für Navigationsaufgaben zu einer deutlich höheren Fähigkeit führt, Sehenswürdigkeiten der realen Welt zu beobachten und wiederzufinden. In der zweiten Fallstudie erweitern wir die Wahrnehmung der Realität durch Überlagerung von Echtzeitinformationen, die für das menschliche Auge normalerweise unsichtbar sind. Durch die Erweiterung des menschlichen Sehvermögens ermöglichen wir den Anwender:innen, Wärmestrahlung visuell wahrzunehmen. Darüber hinaus untersuchen wir, wie sich das Ersetzen von physischen Komponenten durch nicht funktionale, aber greifbare Replikate oder durch die vollständig virtuelle Darstellung auswirkt. In der dritten Fallstudie untersuchen wir, wie virtuelle Realitäten verbessert werden können, damit eine Person, die in der virtuellen Welt verweilt, Text auf einer physischen Tastatur eingeben kann. Unser Versuchsdemonstrator detektiert die Hände und die Tastatur, zeigt diese in der vermischen Realität an und ermöglicht somit die verbesserte Texteingaben. Unsere Analyse der Texteingabequalität zeigte die Wichtigkeit und Wirkung verschiedener Handdarstellungen. Anschließend untersuchen wir, wie man Virtualität berühren kann, indem wir generisches haptisches Feedback für virtuelle Realitäten simulieren. Wir zeigen, wie Quadrokopter taktiles Feedback ermöglichen und dadurch das Präsenzgefühl deutlich steigern können. Unsere letzte Fallstudie untersucht die Benutzerfreundlichkeit und den Eingaberaum von Smartphones in Mixed-Reality-Umgebungen. Hierbei wird das Smartphone der Person als Eingabegerät mit einem sekundären physischen Bildschirm verbunden, um die Ein- und Ausgabemodalitäten zu erweitern. Basierend auf unseren Erkenntnissen aus den einzelnen Fallstudien haben wir eine neuartige Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen entwickelt. Die Taxonomie basiert auf dem menschlichen Sinnessystem und den Artikulationsfähigkeiten. Wir stellen die vielseitige Verwendbarkeit vor und setzen unsere Fallstudien in Kontext, indem wir sie innerhalb des taxonomischen Raums einordnen. Die Gestaltungsrichtlinien sind in nutzerzentrierte und technologiezentrierte Richtlinien unterteilt. Es ist unsere Anliegen, dass diese Gestaltungsrichtlinien zu einer erfolgreichen Zukunft von Mixed-Reality-Systemen beitragen und gleichzeitig die neuen Interaktionsparadigmen hervorheben

Quality of experience in affective pervasive environments

The confluence of miniaturised powerful devices, widespread communication networks and mass remote storage has caused a fundamental shift in the user interaction design paradigm. The distinction between system and user in pervasive environments is evolving into an increasingly integrated loop of interaction, raising a number of opportunities to provide enhanced and personalised experiences. We propose a platform, based on a smart architecture, to address the identified opportunities in pervasive computing. Smart systems aim at acting upon an environment for improving quality of experience: a subjective measure that has been defined as an emotional reaction to products or services. The inclusion of an emotional dimension allows us to measure individual user responses and deliver personalised services with the potential to influence experiences positively. The platform, Cloud2Bubble, leverages pervasive systems to aggregate user and environment data with the goal of addressing personal preferences and supra-functional requirements. This, combined with its societal implications, results in a set of design principles as a concrete fruition of design contractualism. In particular, this thesis describes: - a review of intelligent ubiquitous environments and relevant technologies, including a definition of user experience as a dynamic affective construct; - a specification of main components for personal data aggregation and service personalisation, without compromising privacy, security or usability; - the implementation of a software platform and a methodological procedure for its instantiation; - an evaluation of the developed platform and its benefits for urban mobility and public transport information systems; - a set of design principles for the design of ubiquitous systems, with an impact on individual experience and collective awareness. Cloud2Bubble contributes towards the development of affective intelligent ubiquitous systems with the potential to enhance user experience in pervasive environments. In addition, the platform aims at minimising the risk of user digital exposure while supporting collective action.Open Acces