Mobile text entry behaviour in lab and in-the-wild studies : is it different?

Text entry in smartphones remains a critical element of mobile HCI. It has been widely studied in lab settings, using primarily transcription tasks, and to a far lesser extent through in-the-wild (field) experiments. So far it remains unknown how well user behaviour during lab transcription tasks approximates real use. In this paper, we present a study that provides evidence that lab text entry behaviour is clearly distinguishable from real world use. Using machine learning techniques, we show that it is possible to accurately identify the type of study in which text entry sessions took place. The implications of our findings relate to the design of future studies in text entry, aiming to support input with virtual smartphone keyboards

WearPut : Designing Dexterous Wearable Input based on the Characteristics of Human Finger Motions

Department of Biomedical Engineering (Human Factors Engineering)Powerful microchips for computing and networking allow a wide range of wearable devices to be miniaturized with high fidelity and availability. In particular, the commercially successful smartwatches placed on the wrist drive market growth by sharing the role of smartphones and health management. The emerging Head Mounted Displays (HMDs) for Augmented Reality (AR) and Virtual Reality (VR) also impact various application areas in video games, education, simulation, and productivity tools. However, these powerful wearables have challenges in interaction with the inevitably limited space for input and output due to the specialized form factors for fitting the body parts. To complement the constrained interaction experience, many wearable devices still rely on other large form factor devices (e.g., smartphones or hand-held controllers). Despite their usefulness, the additional devices for interaction can constrain the viability of wearable devices in many usage scenarios by tethering users' hands to the physical devices. This thesis argues that developing novel Human-Computer interaction techniques for the specialized wearable form factors is vital for wearables to be reliable standalone products. This thesis seeks to address the issue of constrained interaction experience with novel interaction techniques by exploring finger motions during input for the specialized form factors of wearable devices. The several characteristics of the finger input motions are promising to enable increases in the expressiveness of input on the physically limited input space of wearable devices. First, the input techniques with fingers are prevalent on many large form factor devices (e.g., touchscreen or physical keyboard) due to fast and accurate performance and high familiarity. Second, many commercial wearable products provide built-in sensors (e.g., touchscreen or hand tracking system) to detect finger motions. This enables the implementation of novel interaction systems without any additional sensors or devices. Third, the specialized form factors of wearable devices can create unique input contexts while the fingers approach their locations, shapes, and components. Finally, the dexterity of fingers with a distinctive appearance, high degrees of freedom, and high sensitivity of joint angle perception have the potential to widen the range of input available with various movement features on the surface and in the air. Accordingly, the general claim of this thesis is that understanding how users move their fingers during input will enable increases in the expressiveness of the interaction techniques we can create for resource-limited wearable devices. This thesis demonstrates the general claim by providing evidence in various wearable scenarios with smartwatches and HMDs. First, this thesis explored the comfort range of static and dynamic touch input with angles on the touchscreen of smartwatches. The results showed the specific comfort ranges on variations in fingers, finger regions, and poses due to the unique input context that the touching hand approaches a small and fixed touchscreen with a limited range of angles. Then, finger region-aware systems that recognize the flat and side of the finger were constructed based on the contact areas on the touchscreen to enhance the expressiveness of angle-based touch input. In the second scenario, this thesis revealed distinctive touch profiles of different fingers caused by the unique input context for the touchscreen of smartwatches. The results led to the implementation of finger identification systems for distinguishing two or three fingers. Two virtual keyboards with 12 and 16 keys showed the feasibility of touch-based finger identification that enables increases in the expressiveness of touch input techniques. In addition, this thesis supports the general claim with a range of wearable scenarios by exploring the finger input motions in the air. In the third scenario, this thesis investigated the motions of in-air finger stroking during unconstrained in-air typing for HMDs. The results of the observation study revealed details of in-air finger motions during fast sequential input, such as strategies, kinematics, correlated movements, inter-fingerstroke relationship, and individual in-air keys. The in-depth analysis led to a practical guideline for developing robust in-air typing systems with finger stroking. Lastly, this thesis examined the viable locations of in-air thumb touch input to the virtual targets above the palm. It was confirmed that fast and accurate sequential thumb touch can be achieved at a total of 8 key locations with the built-in hand tracking system in a commercial HMD. Final typing studies with a novel in-air thumb typing system verified increases in the expressiveness of virtual target selection on HMDs. This thesis argues that the objective and subjective results and novel interaction techniques in various wearable scenarios support the general claim that understanding how users move their fingers during input will enable increases in the expressiveness of the interaction techniques we can create for resource-limited wearable devices. Finally, this thesis concludes with thesis contributions, design considerations, and the scope of future research works, for future researchers and developers to implement robust finger-based interaction systems on various types of wearable devices.ope

Behaviour-aware mobile touch interfaces

Mobile touch devices have become ubiquitous everyday tools for communication, information, as well as capturing, storing and accessing personal data. They are often seen as personal devices, linked to individual users, who access the digital part of their daily lives via hand-held touchscreens. This personal use and the importance of the touch interface motivate the main assertion of this thesis: Mobile touch interaction can be improved by enabling user interfaces to assess and take into account how the user performs these interactions. This thesis introduces the new term "behaviour-aware" to characterise such interfaces. These behaviour-aware interfaces aim to improve interaction by utilising behaviour data: Since users perform touch interactions for their main tasks anyway, inferring extra information from said touches may, for example, save users' time and reduce distraction, compared to explicitly asking them for this information (e.g. user identity, hand posture, further context). Behaviour-aware user interfaces may utilise this information in different ways, in particular to adapt to users and contexts. Important questions for this research thus concern understanding behaviour details and influences, modelling said behaviour, and inference and (re)action integrated into the user interface. In several studies covering both analyses of basic touch behaviour and a set of specific prototype applications, this thesis addresses these questions and explores three application areas and goals: 1) Enhancing input capabilities – by modelling users' individual touch targeting behaviour to correct future touches and increase touch accuracy. The research reveals challenges and opportunities of behaviour variability arising from factors including target location, size and shape, hand and finger, stylus use, mobility, and device size. The work further informs modelling and inference based on targeting data, and presents approaches for simulating touch targeting behaviour and detecting behaviour changes. 2) Facilitating privacy and security – by observing touch targeting and typing behaviour patterns to implicitly verify user identity or distinguish multiple users during use. The research shows and addresses mobile-specific challenges, in particular changing hand postures. It also reveals that touch targeting characteristics provide useful biometric value both in the lab as well as in everyday typing. Influences of common evaluation assumptions are assessed and discussed as well. 3) Increasing expressiveness – by enabling interfaces to pass on behaviour variability from input to output space, studied with a keyboard that dynamically alters the font based on current typing behaviour. Results show that with these fonts users can distinguish basic contexts as well as individuals. They also explicitly control font influences for personal communication with creative effects. This thesis further contributes concepts and implemented tools for collecting touch behaviour data, analysing and modelling touch behaviour, and creating behaviour-aware and adaptive mobile touch interfaces. Together, these contributions support researchers and developers in investigating and building such user interfaces. Overall, this research shows how variability in mobile touch behaviour can be addressed and exploited for the benefit of the users. The thesis further discusses opportunities for transfer and reuse of touch behaviour models and information across applications and devices, for example to address tradeoffs of privacy/security and usability. Finally, the work concludes by reflecting on the general role of behaviour-aware user interfaces, proposing to view them as a way of embedding expectations about user input into interactive artefacts

Software Usability

This volume delivers a collection of high-quality contributions to help broaden developers’ and non-developers’ minds alike when it comes to considering software usability. It presents novel research and experiences and disseminates new ideas accessible to people who might not be software makers but who are undoubtedly software users

Self-service kiosk-based anamnesis system for emergency departments

Dissertação de mestrado integrado em Engenharia InformáticaEmergency departments have a higher number of visits compared to other hospital de partments. Technology has played a crucial role in promoting improvements in hospital management and clinical performance. The number of visits to emergency departments has increased considerably, giving rise to crowding situations that cause several adverse effects. This situation negatively affects the provision of emergency services, impairs the quality of health care and increases the time patients wait for medical check-up. One of the leading causes contributing to the crowding is the high number of patients with low severity clinical condition. These are referred to as non-urgent or inappropriate patients, whose clinical situation should be taken care through self-care or primary health care. It is the responsibility of the institutions to analyse and quantify the possible causes of crowding to find the best solution to mitigate the adverse effects caused. It is believed that non-urgent patients can use the time spent in the waiting room more productively, namely by using a self-service kiosk to which they can provide valuable information to facilitate and accelerate the clinical processing. This work proposes a solution to be used in the waiting room of emergency departments, which aims to reduce the period of medical check-up. The solution uses a self-service kiosk for the patient to provide relevant clinical data that would otherwise have to be collected by the physician during the clinical observation process. In particular, the kiosk will collect vital signs, past medical history, main complaint and usual medication. This data will be processed and provided to the physician in a structured and uniform way before each medical check-up. The primary purpose of this solution is to reduce the period of patients’ medical check-up and thus improve the response capacity of the emergency departments with the same resources. During the Master’s work period, an Android application was implemented for patients to enter the clinical data mentioned above, and a Web application for physicians to access it. Additionally, a data warehouse was implemented to store the data in a consolidated way to discover hidden relationships and patterns in the data. The first moment of evaluation, undertaken in a non-hospital facility, shows positive acceptability by participants, with a large majority considering the system user-friendly. Due to the pandemic, it was impossible to perform the second planned evaluation moment in a real emergency environment.Os serviços de urgência apresentam um número de visitas superior em comparação com outros serviços presentes nas instituições hospitalares. A afluência aos serviços de urgências tem vindo a aumentar consideravelmente, dando origem a situações de lotação que provocam diversos efeitos negativos nas instituições hospitalares. No geral, este fenômeno afeta negativamente a prestação dos serviços de urgência, prejudica a qualidade dos cuidados de saúde e faz aumentar o tempo que os doentes aguardam pela observação clínica na sala de espera. Uma das principais causas apontadas para o surgimento da lotação é o elevado número de doentes com condição clínica de baixa gravidade. Estes são designados como doentes não-urgentes ou inapropriados, cuja condição clínica poderia ser resolvida, idealmente, com recurso ao auto-cuidado ou a cuidados de saúde primários. É da responsabilidade das instituições analisar e quantificar as possíveis causas de lotação, de forma a encontrar a melhor solução para atenuar os efeitos negativos provocados. Acredita-se que os doentes não-urgentes tenham a capacidade de utilizar o tempo na sala de espera de forma mais produtiva, através da utilização de um quiosque self-service. Neste sentido e aliada à tecnologia, esta dissertação contextualiza uma solução para ser utilizada na sala de espera dos serviços de urgência, visando reduzir o período de observação clínico. Esta solução vem complementar a realização do procedimento inicial efetuado pelo médico, no consultório, através do uso de um quiosque. Assim, a recolha dos sinais vitais, história médica prévia, queixa principal e medicação habitual será efetuada pelos doentes no quiosque. Estes dados vão ser fornecidos de forma estruturada e organizada ao médico antes da realização da consulta. O objetivo principal desta solução é reduzir o período de observação clínico e assim melhorar a capacidade de resposta dos serviços de urgência com os mesmos recursos hospitalares. Durante o período da dissertação, foi implementada uma aplicação Android para os pacientes registarem os dados clínicos acima mencionados, e uma aplicação Web para os médicos acederem aos mesmos. Foi implementado também um data warehouse para a descoberta de relações e padrões escondidos nos dados. O primeiro momento de avaliação, realizado num ambiente não hospitalar, mostrou uma aceitabilidade positiva pelos partici pantes, com grande maioria a considerar o sistema user-friendly. Devido à pandemia, não foi possível realizar o segundo momento de avaliação planeado num serviço de urgências

Touchscreen interventions for people with dementia

This project builds on the existing evidence that suggests that art-based interventions can be beneficial for the wellbeing of people with dementia and their carers, and explores whether such interventions can be delivered via a touchscreen tablet device using an application that allows them to view art images. Twelve pairs of volunteers with dementia and their informal carers were recruited through Alzheimer’s Society Dementia Cafés. A quasi-experimental mixed-methods within-subjects study evaluated the impact of art-viewing on wellbeing using visual analogue scales as well as exploring the experiences of participants qualitatively with thematic analysis. Quantitative results showed a significant effect for change in composite wellbeing from viewing session one to session five. Wellbeing subdomain scores showed an impact on wellbeing which tended to increase with the number of sessions. Qualitative findings were reported in relation to shifts in cognition, changes in behaviour, mood and relationships between people with dementia and their carers. These changes tended to be viewed as positive by interviewees. The results suggest that touchscreen based art interventions have the potential to provide an activity people with dementia can engage in with their carers which can benefit their wellbeing. A larger-scale controlled study would help to further determine whether wider dementia care practice implications can be drawn for clinical psychologists and other healthcare providers

Mechanical device or touchscreen widget: the effects of input device and task size on data entry on the primary flight display

ABSTRACT: Due to their customizability, touchscreens continue to advance as a device of choice when designing aircraft cockpits. Previous studies investigated the effect of turbulence on human performance when using touchscreens, but few have evaluated its performance for realistic aviation-specific tasks. In this study, we compared four touchscreen widgets and three mechanical devices during realistic data input on a primary flight display (PFD). Twenty participants took part in the experiment at a constant level of vibration, while simultaneously completing a secondary tracking task. Results indicated that virtual keypads lead to faster completion time for medium to large changes while keeping error rates low. Rotary knobs were fastest for small changes. Virtual keypads also had lower workload and discomfort compared to rotary knobs and drag-based widgets. We found the completion time to be the most important factor in tracking task performance, which translated in higher precision for keypads. These findings suggest that virtual keypads represent an efficient and secure option for numerical data input at low-to-medium vibration

Seamless Authentication for Ubiquitous Devices

User authentication is an integral part of our lives; we authenticate ourselves to personal computers and a variety of other things several times a day. Authentication is burdensome. When we wish to access to a computer or a resource, it is an additional task that we need to perform~-- an interruption in our workflow. In this dissertation, we study people\u27s authentication behavior and attempt to make authentication to desktops and smartphones less burdensome for users. First, we present the findings of a user study we conducted to understand people\u27s authentication behavior: things they authenticate to, how and when they authenticate, authentication errors they encounter and why, and their opinions about authentication. In our study, participants performed about 39 authentications per day on average; the majority of these authentications were to personal computers (desktop, laptop, smartphone, tablet) and with passwords, but the number of authentications to other things (e.g., car, door) was not insignificant. We saw a high failure rate for desktop and laptop authentication among our participants, affirming the need for a more usable authentication method. Overall, we found that authentication was a noticeable part of all our participants\u27 lives and burdensome for many participants, but they accepted it as cost of security, devising their own ways to cope with it. Second, we propose a new approach to authentication, called bilateral authentication, that leverages wrist-wearable technology to enable seamless authentication for things that people use with their hands, while wearing a smart wristband. In bilateral authentication two entities (e.g., user\u27s wristband and the user\u27s phone) share their knowledge (e.g., about user\u27s interaction with the phone) to verify the user\u27s identity. Using this approach, we developed a seamless authentication method for desktops and smartphones. Our authentication method offers quick and effortless authentication, continuous user verification while the desktop (or smartphone) is in use, and automatic deauthentication after use. We evaluated our authentication method through four in-lab user studies, evaluating the method\u27s usability and security from the system and the user\u27s perspective. Based on the evaluation, our authentication method shows promise for reducing users\u27 authentication burden for desktops and smartphones

Haptic feedback in freehand gesture interaction

In this thesis work, haptic feedback in gesture interaction was studied. More precisely, focus was on vibrotactile feedback and freehand gestural input methods. Vibrotactile feedback methods have been studied extensively in the fields of touch-based interaction, remote control and mid-air gestural input, and mostly positive effects on user performance have been found. An experiment was conducted in order to investigate if vibrotactile feedback has an impact on user performance in a simple data entry task. In the study, two gestural input methods were compared and the effects of visual and vibrotactile feedback added to each method were examined. Statistically significant differences in task performance between input methods were found. Results also showed that less keystrokes per character were required with visual feedback. No other significant differences were found between the types of feedback. However, preference for vibrotactile feedback was observed. The findings indicate that the careful design of an input method primarily has an impact on user performance and the feedback method can enhance this performance in diverse ways