The effects of encumbrance and mobility on interactions with touchscreen mobile devices

Mobile handheld devices such as smartphones are now convenient as they allow users to make calls, reply to emails, find nearby services and many more. The increase in functionality and availability of mobile applications also allow mobile devices to be used in many different everyday situations (for example, while on the move and carrying items). While previous work has investigated the interaction difficulties in walking situations, there is a lack of empirical work in the literature on mobile input when users are physically constrained by other activities. As a result, how users input on touchscreen handheld devices in encumbered and mobile contexts is less well known and deserves more attention to examine the usability issues that are often ignored. This thesis investigates targeting performance on touchscreen mobile phones in one common encumbered situation - when users are carrying everyday objects while on the move. To identify the typical objects held during mobile interactions and define a set of common encumbrance scenarios to evaluate in subsequent user studies, Chapter 3 describes an observational study that examined users in different public locations. The results showed that people carried different types of bags and boxes the most frequently. To measure how much tapping performance on touchscreen mobile phones is affected, Chapter 4 examines a range of encumbrance scenarios, which includes holding a bag in-hand or a box underarm, either on the dominant or non-dominant side, during target selections on a mobile phone. Users are likely to switch to a more effective input posture when encumbered and on the move, so Chapter 5 investigates one- and two- handed encumbered interactions and evaluates situations where both hands are occupied with multiple objects. Touchscreen devices afford various multi-touch input types, so Chapter 6 compares the performance of four main one- and two- finger gesture inputs: tapping, dragging, spreading & pinching and rotating, while walking and encumbered. Several main evaluation approaches have been used in previous walking studies, but more attention is required when the effects of encumbrance is also being examined. Chapter 7 examines the appropriateness of two methods (ground and treadmill walking) for encumbered and walking studies, justifies the need to control walking speed and examines the effects of varying walking speed (i.e. walking slower or faster than normal) on encumbered targeting performance. The studies all showed a reduction in targeting performance when users were walking and encumbered, so Chapter 8 explores two ways to improve target selections. The first approach defines a target size, based on the results collected from earlier studies, to increase tapping accuracy and subsequently, a novel interface arrangement was designed which optimises screen space more effectively. The second approach evaluates a benchmark pointing technique, which has shown to improve the selection of small targets, to see if it is useful in walking and encumbered contexts

Comparing Evaluation Methods for Encumbrance and Walking on Interaction with Touchscreen Mobile Devices

In this paper, two walking evaluation methods were compared to evaluate the effects of encumbrance while the preferred walking speed (PWS) is controlled. Users frequently carry cumbersome objects (e.g. shopping bags) and use mobile devices at the same time which can cause interaction difficulties and erroneous input. The two methods used to control the PWS were: walking on a treadmill and walking around a predefined route on the ground while following a pacesetter. The results from our target acquisition experiment showed that for ground walking at 100% of PWS, accuracy dropped to 36% when carrying a bag in the dominant hand while accuracy reduced to 34% for holding a box under the dominant arm. We also discuss the advantages and limitations of each evaluation method when examining encumbrance and suggest treadmill walking is not the most suitable approach to use if walking speed is an important factor in future mobile studies

EdgeGlass: Exploring Tapping Performance on Smart Glasses while Sitting and Walking

Department of Human Factors EngineeringCurrently, smart glasses allow only touch sensing area which supports front mounted touch pads. However, touches on top, front and bottom sides of glass mounted touchpad is not yet explored. We made a customized touch sensor (length: 5-6 cm, height: 1 cm, width: 0.5 cm) featuring the sensing on its top, front, and bottom surfaces. For doing that, we have used capacitive touch sensing technology (MPR121 chips) with an electrode size of ~4.5 mm square, which is typical in the modern touchscreens. We have created a hardware system which consists of a total of 48 separate touch sensors. We investigated the interaction technique by it for both the sitting and walking situation, using a single finger sequential tapping and a pair finger simultaneous tapping. We have divided each side into three equal target areas and this separation made a total of 36 combinations. Our quantitative result showed that pair finger simultaneous tapping touches were faster, less error-prone in walking condition, compared to single finger sequential tapping into walking condition. Whereas, single finger sequence tapping touches were slower, but less error-prone in sitting condition, compared to pair simultaneous tapping in sitting condition. However, single finger sequential tapping touches were slower, much less error-prone in sitting condition compared to walking. Interestingly, double finger tapping touches had similar performance result in terms of both, error rate and completion time, in both sitting and walking conditions. Mental, physical, performance, effort did not have any effect on any temporal tapping???s and body poses experience of workload. In case of the parameter of temporal demand, for single finger sequential tapping mean temporal (time pressure) workload demand was higher than pair finger simultaneous tapping but body poses did not affect temporal (time pressure) workload for both of the sequential and simultaneous tapping type. In case of the parameter of frustration, the result suggested that mean frustration workload was higher for single finger sequential tapping experienced by the participants compared to pair finger simultaneous tapping and among body poses, walking experienced higher frustration mean workload than sitting. The subjective measure of overall workload during the performance study showed no significant difference between both independent variable: body pose (sitting and walking) and temporal tapping (single finger sequential tapping and pair finger simultaneous tapping).ope

Searching on the go : the effects of fragmented attention on mobile web search tasks

Smart phones and tablets are rapidly becoming our main method of accessing information and are frequently used to perform on-the-go search tasks. Mobile devices are commonly used in situations where attention must be divided, such as when walking down a street. Research suggests that this increases cognitive load and, therefore, may have an impact on performance. In this work we conducted a laboratory experiment with both device types in which we simulated everyday, common mobile situations that may cause fragmented attention, impact search performance and affect user perception. Our results showed that the fragmented attention induced by the simulated conditions significantly affected both participants' objective and perceived search performance, as well as how hurried they felt and how engaged they were in the tasks. Furthermore, the type of device used also impacted how users felt about the search tasks, how well they performed and the amount of time they spent engaged in the tasks. These novel insights provide useful information to inform the design of future interfaces for mobile search and give us a greater understanding of how context and device size affect search behaviour and user experience

Searching on the Go: The Effects of Fragmented Attention on Mobile Web Search Tasks

Smart phones and tablets are rapidly becoming our main method of accessing information and are frequently used to perform on-the-go search tasks. Mobile devices are commonly used in situations where attention must be divided, such as when walking down a street. Research suggests that this increases cognitive load and, therefore, may have an impact on performance. In this work we conducted a laboratory experiment with both device types in which we simulated everyday, common mobile situations that may cause fragmented attention, impact search performance and affect user perception. Our results showed that the fragmented attention induced by the simulated conditions significantly affected both participants' objective and perceived search performance, as well as how hurried they felt and how engaged they were in the tasks. Furthermore, the type of device used also impacted how users felt about the search tasks, how well they performed and the amount of time they spent engaged in the tasks. These novel insights provide useful information to inform the design of future interfaces for mobile search and give us a greater understanding of how context and device size affect search behaviour and user experience

Détection de la manualité via les capteurs d'orientation du smartphone lors de la prise en main

National audiencePeople often switch hands while holding their phones, based on task and context. Ideally, we would be able to detect which hand they are using to hold the device, and use this information to optimize the interaction. We introduce a method to use built-in orientation sensors to detect which hand is holding a smartphone prior to first interaction. Based on logs of people picking up and unlocking a smartphone in a controlled study, we show that a dynamic-time warping approach trained with user-specific examples achieves 83.6% accuracy for determining which hand is holding the phone, prior to touching the screen.En fonction de la tâche et du contexte, les utilisateurs de smartphone ont pour habitude de changer de main pour tenir leur appareil. Idéalement, nous souhaiterions connaître la main utilisée afin d'optimiser l'interaction. A cet effet, nous introduisons une méthode utilisant les capteurs d'orientation intégrés afin de déterminer la main tenant le smartphone avant toute interaction. Nous montrons, par l'analyse des données de participants prenant et déverrouillant leurs smartphones durant une expérience contrôlée, qu'une approche utilisant l'algorithme Dynamic-Time Warping permet d'obtenir une précision de 83.6% afin de détecter la main utilisée

The impact of encumbrance on mobile interactions

Part 1: Long and Short Papers (Continued); International audience; This paper investigates the effects of encumbrance (holding different types of objects while using mobile devices) to understand the interaction difficulties that it causes. An experiment was conducted where participants performed a target acquisition task on a touchscreen mobile phone while carrying different types of bags and boxes. Mobility was also evaluated since people carry items from one place to another. Motion capture hardware was used to track hand and arm postures to examine how holding the different types of objects caused excessive movement and instability therefore resulting in performance to decline. The results showed encumbrance and mobility caused target accuracy to decrease although input while holding the box under the non-dominant arm was more accurate and exerted quicker targeting times than holding no objects. Encumbrance affected the dominant hand more than the non-dominant hand as targeting error significantly increased and caused greater hand instability. The issues caused by encumbrance suggest the topic requires more attention from researchers and users would benefit greatly if better interaction techniques and applications are developed to counteract the problems. Document type: Part of book or chapter of boo