Exploring the Use of Wearables to develop Assistive Technology for Visually Impaired People

This thesis explores the usage of two prominent wearable devices to develop assistive technology for users who are visually impaired. Specifically, the work in this thesis aims at improving the quality of life of users who are visually impaired by improving their mobility and ability to socially interact with others. We explore the use of a smart watch for creating low-cost spatial haptic applications. This app explores the use of haptic feedback provided using a smartwatch and smartphone to provide navigation instructions that let visually impaired people safely traverse a large open space. This spatial feedback guides them to walk on a straight path from source to destination by avoiding veering. Exploring the paired interaction between a Smartphone and a Smartwatch, helped to overcome the limitation that smart devices have only single haptic actuator.We explore the use of a head-mounted display to enhance social interaction by helping people with visual impairments align their head towards a conversation partner as well as maintain personal space during a conversation. Audio feedback is provided to the users guiding them to achieve effective face-to-face communication. A qualitative study of this method shows the effectiveness of the application and explains how it helps visually impaired people to perceive non-verbal cues and feel more engaged and assertive in social interactions

Augmenting the Spatial Perception Capabilities of Users Who Are Blind

People who are blind face a series of challenges and limitations resulting from their lack of being able to see, forcing them to either seek the assistance of a sighted individual or work around the challenge by way of a inefficient adaptation (e.g. following the walls in a room in order to reach a door rather than walking in a straight line to the door). These challenges are directly related to blind users' lack of the spatial perception capabilities normally provided by the human vision system. In order to overcome these spatial perception related challenges, modern technologies can be used to convey spatial perception data through sensory substitution interfaces. This work is the culmination of several projects which address varying spatial perception problems for blind users. First we consider the development of non-visual natural user interfaces for interacting with large displays. This work explores the haptic interaction space in order to find useful and efficient haptic encodings for the spatial layout of items on large displays. Multiple interaction techniques are presented which build on prior research (Folmer et al. 2012), and the efficiency and usability of the most efficient of these encodings is evaluated with blind children. Next we evaluate the use of wearable technology in aiding navigation of blind individuals through large open spaces lacking tactile landmarks used during traditional white cane navigation. We explore the design of a computer vision application with an unobtrusive aural interface to minimize veering of the user while crossing a large open space. Together, these projects represent an exploration into the use of modern technology in augmenting the spatial perception capabilities of blind users

First validation of the Haptic Sandwich: a shape changing handheld haptic navigation aid

This paper presents the Haptic Sandwich, a handheld robotic device that designed to provide pedestrian navigation instructions through a novel shape changing modality. The device resembles a cube with an articulated upper half that is able to rotate and translate (extend) relative to the bottom half, which is grounded in the user’s hand when the device is held. The poses assumed by the device simultaneously correspond to heading and proximity to a navigational target. The Haptic Sandwich provides an alternative to screen and/or audio based pedestrian navigation technologies for both visually impaired and sighted users. Unlike other robotic or haptic navigational solutions, the haptic sandwich is discrete in terms of form and sensory stimulus. Due to the novel and unexplored nature of shape changing interfaces, two user studies were undertaken to validate the concept and device. In the first experiment, stationary participants attempted to identify poses assumed by the device, which was hidden from view. In the second experiment, participants attempted to locate a sequence of invisible navigational targets while walking with the device. Of 1080 pose presentations to 10 individuals in experiment one, 80% were correctly identified and 17.5% had the minimal possible error. Multi-DOF errors accounted for only 1.1% of all answers. The role of simultaneous or independent actuator motion on final shape perception was tested with no significant performance difference. The rotation and extension DOF had significantly different perception accuracy. In the second experiment, participants demonstrated good navigational ability with the device after minimal training and were able to locate all presented targets. Mean motion efficiency of the participants was between 32%-56%. Participants made use of both DOF

USER INTERFACES FOR MOBILE DEVICES: TECHNIQUES AND CASE STUDIES

The interactive capabilities of portable devices that are nowadays increasingly available, enable mobile computing in diverse contexts. However, in order to fully exploit the potentialities of such technologies and to let end users benefit from them, effective and usable techniques are still needed. In general, differences in capabilities, such as computational power and interaction resources, lead to an heterogeneity that is sometimes positively referred to as device diversity but also, negatively, as device fragmentation. When designing applications for mobile devices, besides general rules and principles of usability, developers cope with further constraints. Restricted capabilities, due to display size, input modality and computational power, imply important design and implementation choices in order to guarantee usability. In addition, when the application is likely to be used by subjects affected by some impairment, the system has also to comply with accessibility requirements. The aim of this dissertation is to propose and discuss examples of such techniques, aimed to support user interfaces on mobile devices, by tackling design, development and evaluation of specific solutions for portable terminals as well as for enabling interoperability across diverse devices (including desktops, handhelds, smartphones). Usefulness and usability aspects are taken into great consideration by the main research questions that drove the activities of the study. With respect the such questions, the three central chapters of the dissertation are respectively aimed at evaluating: hardware/software solutions for edutainment and accessibility in mobile museum guides, visualization strategies for mobile users visiting smart environments, and techniques for user interface migration across diverse devices in multi-user contexts. Motivations, design, implementation and evaluation about a number of solutions aimed to support several dimensions of user interfaces for mobile devices are widely discussed throughout the dissertation, and some findings are drawn. Each one of the prototypes described in the following chapters has been entirely developed within the research activities of the laboratory where the author performed his PhD. Most activities were related to tasks of international research projects and the organization of this dissertation reflects their evolution chronology

Assisting Human Motion-Tasks with Minimal, Real-time Feedback

Teaching physical motions such as riding, exercising, swimming, etc. to human beings is hard. Coaches face difficulties in communicating their feedback verbally and cannot correct the student mid-action; teaching videos are two dimensional and suffer from perspective distortion. Systems that track a user and provide him real-time feedback have many potential applications: as an aid to the visually challenged, improving rehabilitation, improving exercise routines such as weight training or yoga, teaching new motion tasks, synchronizing motions of multiple actors, etc. It is not easy to deliver real-time feedback in a way that is easy to interpret, yet unobtrusive enough to not distract the user from the motion task. I have developed motion feedback systems that provide real-time feedback to achieve or improve human motion tasks. These systems track the user\u27s actions with simple sensors, and use tiny vibration motors as feedback devices. Vibration motors provide feedback that is both intuitive and minimally intrusive. My systems\u27 designs are simple, flexible, and extensible to large-scale, full-body motion tasks. The systems that I developed as part of this thesis address two classes of motion tasks: configuration tasks and trajectory tasks. Configuration tasks guide the user to a target configuration. My systems for configuration tasks use a motion-capture system to track the user. Configuration-task systems restrict the user\u27s motions to a set of motion primitives, and guide the user to the target configuration by executing a sequence of motion-primitives. Trajectory tasks assume that the user understands the motion task. The systems for trajectory tasks provide corrective feedback that assists the user in improving their performance. This thesis presents the design, implementation, and results of user experiments with the prototype systems I have developed

Somatic ABC's: A Theoretical Framework for Designing, Developing and Evaluating the Building Blocks of Touch-Based Information Delivery

abstract: Situations of sensory overload are steadily becoming more frequent as the ubiquity of technology approaches reality--particularly with the advent of socio-communicative smartphone applications, and pervasive, high speed wireless networks. Although the ease of accessing information has improved our communication effectiveness and efficiency, our visual and auditory modalities--those modalities that today's computerized devices and displays largely engage--have become overloaded, creating possibilities for distractions, delays and high cognitive load; which in turn can lead to a loss of situational awareness, increasing chances for life threatening situations such as texting while driving. Surprisingly, alternative modalities for information delivery have seen little exploration. Touch, in particular, is a promising candidate given that it is our largest sensory organ with impressive spatial and temporal acuity. Although some approaches have been proposed for touch-based information delivery, they are not without limitations including high learning curves, limited applicability and/or limited expression. This is largely due to the lack of a versatile, comprehensive design theory--specifically, a theory that addresses the design of touch-based building blocks for expandable, efficient, rich and robust touch languages that are easy to learn and use. Moreover, beyond design, there is a lack of implementation and evaluation theories for such languages. To overcome these limitations, a unified, theoretical framework, inspired by natural, spoken language, is proposed called Somatic ABC's for Articulating (designing), Building (developing) and Confirming (evaluating) touch-based languages. To evaluate the usefulness of Somatic ABC's, its design, implementation and evaluation theories were applied to create communication languages for two very unique application areas: audio described movies and motor learning. These applications were chosen as they presented opportunities for complementing communication by offloading information, typically conveyed visually and/or aurally, to the skin. For both studies, it was found that Somatic ABC's aided the design, development and evaluation of rich somatic languages with distinct and natural communication units.Dissertation/ThesisPh.D. Computer Science 201

Eyes-Off Physically Grounded Mobile Interaction

This thesis explores the possibilities, challenges and future scope for eyes-off, physically grounded mobile interaction. We argue that for interactions with digital content in physical spaces, our focus should not be constantly and solely on the device we are using, but fused with an experience of the places themselves, and the people who inhabit them. Through the design, development and evaluation of a series ofnovel prototypes we show the benefits of a more eyes-off mobile interaction style.Consequently, we are able to outline several important design recommendations for future devices in this area.The four key contributing chapters of this thesis each investigate separate elements within this design space. We begin by evaluating the need for screen-primary feedback during content discovery, showing how a more exploratory experience can be supported via a less-visual interaction style. We then demonstrate how tactilefeedback can improve the experience and the accuracy of the approach. In our novel tactile hierarchy design we add a further layer of haptic interaction, and show how people can be supported in finding and filtering content types, eyes-off. We then turn to explore interactions that shape the ways people interact with aphysical space. Our novel group and solo navigation prototypes use haptic feedbackfor a new approach to pedestrian navigation. We demonstrate how variations inthis feedback can support exploration, giving users autonomy in their navigationbehaviour, but with an underlying reassurance that they will reach the goal.Our final contributing chapter turns to consider how these advanced interactionsmight be provided for people who do not have the expensive mobile devices that areusually required. We extend an existing telephone-based information service to support remote back-of-device inputs on low-end mobiles. We conclude by establishingthe current boundaries of these techniques, and suggesting where their usage couldlead in the future

A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research

This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigational assistance modules, text-to-speech applications, as well as virtual audio displays which combine audio with haptic channels are becoming integrated into standard mobile devices. This trend, combined with the appearance of increasingly user- friendly interfaces and modes of interaction has opened a variety of new perspectives for the rehabilitation and training of users with visual impairments. The goal of this paper is to provide an overview of these developments based on recent advances in basic research and application development. Using this overview as a foundation, an agenda is outlined for future research in mobile interaction design with respect to users with special needs, as well as ultimately in relation to sensor-bridging applications in genera

A Tracking Topology to Support a Trajectory Monitoring System Directed at Helping Visually Impaired on Paralympic Athletics

peer reviewedThe study reported by this article was based on previous studies about the development of assistive technology for visually impaired athletes and aimed to evaluate a tracking topology that can be applied for the location and orientation of visually impaired on Paralympic Athletics. The paper presents the tracking topology and its integration in a trajectory monitoring system able to provide vibratory stimulus to visually impaired athletes. Moreover, the paper also presents the results of an experiment that was set up to evaluate a trajectory monitoring system prototype by eight visually impaired athletes when performing their training activities