RealPen: Providing Realism in Handwriting Tasks on Touch Surfaces using Auditory-Tactile Feedback

We present RealPen, an augmented stylus for capacitive tablet screens that recreates the physical sensation of writing on paper with a pencil, ball-point pen or marker pen. The aim is to create a more engaging experience when writing on touch surfaces, such as screens of tablet computers. This is achieved by regenerating the friction-induced oscillation and sound of a real writing tool in contact with paper. To generate realistic tactile feedback, our algorithm analyzes the frequency spectrum of the friction oscillation generated when writing with traditional tools, extracts principal frequencies, and uses the actuator's frequency response profile for an adjustment weighting function. We enhance the realism by providing the sound feedback aligned with the writing pressure and speed. Furthermore, we investigated the effects of superposition and fluctuation of several frequencies on human tactile perception, evaluated the performance of RealPen, and characterized users' perception and preference of each feedback type

Haptic Experience and the Design of Drawing Interfaces

Haptic feedback has the potential to enhance users’ sense of being engaged and creative in their artwork. Current work on providing haptic feedback in computer-based drawing applications has focused mainly on the realism of the haptic sensation rather than the users’ experience of that sensation in the context of their creative work. We present a study that focuses on user experience of three haptic drawing interfaces. These interfaces were based on two different haptic metaphors, one of which mimicked familiar drawing tools (such as pen, pencil or crayon on smooth or rough paper) and the other of which drew on abstract descriptors of haptic experience (roughness, stickiness, scratchiness and smoothness). It was found that users valued having control over the haptic sensation; that each metaphor was preferred by approximately half of the participants; and that the real world metaphor interface was considered more helpful than the abstract one, whereas the abstract interface was considered to better support creativity. This suggests that future interfaces for artistic work should have user-modifiable interaction styles for controlling the haptic sensation

Tools in and out of sight : an analysis informed by Cultural-Historical Activity Theory of audio-haptic activities involving people with visual impairments supported by technology

The main purpose of this thesis is to present a Cultural-Historical Activity Theory (CHAT) based analysis of the activities conducted by and with visually impaired users supported by audio-haptic technology.This thesis covers several studies conducted in two projects. The studies evaluate the use of audio-haptic technologies to support and/or mediate the activities of people with visual impairment. The focus is on the activities involving access to two-dimensional information, such as pictures or maps. People with visual impairments can use commercially available solutions to explore static information (raised lined maps and pictures, for example). Solu-tions for dynamic access, such as drawing a picture or using a map while moving around, are more scarce. Two distinct projects were initiated to remedy the scarcity of dynamic access solutions, specifically focusing on two separate activities.The first project, HaptiMap, focused on pedestrian outdoors navigation through audio feedback and gestures mediated by a GPS equipped mobile phone. The second project, HIPP, focused on drawing and learning about 2D representations in a school setting with the help of haptic and audio feedback. In both cases, visual feedback was also present in the technology, enabling people with vision to take advantage of that modality too.The research questions addressed are: How can audio and haptic interaction mediate activ-ities for people with visual impairment? Are there features of the programming that help or hinder this mediation? How can CHAT, and specifically the Activity Checklist, be used to shape the design process, when designing audio haptic technology together with persons with visual impairments?Results show the usefulness of the Activity Checklist as a tool in the design process, and provide practical application examples. A general conclusion emphasises the importance of modularity, standards, and libre software in rehabilitation technology to support the development of the activities over time and to let the code evolve with them, as a lifelong iterative development process. The research also provides specific design recommendations for the design of the type of audio haptic systems involved

Portfolio of original compositions

In the last few decades the area of touch research has become vibrant, exciting and urgent, yet research into touch in music – in the way outlined here as cutaneous contact – is close to non-existent. Sounds emanate from touch but still, touch has not been included in the aesthetics of music. Examining touch in musical literature I argue that touch is fundamental to music while music’s philosophical and aesthetic definition has excluded it. My research reveals that what were previously argued to be the main reasons to marginalize touch in music, are in fact facets of a style of music which has, so far, been marginalized because touch’s inclusion problematizes the tangibility of the musical experience. By contrasting my portfolio of original works with my literary review I demonstrate my interrogation of these marginalized facets and show how music changes by moving away from sounding description to tactile demonstration and exploration of sounding shapes, textures and resonating volumes. By including touch in music instruments become tailor-made for individual bodies engaging with and disrupting the technical skill of the musician, the score stops being a singularly imaginable picture and becomes a fragmented assemblage of tangible forms as a musical instance, which I have called the composed musical instrument. Thus, changing musical organization from the written to the assembled composition changes from writing in time to feeling the mapping of topologies and the manipulation of materials through cutaneous contact. Performance of a composition stops being stage oriented and begins at the first sound of making the composed musical instrument. Most strangely, silence, a key component of music so far, stops being necessary and listening can be done with the skin. Retaining its communicative value as a sound the inclusion of touch creates a semantic sign system that can only be experienced through touch

Making Spatial Information Accessible on Touchscreens for Users who are Blind and Visually Impaired

Touchscreens have become a de facto standard of input for mobile devices as they most optimally use the limited input and output space that is imposed by their form factor. In recent years, people who are blind and visually impaired have been increasing their usage of smartphones and touchscreens. Although basic access is available, there are still many accessibility issues left to deal with in order to bring full inclusion to this population. One of the important challenges lies in accessing and creating of spatial information on touchscreens. The work presented here provides three new techniques, using three different modalities, for accessing spatial information on touchscreens. The first system makes geometry and diagram creation accessible on a touchscreen through the use of text-to-speech and gestural input. This first study is informed by a qualitative study of how people who are blind and visually impaired currently access and create graphs and diagrams. The second system makes directions through maps accessible using multiple vibration sensors without any sound or visual output. The third system investigates the use of binaural sound on a touchscreen to make various types of applications accessible such as physics simulations, astronomy, and video games

Investigation of possible causes for human-performance degradation during microgravity flight

The results of the first year of a three year study of the effects of microgravity on human performance are given. Test results show support for the hypothesis that the effects of microgravity can be studied indirectly on Earth by measuring performance in an altered gravitational field. The hypothesis was that an altered gravitational field could disrupt performance on previously automated behaviors if gravity was a critical part of the stimulus complex controlling those behaviors. In addition, it was proposed that performance on secondary cognitive tasks would also degrade, especially if the subject was provided feedback about degradation on the previously automated task. In the initial experimental test of these hypotheses, there was little statistical support. However, when subjects were categorized as high or low in automated behavior, results for the former group supported the hypotheses. The predicted interaction between body orientation and level of workload in their joint effect on performance in the secondary cognitive task was significant for the group high in automatized behavior and receiving feedback, but no such interventions were found for the group high in automatized behavior but not receiving feedback, or the group low in automatized behavior

Stylus For Fine-Motor Development

The objective of this project was to improve the writing ability of children, between the ages of 4-6 by encouraging proper grip techniques and practice through the medium of touch screen technology. The methods we used were, field testing children, interviewing and academic research of the professional literature on the occupational development of grips, emerging usages of touch screen devices in homes and school, and declining writing ability for children. The conclusions reached by the team are: fine motor development in children has been decreasing, children are using touch screen technologies at an increasing rate, and the styluses developed by the team will improve children handwriting writing development