Making Graphical Scores More Accessible: A Case Study

This paper explores new ways of making graphical scores more accessible for visually impaired users. Existing assistive technologies demonstrate a gap in providing accessible tools for composing and performing contemporary music with non-traditional western notation. The two case studies presented, Blocks Sound and Logothetis Sound examine the interactive relationships and affordances through tac- tile interaction and how this interaction can influence their experience and understanding of both graphic scores and interactive composition of users. We present the process and limitations and propose the use of haptic technology and tangible experience for making contemporary graphic scores more accessible and inclusive

A Haptic Surface Robot Interface for Large-Format Touchscreen Displays

This thesis presents the design for a novel haptic interface for large-format touchscreens. Techniques such as electrovibration, ultrasonic vibration, and external braked devices have been developed by other researchers to deliver haptic feedback to touchscreen users. However, these methods do not address the need for spatial constraints that only restrict user motion in the direction of the constraint. This technology gap contributes to the lack of haptic technology available for touchscreen-based upper-limb rehabilitation, despite the prevalent use of haptics in other forms of robotic rehabilitation. The goal of this thesis is to display kinesthetic haptic constraints to the touchscreen user in the form of boundaries and paths, which assist or challenge the user in interacting with the touchscreen. The presented prototype accomplishes this by steering a single wheel in contact with the display while remaining driven by the user. It employs a novel embedded force sensor, which it uses to measure the interaction force between the user and the touchscreen. The haptic response of the device is controlled using this force data to characterize user intent. The prototype can operate in a simulated free mode as well as simulate rigid and compliant obstacles and path constraints. A data architecture has been created to allow the prototype to be used as a peripheral add-on device which reacts to haptic environments created and modified on the touchscreen. The long-term goal of this work is to create a haptic system that enables a touchscreen-based rehabilitation platform for people with upper limb impairments

空中ジェスチャ・インタラクションにおける触覚提示の高臨場感化および安定化に関する研究

Tohoku University昆陽雅司課

Human factors considerations for ultrasound induced mid-air haptic feedback

The engineering design process can be complex and often involves reiteration of design activities in order to improve outcomes. Traditionally, the design process consists of many physical elements, for example, clay/foam modelling and more recently Additive Manufacturing (AM), with an iterative cycle of user testing of these physical prototypes. The time associated with creating physical prototypes can lengthen the time it takes to develop one product, and thus, comes at a burdensome financial and labour cost. Due to the aforementioned constraints of the conventional design process, more research is being conducted into applications of Virtual Reality (VR) to complement stages of the design process that would otherwise take and cost a significant amount of time and money. VR enables users to create 3D virtual designs and prototypes for evaluation, thus facilitating the rapid correction of design and usability issues. However, VR is not without its pitfalls, for example, it often only facilitates an audio-visual simulation, thus hindering evaluation of the tactile element of design, which is critical to the success of many products. This issue already has a wide body of research associated with it, which explores applications of haptic (tactile) feedback to VR to create a more realistic and accurate virtual experience. However, current haptic technologies can be expensive, cumbersome, hard to integrate with existing design tools, and have limited sensorial output (for example, vibrotactile feedback). Ultrasound Haptic Feedback (UsHF) appears to be a promising technology that offers affordable, unencumbered, integrable and versatile use. The technology achieves this by using ultrasound to create mid-air haptic feedback which users can feel without being attached to a device. However, due to the novel nature of the technology, there is little to no literature dedicated to investigating how users perceive and interpret UsHF stimuli, and how their perception affects the user experience. The research presented in this thesis concerns the human factors of UsHF for engineering design applications. The PhD was borne out of interest from Ultraleap (previously Ultrahaptics), an SME technology developer, on how their mid-air haptic feedback device could be used within the field of engineering. Six studies (five experimental and one qualitative) were conducted in order to explore the human factors of UsHF, with a view of understanding its viability for use in engineering design. This was achieved by exploring the tactile ability of users in mid-air object size discrimination, absolute tactile thresholds, perception of intensity differences, and normalisation of UsHF intensity. These measures were also tested against individual differences in age, gender and fingertip/hand size during the early stages, with latter stages focussing on the same measures when UsHF was compared to 2D multimodal and physical environments. The findings demonstrated no evidence of individual differences in UsHF tactile acuity and perception of UsHF stimuli. However, the results did highlight clear limitations in object size discrimination and absolute tactile thresholds. Interestingly, the results also demonstrated psychophysical variation in the perception of UsHF intensity differences, with intensity differences having a significant effect on how object size is perceived. Comparisons between multimodal UsHF and physical size discrimination were also conducted and found size discrimination accuracy of physical objects to be better than visuo-haptic (UsHF) size discrimination. Qualitative studies revealed an optimistic attitude towards VR for engineering design applications, particularly within the design, review, and prototyping stages, with many suggesting the addition of haptic feedback could be beneficial to the process. This thesis offers a novel contribution to the field of human factors for mid-air haptics, and in particular for the use of this technology as part of the engineering design process. The results indicate that UsHF in its current state could not offer a replacement for all physical prototypes within the design process; however, UsHF may still have a place in the virtual design process where haptic feedback is required but is less reliant on the accurate portrayal of virtual objects, for example, during early stage evaluations supplemented by later physical prototypes, simply to indicate contact with virtual objects, or when sharing designs with stakeholders and multidisciplinary teams

Human factors considerations for ultrasound induced mid-air haptic feedback

The engineering design process can be complex and often involves reiteration of design activities in order to improve outcomes. Traditionally, the design process consists of many physical elements, for example, clay/foam modelling and more recently Additive Manufacturing (AM), with an iterative cycle of user testing of these physical prototypes. The time associated with creating physical prototypes can lengthen the time it takes to develop one product, and thus, comes at a burdensome financial and labour cost. Due to the aforementioned constraints of the conventional design process, more research is being conducted into applications of Virtual Reality (VR) to complement stages of the design process that would otherwise take and cost a significant amount of time and money. VR enables users to create 3D virtual designs and prototypes for evaluation, thus facilitating the rapid correction of design and usability issues. However, VR is not without its pitfalls, for example, it often only facilitates an audio-visual simulation, thus hindering evaluation of the tactile element of design, which is critical to the success of many products. This issue already has a wide body of research associated with it, which explores applications of haptic (tactile) feedback to VR to create a more realistic and accurate virtual experience. However, current haptic technologies can be expensive, cumbersome, hard to integrate with existing design tools, and have limited sensorial output (for example, vibrotactile feedback). Ultrasound Haptic Feedback (UsHF) appears to be a promising technology that offers affordable, unencumbered, integrable and versatile use. The technology achieves this by using ultrasound to create mid-air haptic feedback which users can feel without being attached to a device. However, due to the novel nature of the technology, there is little to no literature dedicated to investigating how users perceive and interpret UsHF stimuli, and how their perception affects the user experience. The research presented in this thesis concerns the human factors of UsHF for engineering design applications. The PhD was borne out of interest from Ultraleap (previously Ultrahaptics), an SME technology developer, on how their mid-air haptic feedback device could be used within the field of engineering. Six studies (five experimental and one qualitative) were conducted in order to explore the human factors of UsHF, with a view of understanding its viability for use in engineering design. This was achieved by exploring the tactile ability of users in mid-air object size discrimination, absolute tactile thresholds, perception of intensity differences, and normalisation of UsHF intensity. These measures were also tested against individual differences in age, gender and fingertip/hand size during the early stages, with latter stages focussing on the same measures when UsHF was compared to 2D multimodal and physical environments. The findings demonstrated no evidence of individual differences in UsHF tactile acuity and perception of UsHF stimuli. However, the results did highlight clear limitations in object size discrimination and absolute tactile thresholds. Interestingly, the results also demonstrated psychophysical variation in the perception of UsHF intensity differences, with intensity differences having a significant effect on how object size is perceived. Comparisons between multimodal UsHF and physical size discrimination were also conducted and found size discrimination accuracy of physical objects to be better than visuo-haptic (UsHF) size discrimination. Qualitative studies revealed an optimistic attitude towards VR for engineering design applications, particularly within the design, review, and prototyping stages, with many suggesting the addition of haptic feedback could be beneficial to the process. This thesis offers a novel contribution to the field of human factors for mid-air haptics, and in particular for the use of this technology as part of the engineering design process. The results indicate that UsHF in its current state could not offer a replacement for all physical prototypes within the design process; however, UsHF may still have a place in the virtual design process where haptic feedback is required but is less reliant on the accurate portrayal of virtual objects, for example, during early stage evaluations supplemented by later physical prototypes, simply to indicate contact with virtual objects, or when sharing designs with stakeholders and multidisciplinary teams