TESSA - Toolkit for experimentation with multimodal sensory substitution and augmentation

TESSA is a toolkit for experimenting with sensory augmentation. It includes hardware and software to facilitate rapid prototyping of interfaces that can enhance one sense using information gathered from another sense. The toolkit contains a range of sensors (e.g. ultrasonics, temperature sensors) and actuators (e.g. tactors or stereo sound), designed modularly so that inputs and outputs can be easily swapped in and out and customized using TESSA’s graphical user interface (GUI), with “real time” feedback. The system runs on a Raspberry Pi with a built-in touchscreen, providing a compact and portable form that is amenable for field trials. At CHI Interactivity, the audience will have the opportunity to experience sensory augmentation effects using this system, and design their own sensory augmentation interfaces

Cross-modal Effects In Tactile And Visual Signaling

Using a wearable tactile display three experiments were conducted in which tactile messages were created emulating five standard US Army and Marine arm and hand signals for the military commands, namely: Attention , Halt , Rally , Move Out , and Nuclear Biological or Chemical event (NBC) . Response times and accuracy rates were collected for novices responding to visual and tactile representations of these messages, which were displayed either alone or together in congruent or incongruent combinations. Results indicated synergistic effects for concurrent, congruent message presentations showing superior response times when compared to individual presentations in either modality alone. This effect was mediated by participant strategy. Accuracy similarly improved when both the tactile and visual presentation were concurrently displayed as opposed to separately. In a low workload condition, participants could largely attend to a particular modality, with little interference from competing signals. If participants were not given instructions as to which modality to attend to, participants chose that modality which was received first. Lastly, initial learning and subsequent training of intuitive tactile signals occurred rapidly with large gains in performance in short training periods. These results confirm the promise for tactile messages to augment visual messaging in challenging and stressful environments particularly when visual messaging is maybe preferred but is not always feasible or possible

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Musical Haptics

Haptic Musical Instruments; Haptic Psychophysics; Interface Design and Evaluation; User Experience; Musical Performanc

Musical Haptics

Haptic Musical Instruments; Haptic Psychophysics; Interface Design and Evaluation; User Experience; Musical Performanc

Image-Based Force Estimation and Haptic Rendering For Robot-Assisted Cardiovascular Intervention

Clinical studies have indicated that the loss of haptic perception is the prime limitation of robot-assisted cardiovascular intervention technology, hindering its global adoption. It causes compromised situational awareness for the surgeon during the intervention and may lead to health risks for the patients. This doctoral research was aimed at developing technology for addressing the limitation of the robot-assisted intervention technology in the provision of haptic feedback. The literature review showed that sensor-free force estimation (haptic cue) on endovascular devices, intuitive surgeon interface design, and haptic rendering within the surgeon interface were the major knowledge gaps. For sensor-free force estimation, first, an image-based force estimation methods based on inverse finite-element methods (iFEM) was developed and validated. Next, to address the limitation of the iFEM method in real-time performance, an inverse Cosserat rod model (iCORD) with a computationally efficient solution for endovascular devices was developed and validated. Afterward, the iCORD was adopted for analytical tip force estimation on steerable catheters. The experimental studies confirmed the accuracy and real-time performance of the iCORD for sensor-free force estimation. Afterward, a wearable drift-free rotation measurement device (MiCarp) was developed to facilitate the design of an intuitive surgeon interface by decoupling the rotation measurement from the insertion measurement. The validation studies showed that MiCarp had a superior performance for spatial rotation measurement compared to other modalities. In the end, a novel haptic feedback system based on smart magnetoelastic elastomers was developed, analytically modeled, and experimentally validated. The proposed haptics-enabled surgeon module had an unbounded workspace for interventional tasks and provided an intuitive interface. Experimental validation, at component and system levels, confirmed the usability of the proposed methods for robot-assisted intervention systems

Musical Haptics

This Open Access book offers an original interdisciplinary overview of the role of haptic feedback in musical interaction. Divided into two parts, part I examines the tactile aspects of music performance and perception, discussing how they affect user experience and performance in terms of usability, functionality and perceived quality of musical instruments. Part II presents engineering, computational, and design approaches and guidelines that have been applied to render and exploit haptic feedback in digital musical interfaces. Musical Haptics introduces an emerging field that brings together engineering, human-computer interaction, applied psychology, musical aesthetics, and music performance. The latter, defined as the complex system of sensory-motor interactions between musicians and their instruments, presents a well-defined framework in which to study basic psychophysical, perceptual, and biomechanical aspects of touch, all of which will inform the design of haptic musical interfaces. Tactile and proprioceptive cues enable embodied interaction and inform sophisticated control strategies that allow skilled musicians to achieve high performance and expressivity. The use of haptic feedback in digital musical interfaces is expected to enhance user experience and performance, improve accessibility for disabled persons, and provide an effective means for musical tuition and guidance

Using pressure input and thermal feedback to broaden haptic interaction with mobile devices

Pressure input and thermal feedback are two under-researched aspects of touch in mobile human-computer interfaces. Pressure input could provide a wide, expressive range of continuous input for mobile devices. Thermal stimulation could provide an alternative means of conveying information non-visually. This thesis research investigated 1) how accurate pressure-based input on mobile devices could be when the user was walking and provided with only audio feedback and 2) what forms of thermal stimulation are both salient and comfortable and so could be used to design structured thermal feedback for conveying multi-dimensional information. The first experiment tested control of pressure on a mobile device when sitting and using audio feedback. Targeting accuracy was >= 85% when maintaining 4-6 levels of pressure across 3.5 Newtons, using only audio feedback and a Dwell selection technique. Two further experiments tested control of pressure-based input when walking and found accuracy was very high (>= 97%) even when walking and using only audio feedback, when using a rate-based input method. A fourth experiment tested how well each digit of one hand could apply pressure to a mobile phone individually and in combination with others. Each digit could apply pressure highly accurately, but not equally so, while some performed better in combination than alone. 2- or 3-digit combinations were more precise than 4- or 5-digit combinations. Experiment 5 compared one-handed, multi-digit pressure input using all 5 digits to traditional two-handed multitouch gestures for a combined zooming and rotating map task. Results showed comparable performance, with multitouch being ~1% more accurate but pressure input being ~0.5sec faster, overall. Two experiments, one when sitting indoors and one when walking indoors tested how salient and subjectively comfortable/intense various forms of thermal stimulation were. Faster or larger changes were more salient, faster to detect and less comfortable and cold changes were more salient and faster to detect than warm changes. The two final studies designed two-dimensional structured ‘thermal icons’ that could convey two pieces of information. When indoors, icons were correctly identified with 83% accuracy. When outdoors, accuracy dropped to 69% when sitting and 61% when walking. This thesis provides the first detailed study of how precisely pressure can be applied to mobile devices when walking and provided with audio feedback and the first systematic study of how to design thermal feedback for interaction with mobile devices in mobile environments

Space Station Systems: a Bibliography with Indexes (Supplement 8)

This bibliography lists 950 reports, articles, and other documents introduced into the NASA scientific and technical information system between July 1, 1989 and December 31, 1989. Its purpose is to provide helpful information to researchers, designers and managers engaged in Space Station technology development and mission design. Coverage includes documents that define major systems and subsystems related to structures and dynamic control, electronics and power supplies, propulsion, and payload integration. In addition, orbital construction methods, servicing and support requirements, procedures and operations, and missions for the current and future Space Station are included

Investigating systemic factors affecting science learning in Curriculum 2005 : case studies of two schools

The thesis illustrates the contention that an outcomes-based system with its underlying philosophy of social constructivism cannot operate effectively within a traditional school system. Restructuring of an institution is necessary to accommodate the outcomes-based system. Using the research instruments of interviews, questionnaires, journals, participant observations and collection of physical artefacts, two case studies investigating systemic factors as they influence science learning were conducted in two South African schools. The one school, St Sebastian's College, was an extremely well resourced school while the other, Mtunzini High School was a middle class school in comparatively deprived circumstances. Attempts were made to introduce an outcomes-based education course involving a group of grade 8 learners in the respective schools. Difficulties in implementation were encountered and at best only very limited success was achieved. There were two reasons for this. First, particularly in the case of St Sebastian's College, I designed a course which was over ambitious in that it was not suited to the developmental stage of the learners. Second, traditional schooling systems follow a perspective of education termed 'the structure of the disciplines' which fosters a system of rigid time tabling, compartmentalisation of subjects and emphasis upon summative assessment. In contrast, the curricula I designed involved a 'cognitive' perspective which required flexible time scheduling, integration of subjects and developmental assessment. The conflicts which arose include time constraints and resistant attitudes on the part of learners and teachers. The thesis culminates with some suggested steps to follow should a school community wish to restructure. These include a shared vision, employing organised abandonment, capacity building and commitment to a systemic perspective