On the Transfer of Vibrotactile Wearables from Science to Application

Abstract

Vibrotactile interfaces have been scientifically investigated and evaluated for more than a century. There is abundant evidence for multimodal integration and that tactile cues can deliver complementary information in a variety of contexts. Thus, tactile cues can relieve the strain on audition and vision in situations with high attentional load and broaden the bottleneck of attention. Further, in some contexts, tactile cues are processed intuitively and effortlessly. It has been shown that tactile information can support, supplement, and even substitute information processed by other modalities. The current strong trend towards wearable technology is opportune for vibrotactile applications. Despite the large body of research, many questions are still not resolved and comparatively few tactile appliances have found their way to commercialized everyday application so far. Yet, given the myriad possible application contexts, establishing tactile devices in everyday application seems adequate. Together with recent developments in the miniaturization of electronics and trends in wearable technology, the transfer of tactile devices to commercialization is a reasonable and possible undertaking. In this thesis, I document our pioneering work regarding the transfer of wearable vibrotactile hardware devices from science to practice. For this, I take three different transfer projects as case studies: the feelSpace naviGürtel®, a tactile orientation and navigation device for the blind; nextStep, a tactile cueing device for people suffering from Parkinson’s disease; and further, a yet unnamed approach to vibrotactile enhancement for trimodal digital learning. Case study 1: We transferred the feelSpace naviGürtel® after previous studies revealed the potential of a tactile belt as an intuitive, hands-free navigation and orientation device for people with visual impairment. For those with little or no residual vision, orienting and navigating is often challenging and stressful. Building on 10 years of research, we applied for and received an eXIST grant to conceptualize a commercial tactile navigation belt and research the market environment. In the process, we founded feelSpace GmbH. We applied for and received several other grants to investigate the needs of the users and to further develop hardware and software. In 2018, our feelSpace naviGürtel® reached the market: A belt with 16 vibromotors which can give directional information, either in a stand-alone manner as a tactile compass, or, in combination with a smartphone app, as a tactile navigation device. The naviGürtel® was well received by users. In 2020, we conducted a study with 11 blind participants to evaluate the everyday use of the device. We found a significant beneficial impact on emotional states of the users during navigation tasks. The transfer process of the naviGürtel®, from conceptualization to implementation and later evaluation, is documented in this thesis. Case study 2: In 2019, building on our experience with the transfer of tactile hardware, we joined the effort to transfer tactile cueing for patients suffering from Parkinson’s disease (PD). These patients often experience gait irregularities; in later stages of the disease, 60% suffer from freezing of gait (FOG), a sudden freezing of the movement when stepping. The sudden inability to move often leads to a fall. Research has shown that tactile cues can help to overcome the freeze of gait and help to initiate the next step. To gain experience with tactile cueing in this context, we conducted two studies. In the first study, set in a laboratory environment, we interviewed patients and gained first-hand insight into the effect of tactile cueing in a parkour task that was designed to elicit freezing of gait. In a second study, set up as a field study, we gave tactile belts as cueing aids to 8 participants and observed their use of the device for a period of 2 to 4 weeks, using interviews, questionnaires and behavioral measures. The two studies are documented in this thesis; building on the results, we further distilled a first blueprint for the transfer of a tactile cueing device, the NextStep. Case study 3: In a third application context, we investigated the concept of trimodal learning with tactile cueing. Given the rapid evolution of digital learning methods and their potential for individualized, multimodal learning approaches, we investigated in two studies whether tactile cues are beneficial in category learning, evaluating a transfer as a tactile learning aid. We taught participants a mapping with four tactile actuators (wrist, shoulder, belly button and center of the back) to four colors (red, green, blue and yellow). In the first study with 10 participants, we investigated whether tactile support would improve performance in a Stroop task with colors; we found that tactile signals had a significantly positive impact on reaction time and were processed in parallel to visual signals. In a second study with 14 participants, we investigated whether participants were supported by tactile signals when memorizing the font color of a fantasy word. While the tactile signals neither supported nor impaired response accuracy, we found evidence that the tactile information was processed in parallel to visual information during the memorization process. The two studies, documented in this thesis, are first steps to investigate and understand the opportunities of tactile signals in digital learning environments. Building on the scientific results and the practical experience, I discuss general phases and challenges we encountered during these transfers, shedding light on the “gap” between science and the market environment for vibrotactile wearables. I put our work in perspective and lay out a possible future of vibrotactile applications within the greater scope of personalized digital assistance. In summary, we find that there is a need for vibrotactile interfaces. At any point in time, there is more information available than the human brain can process. Processing a multitude of information in parallel, choosing the bits of information that are most relevant for an optimal interaction with the environment, and providing modern medical care through the use of digital technology are interesting contexts for tactile applications. We currently continue our transfer efforts with further tactile aiding devices, such as a tactile grasping aid for blind users. This work enhances the understanding of tactile sensory processing both in laboratory settings and in real-world applications, demonstrating the significant potential of vibrotactile cueing, particularly in the medical field