Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Advancing proxy-based haptic feedback in virtual reality

This thesis advances haptic feedback for Virtual Reality (VR). Our work is guided by Sutherland's 1965 vision of the ultimate display, which calls for VR systems to control the existence of matter. To push towards this vision, we build upon proxy-based haptic feedback, a technique characterized by the use of passive tangible props. The goal of this thesis is to tackle the central drawback of this approach, namely, its inflexibility, which yet hinders it to fulfill the vision of the ultimate display. Guided by four research questions, we first showcase the applicability of proxy-based VR haptics by employing the technique for data exploration. We then extend the VR system's control over users' haptic impressions in three steps. First, we contribute the class of Dynamic Passive Haptic Feedback (DPHF) alongside two novel concepts for conveying kinesthetic properties, like virtual weight and shape, through weight-shifting and drag-changing proxies. Conceptually orthogonal to this, we study how visual-haptic illusions can be leveraged to unnoticeably redirect the user's hand when reaching towards props. Here, we contribute a novel perception-inspired algorithm for Body Warping-based Hand Redirection (HR), an open-source framework for HR, and psychophysical insights. The thesis concludes by proving that the combination of DPHF and HR can outperform the individual techniques in terms of the achievable flexibility of the proxy-based haptic feedback.Diese Arbeit widmet sich haptischem Feedback für Virtual Reality (VR) und ist inspiriert von Sutherlands Vision des ultimativen Displays, welche VR-Systemen die Fähigkeit zuschreibt, Materie kontrollieren zu können. Um dieser Vision näher zu kommen, baut die Arbeit auf dem Konzept proxy-basierter Haptik auf, bei der haptische Eindrücke durch anfassbare Requisiten vermittelt werden. Ziel ist es, diesem Ansatz die für die Realisierung eines ultimativen Displays nötige Flexibilität zu verleihen. Dazu bearbeiten wir vier Forschungsfragen und zeigen zunächst die Anwendbarkeit proxy-basierter Haptik durch den Einsatz der Technik zur Datenexploration. Anschließend untersuchen wir in drei Schritten, wie VR-Systeme mehr Kontrolle über haptische Eindrücke von Nutzern erhalten können. Hierzu stellen wir Dynamic Passive Haptic Feedback (DPHF) vor, sowie zwei Verfahren, die kinästhetische Eindrücke wie virtuelles Gewicht und Form durch Gewichtsverlagerung und Veränderung des Luftwiderstandes von Requisiten vermitteln. Zusätzlich untersuchen wir, wie visuell-haptische Illusionen die Hand des Nutzers beim Greifen nach Requisiten unbemerkt umlenken können. Dabei stellen wir einen neuen Algorithmus zur Body Warping-based Hand Redirection (HR), ein Open-Source-Framework, sowie psychophysische Erkenntnisse vor. Abschließend zeigen wir, dass die Kombination von DPHF und HR proxy-basierte Haptik noch flexibler machen kann, als es die einzelnen Techniken alleine können

FPGA design and implementation of a framework for optogenetic retinal prosthesis

PhD ThesisThere are 285 million people worldwide with a visual impairment, 39 million of whom are completely blind and 246 million partially blind, known as low vision patients. In the UK and other developed countries of the west, retinal dystrophy diseases represent the primary cause of blindness, especially Age Related Macular Degeneration (AMD), diabetic retinopathy and Retinitis Pigmentosa (RP). There are various treatments and aids that can help these visual disorders, such as low vision aids, gene therapy and retinal prosthesis. Retinal prostheses consist of four main stages: the input stage (Image Acquisition), the high level processing stage (Image preparation and retinal encoding), low level processing stage (Stimulation controller) and the output stage (Image displaying on the opto-electronic micro-LEDs array). Up to now, a limited number of full hardware implementations have been available for retinal prosthesis. In this work, a photonic stimulation controller was designed and implemented. The main rule of this controller is to enhance framework results in terms of power and time. It involves, first, an even power distributor, which was used to evenly distribute the power through image sub-frames, to avoid a large surge of power, especially with large arrays. Therefore, the overall framework power results are improved. Second, a pulse encoder was used to select different modes of operation for the opto-electronic micro-LEDs array, and as a result of this the overall time for the framework was improved. The implementation is completed using reconfigurable hardware devices, i.e. Field Programmable Gate Arrays (FPGAs), to achieve high performance at an economical price. Moreover, this FPGA-based framework for an optogenetic retinal prosthesis aims to control the opto-electronic micro-LED array in an efficient way, and to interface and link between the opto-electronic micro-LED array hardware architecture and the previously developed high level retinal prosthesis image processing algorithms.University of Jorda

A Hybrid Visual Control Scheme to Assist the Visually Impaired with Guided Reaching Tasks

In recent years, numerous researchers have been working towards adapting technology developed for robotic control to use in the creation of high-technology assistive devices for the visually impaired. These types of devices have been proven to help visually impaired people live with a greater degree of confidence and independence. However, most prior work has focused primarily on a single problem from mobile robotics, namely navigation in an unknown environment. In this work we address the issue of the design and performance of an assistive device application to aid the visually-impaired with a guided reaching task. The device follows an eye-in-hand, IBLM visual servoing configuration with a single camera and vibrotactile feedback to the user to direct guided tracking during the reaching task. We present a model for the system that employs a hybrid control scheme based on a Discrete Event System (DES) approach. This approach avoids significant problems inherent in the competing classical control or conventional visual servoing models for upper limb movement found in the literature. The proposed hybrid model parameterizes the partitioning of the image state-space that produces a variable size targeting window for compensatory tracking in the reaching task. The partitioning is created through the positioning of hypersurface boundaries within the state space, which when crossed trigger events that cause DES-controller state transition that enable differing control laws. A set of metrics encompassing, accuracy ($D$), precision ($\theta_{e}$), and overall tracking performance ($\psi$) are also proposed to quantity system performance so that the effect of parameter variations and alternate controller configurations can be compared. To this end, a prototype called \texttt{aiReach} was constructed and experiments were conducted testing the functional use of the system and other supporting aspects of the system behaviour using participant volunteers. Results are presented validating the system design and demonstrating effective use of a two parameter partitioning scheme that utilizes a targeting window with additional hysteresis region to filtering perturbations due to natural proprioceptive limitations for precise control of upper limb movement. Results from the experiments show that accuracy performance increased with the use of the dual parameter hysteresis target window model ($0.91 \leq D \leq 1$, $\mu(D)=0.9644$, $\sigma(D)=0.0172$) over the single parameter fixed window model ($0.82 \leq D \leq 0.98$, $\mu(D)=0.9205$, $\sigma(D)=0.0297$) while the precision metric, $\theta_{e}$, remained relatively unchanged. In addition, the overall tracking performance metric produces scores which correctly rank the performance of the guided reaching tasks form most difficult to easiest

Clinical translation of a regeneration strategy for spinal cord injury

The complex and vulnerable tissue of the spinal cord does not heal after injury, leaving patients with lifelong disability after spinal cord injury (SCI). Many milestones have been reached during the last century through specialized centers for SCI, greatly increasing life expectancy and quality of life by battling common medical problems such as urinary tract infections, pressure ulcers, spasticity, neurogenic pain, and sexual function as well as providing means of rehabilitation to a meaningful and productive life after SCI. Despite the advances in preclinical knowledge of mechanisms in SCI and several clinical trials completed, to date no pivotal treatment exists for acute spinal cord injury or for the regeneration of lost function in the chronic state. The first reports of experimental regeneration of central axons through peripheral nerve grafts are more than a century old. In the last decades, regeneration of function after SCI has been reported by several research groups in different species using peripheral nerve grafts and FGF1. The regeneration strategy was furthered refined in our group by the use of a biodegradable scaffold for exact positioning of the nerve grafts. This thesis describes the translational process to reach a clinical trial of glial scar resection and implantation of peripheral nerve grafts and FGF1 using a biodegradable guiding scaffold. In paper I, we show that both the cranial and caudal demarcation of a thoracic spinal cord injury can be defined with electromyography of intercostal muscles in chronic SCI patients. We also present an MRI protocol with acceptable image contrast despite the presence of spinal instrumentation and showed that the injury length found with electromyography correlates well with length of injury on MRI. In paper II, we use a novel conversion table between spinal cord neuronal segments and vertebral segments and combine data on human spinal cord cross-sectional diameters from different published sources to yield continuous estimates on human spinal cord size and variability. In paper III, we describe the design of a set of spinal cord injury guiding devices based on the data from paper II, covering the normal variability found in human thoracic spinal cord segments T2–T12 with an acceptable error-of-fit for the elliptical shape as well as guiding channels proposed. In paper IV, we detail the adverse events reported during the first 60 days postoperatively in the ongoing clinical trial “Safety and Efficacy of SC0806 (Fibroblast Growth Factor 1 and a Device) in Traumatic Spinal Cord Injury Subjects.” Early results from the first six complete (AIS-A) thoracic spinal cord injury subjects operated on in the ongoing trial show that with precise preoperative and intraoperative neurophysiology, surgery and implantation can be performed without negative effects on neurological level, and safety and tolerability are acceptable to merit the continuation of the trial. In paper V, we describe the construction of a cost-effective light-sheet microscope by modification of an outdated microarray-scanner. The microscope was applied to an experimental model of hypoglossal nerve avulsion injury, and proliferation of Iba1+ cells could be quantified automatically demonstrating a possible application of the microscope. In conclusion, reaching clinical trial in a translational process is a significant and collaborative undertaking requiring co-operation of multiple institutions and professions as well as rigorous external control of data quality and adverse events to ensure safety of study subjects. The papers in this thesis detail some relevant steps necessary for the clinical translation of regeneration strategies in chronic SCI

Enhanced Virtuality: Increasing the Usability and Productivity of Virtual Environments

Mit stetig steigender Bildschirmauflösung, genauerem Tracking und fallenden Preisen stehen Virtual Reality (VR) Systeme kurz davor sich erfolgreich am Markt zu etablieren. Verschiedene Werkzeuge helfen Entwicklern bei der Erstellung komplexer Interaktionen mit mehreren Benutzern innerhalb adaptiver virtueller Umgebungen. Allerdings entstehen mit der Verbreitung der VR-Systeme auch zusätzliche Herausforderungen: Diverse Eingabegeräte mit ungewohnten Formen und Tastenlayouts verhindern eine intuitive Interaktion. Darüber hinaus zwingt der eingeschränkte Funktionsumfang bestehender Software die Nutzer dazu, auf herkömmliche PC- oder Touch-basierte Systeme zurückzugreifen. Außerdem birgt die Zusammenarbeit mit anderen Anwendern am gleichen Standort Herausforderungen hinsichtlich der Kalibrierung unterschiedlicher Trackingsysteme und der Kollisionsvermeidung. Beim entfernten Zusammenarbeiten wird die Interaktion durch Latenzzeiten und Verbindungsverluste zusätzlich beeinflusst. Schließlich haben die Benutzer unterschiedliche Anforderungen an die Visualisierung von Inhalten, z.B. Größe, Ausrichtung, Farbe oder Kontrast, innerhalb der virtuellen Welten. Eine strikte Nachbildung von realen Umgebungen in VR verschenkt Potential und wird es nicht ermöglichen, die individuellen Bedürfnisse der Benutzer zu berücksichtigen. Um diese Probleme anzugehen, werden in der vorliegenden Arbeit Lösungen in den Bereichen Eingabe, Zusammenarbeit und Erweiterung von virtuellen Welten und Benutzern vorgestellt, die darauf abzielen, die Benutzerfreundlichkeit und Produktivität von VR zu erhöhen. Zunächst werden PC-basierte Hardware und Software in die virtuelle Welt übertragen, um die Vertrautheit und den Funktionsumfang bestehender Anwendungen in VR zu erhalten. Virtuelle Stellvertreter von physischen Geräten, z.B. Tastatur und Tablet, und ein VR-Modus für Anwendungen ermöglichen es dem Benutzer reale Fähigkeiten in die virtuelle Welt zu übertragen. Des Weiteren wird ein Algorithmus vorgestellt, der die Kalibrierung mehrerer ko-lokaler VR-Geräte mit hoher Genauigkeit und geringen Hardwareanforderungen und geringem Aufwand ermöglicht. Da VR-Headsets die reale Umgebung der Benutzer ausblenden, wird die Relevanz einer Ganzkörper-Avatar-Visualisierung für die Kollisionsvermeidung und das entfernte Zusammenarbeiten nachgewiesen. Darüber hinaus werden personalisierte räumliche oder zeitliche Modifikationen vorgestellt, die es erlauben, die Benutzerfreundlichkeit, Arbeitsleistung und soziale Präsenz von Benutzern zu erhöhen. Diskrepanzen zwischen den virtuellen Welten, die durch persönliche Anpassungen entstehen, werden durch Methoden der Avatar-Umlenkung (engl. redirection) kompensiert. Abschließend werden einige der Methoden und Erkenntnisse in eine beispielhafte Anwendung integriert, um deren praktische Anwendbarkeit zu verdeutlichen. Die vorliegende Arbeit zeigt, dass virtuelle Umgebungen auf realen Fähigkeiten und Erfahrungen aufbauen können, um eine vertraute und einfache Interaktion und Zusammenarbeit von Benutzern zu gewährleisten. Darüber hinaus ermöglichen individuelle Erweiterungen des virtuellen Inhalts und der Avatare Einschränkungen der realen Welt zu überwinden und das Erlebnis von VR-Umgebungen zu steigern

The Motion-Lab: A Virtual Reality Laboratory for Spatial Updating Experiments

The main question addressed in the Motion-Lab is: How do we know where we are? Normally, humans know where they are with respect to the immediate surround. The overall perception of this environment results from the integration of multiple sensory modalities. Here we use Virtual Reality to study the interaction of visual, vestibular, and proprioceptive senses and explore the way these senses might be integrated into a coherent perception of spatial orientation and location. This Technical Report describes a Virtual Reality laboratory, its technical implementation as a distributed network of computers and discusses its usability for experiments designed to investigate questions of spatial orientation

Investigating the Feasibility of Using Focussed Airborne Ultrasound as Tactile Feedback in Medical Simulators

Novice medical practitioners commonly practice on live patients in real medical procedures. However, due to the inexperience of the practitioner, mistakes are likely which exposes the patient to undue risk. To improve the training of novices, medical simulators create a virtual patient providing a safe environment for the user to practice within. An important clinical skill is palpation, a physical examination technique. The practitioners use their hands to feel the body of the patient to make diagnosis. A virtual patient has a visual representation but as it is virtual, the patient is not physically present. Haptics technology provide additional benefits to the training session by stimulating the physical sense of touch. A novel technique has recently emerged for stimulating tactile sensation called acoustic radiation pressure from focussed airborne ultrasound. Acoustic radiation creates a focal point of concentrated acoustic pressure in a three-dimensional field producing a force in mid-air. Airborne ultrasound has several advantages over conventional technologies. It was also initially theorised that using airborne ultrasound to simulate palpation compared to a previous system called PalpSim which consists of a rubber tube filled with water permanently embedded in a block of silicone, will offer better controllability over the displayed sensation to simulate various tactile sensations. The thesis has investigated the feasibility of using focussed airborne ultrasound as tactile feedback in medical simulators. A tactile device called UltraSendo was completely custom built to simulate an arterial pulse and a thrill sensation. UltraSendo was integrated with an augmented reality simulator displaying a virtual patient for user interaction. The simulator was brought to Ysbyty Glan Clwyd hospital for user feedback. A wide range of user responses were gathered. The majority of responses felt the arterial pulse was not sufficiently realistic whilst there were higher ratings for the thrill sensation which is acceptably realistic. Positive feedback suggests that airborne ultrasound can indeed provide tactile feedback in a medical context and is better at simulating a thrill sensation compared to a pulse sensation