Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Prosthetic Control and Sensory Feedback for Upper Limb Amputees

Hand amputation could dramatically degrade the life quality of amputees. Many amputees use prostheses to restore part of the hand functions. Myoelectric prosthesis provides the most dexterous control. However, they are facing high rejection rate. One of the reasons is the lack of sensory feedback. There is a need for providing sensory feedback for myoelectric prosthesis users. It can improve object manipulation abilities, enhance the perceptual embodiment of myoelectric prostheses and help reduce phantom limb pain. This PhD work focuses on building bi-directional prostheses for upper limb amputees. In the introduction chapter, first, an overview of upper limb amputee demographics and upper limb prosthesis is given. Then the human somatosensory system is briefly introduced. The next part reviews invasive and non-invasive sensory feedback methods reported in the literature. The rest of the chapter describes the motivation of the project and the thesis organization. The first step to build a bi-directional prostheses is to investigate natural and robust multifunctional prosthetic control. Most of the commerical prostheses apply non-pattern recognition based myoelectric control methods, which offers only limited functionalities. In this thesis work, pattern recognition based prosthetic control employing three commonly used and representative machine learning algorithms is investigated. Three datasets involving different levels of upper arm movements are used for testing the algorithm effectiveness. The influence of time-domain features, window and increment sizes, algorithms, and post-processing techniques are analyzed and discussed. The next three chapters address different aspects of providing sensory feedback. The first focus of sensory feedback process is the automatic phantom map detection. Many amputees have referred sensation from their missing hand on their residual limbs (phantom maps). This skin area can serve as a target for providing amputees with non-invasive tactile sensory feedback. One of the challenges of providing sensory feedback on the phantom map is to define the accurate boundary of each phantom digit because the phantom map distribution varies from person to person. Automatic phantom map detection methods based on four decomposition support vector machine algorithms and three sampling methods are proposed. The accuracy and training/ classification time of each algorithm using a dense stimulation array and two coarse stimulation arrays are presented and compared. The next focus of the thesis is to develop non-invasive tactile display. The design and psychophysical testing results of three types of non-invasive tactile feedback arrays are presented: two with vibrotactile modality and one with multi modality. For vibrotactile, two types of miniaturized vibrators: eccentric rotating masses (ERMs) and linear resonant actuators (LRAs) were first tested on healthy subjects and their effectiveness was compared. Then the ERMs are integrated into a vibrotactile glove to assess the feasibility of providing sensory feedback for unilateral upper limb amputees on the contralateral hand. For multimodal stimulation, miniature multimodal actuators integrating servomotors and vibrators were designed. The actuator can be used to deliver both high-frequency vibration and low-frequency pressures simultaneously. By utilizing two modalities at the same time, the actuator stimulates different types of mechanoreceptors and thus h

Thermal and wind devices for multisensory human-computer interaction: an overview

In order to create immersive experiences in virtual worlds, we need to explore different human senses (sight, hearing, smell, taste, and touch). Many different devices have been developed by both industry and academia towards this aim. In this paper, we focus our attention on the researched area of thermal and wind devices to deliver the sensations of heat and cold against people’s skin and their application to human-computer interaction (HCI). First, we present a review of devices and their features that were identified as relevant. Then, we highlight the users’ experience with thermal and wind devices, highlighting limitations either found or inferred by the authors and studies selected for this survey. Accordingly, from the current literature, we can infer that, in wind and temperature-based haptic systems (i) users experience wind effects produced by fans that move air molecules at room temperature, and (ii) there is no integration of thermal components to devices intended for the production of both cold or hot airflows. Subsequently, an analysis of why thermal wind devices have not been devised yet is undertaken, highlighting the challenges of creating such devices.Espírito Santo Research and Innovation Foundation (FAPES, Brazil) - Finance Code 2021-GL60J), the Coordination for the Improvement of Higher Education Personnel (CAPES, Brazil) - Finance Code 88881.187844/2018-01 and 88887.570688/2020-00 and by the National Council for Scientific and Technological (CNPq, Brazil) - Finance Code 307718/2020-4. The work was also funded by the European Union’s Horizon 2020 Research and Innovation programme under Grant Agreement no. 688503. E. B. Saleme additionally acknowledges aid from the Federal Institute of Espírito Santo

Requirements for a tactile display of softness

Developing tactile displays is an important aspect of improving the realism of feeling softness in laparoscopic surgery. One of the major challenges of designing a tactile display is to understand how the perception of touch can be perceived with differences in material properties. This project seeks to address this limitation by investigating how the interaction of material properties affects perception of softness and to present the perception of softness through a tactile display. The first aim explores how the interaction of material properties affects perception of softness through the use of two psychophysical experiments. Experiments used a set of nine stimuli representing three materials of different compliance, with three different patterns of surface roughness or with three different coatings of stickiness. The results indicated that compliance affected perception of softness when pressing the finger, but not when sliding; and that compliance, friction and thermal conductivity all influenced the perception of softness. To achieve the second aim of reproducing various levels of softnesses, the tactile display was built at the University of Leeds. The displayed softness was controlled by changing the contact area and tension of a flexible sheet. Psychophysical experiments were conducted to evaluate how well humans perceive softness through the display. The data was analysed using MatLab to plot psychometric functions. The results indicated that the tactile display might be good for some applications which need to compare between simulated softnesses, but it might be insufficient for other applications which need to compare between simulated softness and real samples

Development of a Tactile Thimble for Augmented and Virtual Reality Applications

The technologies that have gained a renewed interest during the recent years are Virtual Reality (VR) and Augmented Reality (AR), as they become more accessible and affordable for mass-production. The input device which allows us to interact with the virtual environment is a very crucial aspect. One of the main barriers to immerse ourselves in virtual reality is the lack of realistic feedback. The user has to almost rely entirely on visual feedback without any haptic feedback, and this increases the user's workload and decreases the performance. In this thesis, a functional demonstrator of a tactile feedback device which conveys compelling interactions with not just VR, but also AR is presented. The device is designed such that there is realistic feedback for virtual touches and least obstruction during contact of a real object in AR applications. New design principle of introducing small actuators allows the device to be compact and increases its portability. In contrast to actuators that are placed on the finger pad in most of the available input devices for VR, a tactile device with two actuators that are arranged laterally on the finger, so that the underside of the fingertip is free is proposed. The output from these actuators generate a tactile stimulus by stimulating a sense of touch, which helps the user to manipulate virtual objects. The actuators are designed to independently generate vibrations and this coupled tactile feedback enhances the stimulation resulting in a wide variety of stimulation patterns for the sense of touch. Preliminary experimental evaluation for design and location of actuators has been carried out to measure the vibration intensity. In addition, user experiments for design evaluation of the two actuators based on different vibration patterns have also been conducted

Application of ultrasonic motors to MR-compatible haptic interfaces

Functional Magnetic Resonance Imagery (fMRI) is an imaging technique allowing the observation of brain activity. Haptic interfaces can be used in conjunction with fMRI to stimulate the subject while measuring brain activity. Using robotic stimulation over conventional methods offers repeatability, flexibility and the possibility of logging of different experiment variables. Such system becomes a powerful tool for neuroscience study, diagnostic and rehabilitation. The MR scanner with its high magnetic fields and radio frequency pulses is a harsh environment for a robotic system. Robots that can operate safely and do not induce disturbances in the imaging of the scanner are qualified as MR-compatible. The actuation of these robots is an important issue. Electrical power brought to the actuator represents an important source of interferences with the scanner. Since electrical motors cannot be introduced in the MR room, haptic interfaces are conventionally remotely actuated over a long transmission with the actuators placed outside of the MR room. In particular cases, such as the study of finger motion, small haptic interfaces with limited force ranges are required. Remote actuation methods impose transmission lengths and means that cannot be reduced nor scaled down thus imposing a trade-off between performances and size reduction in these applications. This work investigates an alternative actuator that can achieve high-quality force-interactions with the fingers. The Ultrasonic Motor (USM) is MR-compatible and offers good performances. But it is not well suited for force-feedback and may be hazardous for the users. To address these issues, mechanical solutions are investigated by using an electrical analogy applied to mechanical systems. A novel actuation system using the USM as a power source and a clutch to control the output torque is proposed: the Hybrid USM Clutch Actuator (HUCA). The first prototype validates the different mechanical concepts developed in this work. The second, MR-compatible, integrates a clutch based on electrorheological fluids (ER). MR-compatibility has been validated and performances evaluated. Since the HUCA has the unique property of behaving both like a force source and a velocity source, dedicated control schemes have been developed to implement impedance and admittance force control. These enable the display of stiff walls and the rendering of a wide range of impedances thanks to the overlap of their range of displayable impedances. Compared to the hydrostatic transmission actuation, the HUCA shows higher performances and user safety. Furthermore, the powering through electrical wires allows developments of multi-DOF interfaces

How to Build an Embodiment Lab: Achieving Body Representation Illusions in Virtual Reality

Advances in computer graphics algorithms and virtual reality (VR) systems, together with the reduction in cost of associated equipment, have led scientists to consider VR as a useful tool for conducting experimental studies in fields such as neuroscience and experimental psychology. In particular virtual body ownership, where the feeling of ownership over a virtual body is elicited in the participant, has become a useful tool in the study of body representation, in cognitive neuroscience and psychology, concerned with how the brain represents the body. Although VR has been shown to be a useful tool for exploring body ownership illusions, integrating the various technologies necessary for such a system can be daunting. In this paper we discuss the technical infrastructure necessary to achieve virtual embodiment. We describe a basic VR system and how it may be used for this purpose, and then extend this system with the introduction of real-time motion capture, a simple haptics system and the integration of physiological and brain electrical activity recordings

Development of Ultrasonic Devices for Non-destructive Testing: Ultrasonic Vibro-tactile Sensor and FPGA-Based Research Platform

This thesis is focused on the development of ultrasonic devices for industrial non-destructive testing (NDT). Ultrasound is generated from mechanical vibrations and then propagates through the medium. Ultrasonic devices can make use of the ultrasound in both aspects, vibrations and propagations, to perform inspections of the objects. To this end, two devices were developed in this research, each pertaining to NDT of the objects. The first device is the vibro-tactile sensor which aims to estimate the elastic modules of soft materials with minimally invasive technique. Inspired by load sensitivity studies in the high-power ultrasonic applications, vibration characteristics in resonance were utilized to perform the inspection. Only a minimal force to ensure contact with the object surface needs to be applied for a vibro-tactile sensor to perform inspection of the object; hence, it can be used for in-vivo measurement of the soft materials’ elastic moduli without causing severe surface deformation. The design and analysis of the device were carried out using the electro-mechanical analogy to address the electro-mechanical nature of piezoelectric devices. The designed vibro-tactile sensor resonates at ~40 kHz and can be applied to differentiate the elastic modulus of isotropic soft samples with a range from 10 kPa to 70 kPa. The second device developed is a field-programmable development platform for ultrasonic pulse-echo testing. Ultrasonic testing, utilizing sound wave propagation, is a widely used technique in the industry. The commercially available equipment for industrial NDT is highly dependent on the competence of the inspector and rarely provides the access to raw data. For successful transition from traditional labor-intensive manufacturing to the next generation “smart factory” where intelligent machines replace human labor, inspection equipment with automated in-line data collection and processing capability is highly needed. To this end, a flexible platform which provides the access to raw data for algorithm development and implementation should be established. Therefore, an affordable, versatile, and researcher-friendly development platform based on field-programmable gate array (FPGA) was developed in the research. Both hardware and software development tools and procedures were discussed. In the lab experiment, the developed prototype exhibited its competence in NDT applications and successfully carried out hardware-based auto-detection algorithm for mm-level defects on steel and aluminum specimens. Comparisons with commercial systems were provided to guide future development

Multi-Finger Haptic Devices Integrating Miniature Short-Stroke Actuators

The omnipresence of electronic devices in our everyday life goes together with a trend that makes us always more immersed during their utilization. By immersion, we mean that during the development of a new product, it is more and more required to stimulate several senses of the user so as to make the product more attractive. The sense of touch does not escape the rule and is more and more considered. Definitely democratized by its integration in smart phones with touchscreens, the haptic feedback allows enhancing the human-machine interactions in many ways. For instance by improving the comfort of use of a button through the modification of its force feedback. It can also offer an interactive experience during the manipulation of digital information and even improve the communication, particularly through the internet and for blind people, with the introduction of non-verbal signals. For these reasons, the present thesis focuses on the conception of multi-finger haptic devices, a new kind of peripherals integrating multiple actuators and capable of providing a fully programmable force feedback to the user's fingers. A global methodology is presented, outlining the different constituents necessary for their conception: actuator, sensor, control, communication and software user interface. Then, generic tools corresponding to the two first elements are presented. An accurate modeling of miniature electromagnetic short-stroke actuators is made possible thanks to the combination of 3D finite element modeling (FEM) and design of experiments (DOE). The non-usual behavior of magnetic flux lines in miniature actuators with relatively large airgaps imposes to avoid simplified analytical models and to use the reliable results of finite elements. The long computation times required by 3D FEM are balanced by the use of selective DOE making the modeling methodology easily adaptable, rapid and accurate. The parametrical model of the force provided by the modeling methodology is then integrated in a full parametrical setup allowing for the optimization of the actuator force using a conventional algorithm. The advantage of the parametrical optimization is that complementary non-linear constraints such as weight and temperature can be added, making the model multi-physic. Then, several original position measurement techniques using existing sensors are developed including a low-cost custom single-photointerrupter sensor allowing for direction discrimination for fast-prototyping and a hybrid sensing method using tiny Hall sensors and taking advantage of the leaks of the main actuator magnet. Two innovative self-sensing methods are then presented, allowing for the measurement of the mover position of linear short-stroke actuators. The first solution estimates the position of the coil by measuring the acceleration through the back emf. However in this case, a constant acceleration is required, which strongly restrains the application scope. The second solution allows for a real-time measurement of the position thanks to a passive oscillating RLC circuit influenced by the variation of the coil impedance. All the solutions presented are low-cost, compact and require few computation resources. Finally, in order to illustrate the methodology proposed along the thesis, several prototypes are fabricated, giving an overview of the possibilities offered by multi-finger haptic devices. A haptic numeric pad is notably used in an experiment made in collaboration with the University Service of Child and Adolescent Psychiatry in Lausanne with the aim of improving the impaired emotional processing of psychotic adolescents. Moreover, the successful identification of several touch sensations on the same haptic pad lays the first stones of a new tactile language