Artificial Tactile System and Signal Processing for Haptic applications

Human have the ability to interact with the external environment through five main senses which are vision, hearing, smell, taste and touch. Most of all, the sensation like vision or hearing have been well developed and the use of various applications like TV, Camera, or artificial cochlear have been widely generalized. As the next steps, recently, the tactile sensor to mimic the tactile system of human have been attracted by many groups. Especially, after the development of Apple’s iPhone, the public interest about touch sensing applications have been increased explosively. Other researches for tactile sensing have focused on enhancing the performance of tactile sensor like the sensitivity, stability, response time and so on. As a result, there are some researches that the sensor performance of certain criteria is better than that of human tactile system. However, a human tactile system is not only very sensitive but also complex. In other words, ultimately, the tactile system mimicking the human tactile sensation should detect various parameters such as the pressure, temperature, hardness or roughness and also decide the psychological feeling like the pain by a hot material in touching or the smooth/roughness feeling in sliding the certain material. Therefore, in this thesis, it has been studied for the development of multifunctional tactile sensing system detecting various tactile parameters and deciding the kinds of psychological tactile feeling by measured stimulation. As the first step for the development of tactile system, we have studied the tactile sensor using ZnO nanowire. Therefore, in this chapter, the basic characteristics of ZnO nanowire are investigated to confirm the possibility for the tactile sensor. In addition, structural design factors of sensor units have been studied in order to enhance the sensitivity of ZnO nanowire-based tactile sensor. We have primarily demonstrated the effect of a square pattern array design in a pressure sensor using ZnO nanowires. Nanowires grown on the edge of cells can be bent easily because of growth direction, density of nanowires, and buckling effect. Since smaller square pattern arrays induce a higher circumference to cell area ratio, if one sensor unit consists of many micro-level square pattern arrays, the design enhances the piezoelectric efficiency and the sensitivity. As a result, 20um × 20um cell arrays showed three times higher pressure sensitivity than 250um × 250um cell array structures at a pressure range from 4kPa to 14kPa. The induced piezoelectric voltage with the same pressure level also increased drastically. Therefore, the smaller pattern array design is more appropriate for a high-sensitive pressure sensor than a simple one-body cell design for tactile systems, and it has the advantage of better power efficiency, which is also important for artificial tactile systems. Even if, in previous experiments, the possibility of piezoelectric materials as the tactile sensor and the method for the enhancement of pressure sensitivity are confirmed well, the tactile sensor for mimicking the human tactile sensation should measure various parameters as well as the pressure. However, many studies about ‘smooth-rough’ sensation depend on the machine learning technology with simple tactile sensors rather than developing the sensors that can measure various parameters like surface topography, hardness, quality of materials at the same time. Therefore, after the development of the pressure sensor, specific structures based on PDMS are proposed to measure and analyze above-mentioned parameters related to ‘smooth-rough’ decision, as like fingerprint of human. To find the optimized structure, three kinds of the structure shape (cone, cylinder and dome) are fabricated and the pressure sensitivity according to the shape are also measured. FEM simulation is also carried out to support the experimental result. Our tactile sensor with optimized dome structure (500um height) provides high shear force sensitivity, fast response time, stability, and durability. The high sensitivity about the shear force enables better the tactile sensor to measure the various surface information such as the pitch of pattern, the depth, the sliding velocity, the hardness and so on. In addition, after the study to measure the various surface information by dome structure, the research to measure the other surface information is also followed. In our previous study, we confirmed that the surface topography can be reconstructed by mapping the piezoelectric signals according to the location. In this research, to reduce the number of measurements from dozens to once and minimize the data loss at the empty space between adjacent sensors, the electrode array of Zig-Zag type is applied to the tactile sensor. As a result, with just one measurement, the surface topography of broad region can be successfully reconstructed by our tactile sensor as the high-resolution image. Additionally, the temperature sensor based on the resistive mechanism is fabricated between the Zig-Zag electrode lines to measures the temperature of surface materials when the tactile sensor rubs on the materials in real time. Over the development of the tactile sensing applications, the demand for an artificial system like human tactile sensation have been much more increased. Therefore, in this study, as a surrogate for human tactile sensation, we propose an artificial tactile sensing system based on the developed sensors in previous sections. For this, the piezoelectric tactile signal generated by touching and rubbing the material is transferred to DAQ system connected with our tactile sensor. First, the system decides whether the contacted material is dangerous or not. If dangerous like sharp or hot materials, the warning signal is generated by our artificial tactile system. If not, the sensor connected with the system rubs the materials and detects the roughness of the materials. Especially, the human test data related to ‘soft-rough’ detection is applied to a deep learning structure allowing personalization of the system, because tactile responses vary among humans. This approach could be applied to electronic devices with tactile emotional exchange capabilities, as well as various advanced digital experiences. In this thesis, human-like tactile sensing system based on the piezoelectric effect is successfully confirmed through various experiments. Although there are still some issues that need to be improved, this research is expected to be fundamental results for human-like tactile sensing system detecting a variety of the parameters such as the pressure, temperature, surface morphology, hardness, roughness and so on. In the future, through collaborative research with other fields like brain science, signal processing, we hope that this research can mimic psychological tactile sensations and communicate emotional exchange with external environment like real human skin.YList of Contents Abstract i List of contents iii List of tables vi List of figures vii Ⅰ. INTRODUCTION 1 1.1 Motivation 1 1.2 Various transduction mechanisms for the tactile sensor 5 1.2.1 Capacitive mechanism 5 1.2.2 Resistive mechanism 6 1.2.3 Triboelectric effect 7 1.2.4 Piezoelectric effect 9 1.3 Objectives 12 1.4 Reference 13 II. BASIC CHARACTERISTICS AND THE METHOD FOR ENHANC-ING THE PRESSURE SENSITIVITY OF THE TACTILE SENSOR BASED ON ZnO NANOWIRE 19 2.1 Introduction 19 2.2 Basic characteristics of ZnO nanowire 22 2.3 Device Fabrication 31 2.4 Morphological and Electrical characteristics 33 2.5 Pattern structure for enhanced for pressure sensitivity 38 2.6 Simulation result of piezoelectric effect for pattern structure 42 2.7 Reference 46 III. DOME STRUCTURE TO MEAUSRE THE SURFACE INFOR-MATION 52 3.1 Introduction 52 3.2 Basic characteristics of P(VDF-TrFE) 53 3.3 Device fabrication 61 3.4 Interaction mechanism between dome structure and surface material 63 3.5 Simulation and Experimental result comparing cone, cylinder, and dome structure 64 3.6 Simulation and Experimental result of the sensitivity enhancement ef-fect by dome structure 66 3.7 Depth measurement by tactile sensor with dome structure 72 3.8 Pattern of pitch by multi-array tactile sensor with dome structure 77 3.9 Hardness measurement by the tactile sensor with dome structure 79 3.10 Reference 83 IV. ZIG-ZAG ARRAYED TACTILE SENSOR BASED ON PIEZOE-LECTRIC-RESISTIVE MECHANISM TO DETECT THE SURFACE TOPOG-RAPHY AND TEMPERATURE 87 4.1 Introduction 87 4.2 Device fabrication 88 4.3 Piezoelectric characteristics of fabricated tactile sensor 90 4.4 Surface rendering method by the piezoelectric effect 95 4.5 Surface rendering result of 3D printed materials 96 4.6 Temperature sensing in sliding the high temperature material on Zig-Zag tactile sensor 99 4.7 Reference 103 V. TACTILE SENSING SYSTEM FOR PAIN AND SMOOTH/ROUGH DETECTION 105 5.1 Introduction 105 5.2 Components of the tactile sensing system 107 5.3 Artificial tactile sensing system for generating the pain warning 108 5.4 Artificial tactile sensing system for smooth/rough sensing 112 5.5 Reference 117 VⅠ. CONCLUSION 120DoctordCollectio

Seedless hydrothermal growth of ZnO nanorods as a promising route for flexible tactile sensors

Hydrothermal growth of ZnO nanorods has been widely used for the development of tactile sensors, with the aid of ZnO seed layers, favoring the growth of dense and vertically aligned nanorods. However, seed layers represent an additional fabrication step in the sensor design. In this study, a seedless hydrothermal growth of ZnO nanorods was carried out on Au-coated Si and polyimide substrates. The effects of both the Au morphology and the growth temperature on the characteristics of the nanorods were investigated, finding that smaller Au grains produced tilted rods, while larger grains provided vertical rods. Highly dense and high-aspect-ratio nanorods with hexagonal prismatic shape were obtained at 75 °C and 85 °C, while pyramid-like rods were grown when the temperature was set to 95 °C. Finite-element simulations demonstrated that prismatic rods produce higher voltage responses than the pyramid-shaped ones. A tactile sensor, with an active area of 1 cm2, was fabricated on flexible polyimide substrate and embedding the nanorods forest in a polydimethylsiloxane matrix as a separation layer between the bottom and the top Au electrodes. The prototype showed clear responses upon applied loads of 2–4 N and vibrations over frequencies in the range of 20–800 Hz

Distributed Sensing and Stimulation Systems Towards Sense of Touch Restoration in Prosthetics

Modern prostheses aim at restoring the functional and aesthetic characteristics of the lost limb. To foster prosthesis embodiment and functionality, it is necessary to restitute both volitional control and sensory feedback. Contemporary feedback interfaces presented in research use few sensors and stimulation units to feedback at most two discrete feedback variables (e.g. grasping force and aperture), whereas the human sense of touch relies on a distributed network of mechanoreceptors providing high-fidelity spatial information. To provide this type of feedback in prosthetics, it is necessary to sense tactile information from artificial skin placed on the prosthesis and transmit tactile feedback above the amputation in order to map the interaction between the prosthesis and the environment. This thesis proposes the integration of distributed sensing systems (e-skin) to acquire tactile sensation, and non-invasive multichannel electrotactile feedback and virtual reality to deliver high-bandwidth information to the user. Its core focus addresses the development and testing of close-loop sensory feedback human-machine interface, based on the latest distributed sensing and stimulation techniques for restoring the sense of touch in prosthetics. To this end, the thesis is comprised of two introductory chapters that describe the state of art in the field, the objectives and the used methodology and contributions; as well as three studies distributed over stimulation system level and sensing system level. The first study presents the development of close-loop compensatory tracking system to evaluate the usability and effectiveness of electrotactile sensory feedback in enabling real-time close-loop control in prosthetics. It examines and compares the subject\u2019s adaptive performance and tolerance to random latencies while performing the dynamic control task (i.e. position control) and simultaneously receiving either visual feedback or electrotactile feedback for communicating the momentary tracking error. Moreover, it reported the minimum time delay needed for an abrupt impairment of users\u2019 performance. The experimental results have shown that electrotactile feedback performance is less prone to changes with longer delays. However, visual feedback drops faster than electrotactile with increased time delays. This is a good indication for the effectiveness of electrotactile feedback in enabling close- loop control in prosthetics, since some delays are inevitable. The second study describes the development of a novel non-invasive compact multichannel interface for electrotactile feedback, containing 24 pads electrode matrix, with fully programmable stimulation unit, that investigates the ability of able-bodied human subjects to localize the electrotactile stimulus delivered through the electrode matrix. Furthermore, it designed a novel dual parameter -modulation (interleaved frequency and intensity) and compared it to conventional stimulation (same frequency for all pads). In addition and for the first time, it compared the electrotactile stimulation to mechanical stimulation. More, it exposes the integration of virtual prosthesis with the developed system in order to achieve better user experience and object manipulation through mapping the acquired real-time collected tactile data and feedback it simultaneously to the user. The experimental results demonstrated that the proposed interleaved coding substantially improved the spatial localization compared to same-frequency stimulation. Furthermore, it showed that same-frequency stimulation was equivalent to mechanical stimulation, whereas the performance with dual-parameter modulation was significantly better. The third study presents the realization of a novel, flexible, screen- printed e-skin based on P(VDF-TrFE) piezoelectric polymers, that would cover the fingertips and the palm of the prosthetic hand (particularly the Michelangelo hand by Ottobock) and an assistive sensorized glove for stroke patients. Moreover, it developed a new validation methodology to examine the sensors behavior while being solicited. The characterization results showed compatibility between the expected (modeled) behavior of the electrical response of each sensor to measured mechanical (normal) force at the skin surface, which in turn proved the combination of both fabrication and assembly processes was successful. This paves the way to define a practical, simplified and reproducible characterization protocol for e-skin patches In conclusion, by adopting innovative methodologies in sensing and stimulation systems, this thesis advances the overall development of close-loop sensory feedback human-machine interface used for restoration of sense of touch in prosthetics. Moreover, this research could lead to high-bandwidth high-fidelity transmission of tactile information for modern dexterous prostheses that could ameliorate the end user experience and facilitate it acceptance in the daily life

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human–machine interfacing

Tactile feedback is relevant in a broad range of human–machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject’s hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy (i.e. median{IQR} of 88.6{11}% for static and 93.3{5}% for dynamic patterns). The proposed system is an important step towards the development of a high-density human–machine interfacing between the user and a robotic han

Piezo-Tribo Dual Effect Hybrid Nanogenerators for Health Monitoring

Over the years, nanogenerators for health monitoring have become more and more attractive as they provide a cost-effective and continuous way to successfully measure vital signs, physiological status, and environmental changes in/around a person. Using such sensors can positively affect the way healthcare workers diagnose and prevent life-threatening conditions. Recently, the dual piezo-tribological effect of hybrid nanogenerators (HBNGs) have become a subject of investigation, as they can provide a substantial amount of data, which is significant for healthcare. However, real-life exploitation of these HBNGs in health monitoring is still marginal. This review covers piezo-tribo dual-effect HBNGs that are used as sensors to measure the different movements and changes in the human body such as blood circulation, respiration, and muscle contractions. Piezo-Tribo dual-effect HBNGs are applicable within various healthcare settings as a means of powering noninvasive sensors, providing the capability of constant patient monitoring without interfering with the range of motion or comfort of the user. This review also intends to suggest future improvements in HBNGs. These include incorporating surface modification techniques, utilizing nanowires, nanoparticle technologies, and other means of chemical surface modifications. These improvements can contribute significantly in terms of the electrical output of the HBNGs and can enhance their prospects of applications in the field of health monitoring, as well as various in vitro/in vivo biomedical applications. While a promising option, improved HBNGs are still lacking. This review also discusses the technical issue which has prevented so far, the real use of these sensors

Tactile sensing using elastomeric sensors

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 99-111).GelSight, namely, elastomeric sensor, is a novel tactile sensor to get the 3D information of contacting surfaces. Using GelSight, some tactile properties, such as softness and roughness, could be gained through image processing techniques. In this thesis, I implemented GelSight principle to reconstruct surface geometry of tested surfaces, based on which, the roughness comparison and lump detection experiment are conducted. Roughness of five different types of sandpapers are successfully compared using GelSight Ra value. In the lump detection experiment, a visual display for tactile information is presented. To get binary feedback of lump presence or not, a simple threshold method is introduced in this thesis. To evaluate the performance of GelSight sensor, human psychological experiments are conducted. In similar tasks, GelSight sensor outperforms humans in lump detection.by Xiaodan (Stella) Jia.S.M

Tactile Arrays for Virtual Textures

This thesis describes the development of three new tactile stimulators for active touch, i.e. devices to deliver virtual touch stimuli to the fingertip in response to exploratory movements by the user. All three stimulators are designed to provide spatiotemporal patterns of mechanical input to the skin via an array of contactors, each under individual computer control. Drive mechanisms are based on piezoelectric bimorphs in a cantilever geometry. The first of these is a 25-contactor array (5 × 5 contactors at 2 mm spacing). It is a rugged design with a compact drive system and is capable of producing strong stimuli when running from low voltage supplies. Combined with a PC mouse, it can be used for active exploration tasks. Pilot studies were performed which demonstrated that subjects could successfully use the device for discrimination of line orientation, simple shape identification and line following tasks. A 24-contactor stimulator (6 × 4 contactors at 2 mm spacing) with improved bandwidth was then developed. This features control electronics designed to transmit arbitrary waveforms to each channel (generated on-the-fly, in real time) and software for rapid development of experiments. It is built around a graphics tablet, giving high precision position capability over a large 2D workspace. Experiments using two-component stimuli (components at 40 Hz and 320 Hz) indicate that spectral balance within active stimuli is discriminable independent of overall intensity, and that the spatial variation (texture) within the target is easier to detect at 320 Hz that at 40 Hz. The third system developed (again 6 × 4 contactors at 2 mm spacing) was a lightweight modular stimulator developed for fingertip and thumb grasping tasks; furthermore it was integrated with force-feedback on each digit and a complex graphical display, forming a multi-modal Virtual Reality device for the display of virtual textiles. It is capable of broadband stimulation with real-time generated outputs derived from a physical model of the fabric surface. In an evaluation study, virtual textiles generated from physical measurements of real textiles were ranked in categories reflecting key mechanical and textural properties. The results were compared with a similar study performed on the real fabrics from which the virtual textiles had been derived. There was good agreement between the ratings of the virtual textiles and the real textiles, indicating that the virtual textiles are a good representation of the real textiles and that the system is delivering appropriate cues to the user

Flexible tactile digital feedback for clinical applications

Trauma and damage to the delicate structures of the inner ear frequently occurs during insertion of electrode array into the cochlea. This is strongly related to the excessive manual insertion force of the surgeon without any tool/tissue interaction feedback. The research is examined tool-tissue interaction of large prototype scale (12.5:1) digit embedded with distributive tactile sensor based upon cochlear electrode and large prototype scale (4.5:1) cochlea phantom for simulating the human cochlear which could lead to small scale digit requirements. This flexible digit classified the tactile information from the digit-phantom interaction such as contact status, tip penetration, obstacles, relative shape and location, contact orientation and multiple contacts. The digit, distributive tactile sensors embedded with silicon-substrate is inserted into the cochlea phantom to measure any digit/phantom interaction and position of the digit in order to minimize tissue and trauma damage during the electrode cochlear insertion. The digit is pre-curved in cochlea shape so that the digit better conforms to the shape of the scala tympani to lightly hug the modiolar wall of a scala. The digit have provided information on the characteristics of touch, digit-phantom interaction during the digit insertion. The tests demonstrated that even devices of such a relative simple design with low cost have potential to improve cochlear implants surgery and other lumen mapping applications by providing tactile feedback information by controlling the insertion through sensing and control of the tip of the implant during the insertion. In that approach, the surgeon could minimize the tissue damage and potential damage to the delicate structures within the cochlear caused by current manual electrode insertion of the cochlear implantation. This approach also can be applied diagnosis and path navigation procedures. The digit is a large scale stage and could be miniaturized in future to include more realistic surgical procedures