Tactile Avatar: Tactile Sensing System Mimicking Human Tactile Cognition

As a surrogate for human tactile cognition, an artificial tactile perception and cognition system are proposed to produce smooth/soft and rough tactile sensations by its user's tactile feeling; and named this system as “tactile avatar”. A piezoelectric tactile sensor is developed to record dynamically various physical information such as pressure, temperature, hardness, sliding velocity, and surface topography. For artificial tactile cognition, the tactile feeling of humans to various tactile materials ranging from smooth/soft to rough are assessed and found variation among participants. Because tactile responses vary among humans, a deep learning structure is designed to allow personalization through training based on individualized histograms of human tactile cognition and recording physical tactile information. The decision error in each avatar system is less than 2% when 42 materials are used to measure the tactile data with 100 trials for each material under 1.2N of contact force with 4cm s−1 of sliding velocity. As a tactile avatar, the machine categorizes newly experienced materials based on the tactile knowledge obtained from training data. The tactile sensation showed a high correlation with the specific user's tendency. This approach can be applied to electronic devices with tactile emotional exchange capabilities, as well as advanced digital experiences. © 2021 The Authors. Advanced Science published by Wiley-VCH GmbH1

촉각 신호 처리 및 햅틱 인터페이스 시스템

Tactile sensor, Tactile actuator, Piezoelectricity, Electrical stimulation, Tactile signal processing, Haptic interface system요즈음 촉각에 대한 연구에 관심이 높아지면서 햅틱 인터페이스 시스템도 많이 발전되고 있다. 최근에 많은 사람들이 스마트폰, 터치 디스플레이 및 VR 장비를 통해 햅틱 기술을 사용하고 있으며, 다양한 유형의 장치들이 감도, 효율성, 신뢰성 및 유연성과 같은 성능 향상을 위해 개발되고 있다. 예를 들어 기존의 실리콘 기반 센서에서 벗어나 최근에는 웨어러블 하고 플랙서블한 햅틴 센서뿐만 아니라 사람의 피부를 모방한 E-skin 디바이스가 개발되는 중이며, 사람의 촉각 시스템보다 더 민감하고 고해상도 성능을 갖는 다양한 햅틱 장치 또한 다양하게 연구되고 있는 중이다. 햅틱 인터페이스 시스템은 일반적으로 다양한 자극을 감지하는 촉각 센서와 햅틱 피드백을 사용하여 인공 촉각 감각을 재현하는 촉각 액추에이터로 구성되어 있으며 센서와 액추에이터의 발전덕분에 최근에는 수술용 로봇 팔, 재활 기기 등에도 햅틱 시스템이 활용되고 있다. 그러나 이러한 기술의 발전에도 불구하고, 사람은 압력이나 온도와 같은 물리적 자극뿐만 아니라 거칠기, 딱딱함, 심지어는 고통과 같은 정신감각적 자극 느끼기 때문에, 인간의 촉각 시스템을 완벽하게 모방하는 데에는 많은 과제가 남아 있다. 게다가 사람마다 이러한 정신감각적인 느낌을 감지하는 기준이 다르기 때문에 햅틱 피드백을 통해 인공적으로 촉감을 재현하는 것 또한 많은 이슈가 있다. 따라서 기존의 햅틱 시스템이 가지는 한계를 뛰어넘어 물리적, 정신감각적 감정을 모두 다룰 수 있는 첨단 햅틱 인터페이스 시스템에 대한 연구가 필요하다. 본 논문은 압전 촉각 센서와 전기 자극 액추에이터를 이용한 첨단 햅틱 인터페이스 시스템의 개발에 중점을 두고 있다. 압전 효과는 빠른 응답속도와 높은 민감도를 가지기 때문에 정적뿐만 아니라 동적 모션을 감지하고 센싱하는데 매우 적합합니다. 또한 압전효과는 외부에서 가해진 물리적인 스트레스로부터 스스로 전기를 만들 수 있는 독특한 특징을 가지고 있어서, 시스템을 간략화 할 수 있으며 추가적인 외부 전원이 불 필요하다는 장점이 있다. 햅틱 피드백을 위한 전기자극 액추에이터는 주파수, 신호파형, 진폭 등 여러 변수를 컨트롤 할 수 있어서 사람에게 인공 촉감을 재현할 수 있는 효과적이고 효율적인 방법 중 하나이다. 또한 SA 및 FA 촉각 수용체의 메커니즘을 활용하면 다양한 인공 촉각을 재현할 수 있을 것으로 기대된다. 첨단 햅틱 인터페이스 시스템 개발을 위하여 먼저 표면 정보를 측정할 수 있는 압전 소재 기반의 유연 촉각 센서를 연구를 진행했다. 압전 유연 센서는 고 해상도와 높은 정확도를 위하여 어레이 구조로 디자인되었으며, 센서에서 발생되는 압전 신호를 분석하여 표면 정보를 추출하는 신호처리 기술을 연구하였다. 표면이 가지는 특정한 패턴, 모양 및 변위와 같은 측정 가능한 물리적 인자들을 효과적으로 측정하기 위하여 슬라이딩 모션을 사용했으며, 동적인 모션을 사용했기에 측정되는 압전신호들은 시간 도메인에서 분석되었다. 개발된 압전 센서는 높은 선형성과 우수한 신뢰성 및 안정적인 감도를 가졌으며, 200μmm 넓이와, 500μm의 미세한 패턴을 측정할 수 있음을 확인했다. 두 번째로, 기존의 압전 센서를 핀 형태의 모듈과 결합하여 깊이 정보를 측정할 수 있도록 개선하는 연구를 진행했다. 핀 형태의 모듈은 스프링 및 벌크한 구조를 가지지만, 3차원 정보 측정의 한계를 극복할 수 있는 센싱 메커니즘을 제안한다. 또한, 압전 신호로 부터 깊이 정보를 추출할 수 있는 신호 처리를 개발하여, 표면에 있는 다양한 모양, 간격, 접촉각도 등을 측정 후 신호 처리하여 표면 정보를 이미지로 렌더링할 수 있다. 이러한 결과를 기반으로 하여, 직물과 같은 부드러운 재료의 표면 정보 또한 3D 이미지화 할 수 있었다. 또한 슬라이딩 조건에서 압전 신호에 대한 근본적이고 기초적인 분석을 진행하였다. 기존의 압전 신호 분석은 주로 피크 출력 전압과 관련이 있기 때문에 동적 상황에 해당하는 시간 경과에 따라 압전 신호를 분석할 때 많은 이슈와 한계가 있었다. 이를 극복하기 위하여 작은 세그먼트와 유닛 전극에서 발생되는 압전 신호를 비교하여 압전 신호 발생 과정에 대한 단서를 얻고 압전신호를 수학적으로 분석하여 일반해를 구하였다. 압전 신호를 익스포넨셜 및 오차 함수로 표모델링 된 압전 신호를 기반으로 인공 압전 신호를 만들었고, 이러한 인공 신호와 실제 압전 신호를 비교하여 일반해를 최적화하여 인공 압전신호의 신뢰성과 재현성을 향상시켰다. 만들어진 인공 신호들을 조합에서 다양한 깊이 프로파일들을 측정하였다. 마지막으로 전기 자극 메커니즘을 이용하여 인공 촉각 재생을 위한 촉각 액츄에이터를 개발하였다. 제안된 액추에이터는 PCB 기반의 어레이 전극을 가지며 전극의 크기, 전극의 피치 및 접지 구조와 같은 구조상의 변수들을 고려하여 전기 자극을 최적화한다. 또한 Labview 프로그램으로 전기자극에 사용될 다양한 전기 신호를 개발하고 주파수, 신호 파형 및 진폭을 제어하여 커스텀 및 최적화를 진행하였다. 다양한 변수들을 조절한 결과를 바탕으로, 참가자들의 공간적 해상도 향상을 확인했으며, 이러한 방식을 최적화한다면 다양한 자극을 재현할 수 있을 것으로 기대된다. |The study of the human tactile sense is getting interested, along with the development of haptic technology. Haptic technology is one of the advanced interactions between humans and devices, so it has been developed for bilateral communication and surgical robot. Therefore, many types of sensor mechanisms and advanced materials have been studied to mimic the characteristics and responses of human tactile system. With the development of tactile sensors, tactile actuators have also been introduced to reproduce artificial tactile sensations. As a result of the development of tactile sensors and actuators, haptic interface systems with characteristics of sensor and actuator are studied to not only sense various tactile stimuli but also feedback realistic tactile sensations in field of VR/AR devices, and rehabilitation devices. However, despite advances in these technologies, there remain numerous challenges to perfectly mimicking the human tactile system because it is not easy to sense psychological sensations such as roughness, hardness, and texture. In addition, it is not easy to create artificial tactile feedback because people have different criteria for sensing these psychological feelings. Therefore, it is necessary to make multimodal tactile sensors that can sense various surface properties, and advanced tactile actuators which can create realistic tactile feedback for a haptic interface system. This thesis focuses on developing an advanced haptic interface system with a piezoelectric tactile sensor and electrical stimulation actuator. The piezoelectric effect offers rapid responses and high sensitivity, so that it is appropriate to detect static and dynamic motions. In addition, piezoelectricity can generate electrical signals in response to applied stress, taking advantage of the self-powered characteristics. A sliding motion of sensor was studied mainly here because various surface information such as pattern, shape, and displacement can be sensed rather than simple touch motion. Besides, a specific signal processing was suggested and applied to surface topography reproduction. Electrical stimulation was studied for tactile actuators because this method can make artificial tactile feelings without any touch motions by adjusting frequency, signal waveform, and amplitude. Therefore, it is expected to be utilized for realistic artificial tactile feedback. As the first approach for developing the advanced haptic interface system, a tactile array sensor with piezoelectric material was studied to measure surface topography. Array sensor structure and signal analysis skills were proposed to measure surface topography with high accuracy. The suggested array sensor showed high linearity (R^2=0.98), good reliability (250 times), and stable sensitivity (0.1V/N), as well as could measure a contact width and pitch information based on the measured piezoelectric voltage signals. Additionally, the piezoelectric sensor could calculate an angle of diagonal sliding motion. It was confirmed that the sensor could get x and y-axis information with high accuracy in any sliding direction. However, because of the limit of flexibility in senor structure which resulted in low resolution of the z-axis, 3D surface topography was not reproduced perfectly. Therefore, additional treatment is required for high sensing performance. In order to overcome the issue and to improve sensing performance in z-axis information, a piezoelectric sensor e with a pin-module was proposed. The sensor combining a pin-type module showed excellent monitoring in the depth direction by pin and spring motion, so that it could have an excellent resolution of the depth information. The depth information was calculated by integrated piezoelectric voltage signals because the integral values of the piezoelectric signal have a linear relationship with various depth conditions. The integrated piezoelectric signals were used for measuring contact width, spacing, and contact angle. Therefore, surface topography was reproduced with high accuracy. As one of real applications, the surface information of soft fabric sample was measured with high accuracy by the sensor. Additionally, we demonstrated a fundamental analysis of a piezoelectric signal in a sliding condition. Because a conventional analysis of piezoelectric signals is mainly related to the peak output voltage, it has limits of sensing resolution for time domain. Therefore, piezoelectric signals were analyzed in the time domain corresponding to dynamic situations. We compared the piezoelectric signals between small segments and a unit cell to obtain a clue about the signal process and confirm the mathematical formula. Based on the results of piezoelectric signal fitting with the exponential and error function, general solutions that can model piezoelectric signals are proposed to create artificial piezoelectric signals. By comparing the artificial and measured signals, the general solution is optimized, and the created artificial piezoelectric signals are found to have good reliability and reproducibility. Various depth profiles can be calculated from the combination of the artificial signals. In addition, the simultaneously measurable depth profiles are improved with more artificial signals. When tactile sensors can deform or bend well on a touching object, the signal process can estimate multiple depth profiles of the object. With the development of a tactile sensor, a tactile actuator for artificial tactile sensation was studied by using electrical stimulation mechanism. The proposed actuator had an array of electrodes for stimulation. Several design parameters such as the size of the electrode, the pitch of electrodes, and ground structure were considered to optimize electrical stimulation. Various voltage signals were developed and customized by controlling a frequency, a signal waveform, and an amplitude. For electrical signal optimization, signal waveforms (sine, square, triangle, and sawtooth) were compared, and effective frequency ranges were also analyzed. Then, spatial resolution was analyzed by varying stimulated areas with human perception test. Most of the participants had a higher resolution sensation in the case of a complex signal using various frequencies and waveforms with small area stimulation. Therefore, it was revealed that composite signals and stimulation areas were important major parameters to reproduce various tactile sensations based on electrical signals. This thesis introduced a tactile sensor based on the piezoelectric effect and a tactile actuator with electrical stimulation for a haptic interface system. The proposed piezoelectric sensor showed an excellent sensing performance with x and y-axis information. A structural improvement method with pin-module and fundamental signal analysis were proposed to analyze depth profiles. In addition, a tactile actuator system with electrical stimulation was introduced, and electrical signals were optimized by adjusting signal waveforms, frequencies, and stimulation areas. Although there are still challenges that need to be improved, the results of this thesis are expected to be a fundamental study of a multi-functional system that is able to deal with both physical and psychological sensations for an advanced haptic interface.YⅠ. INTRODUCTION 1 1.1 Haptic Interface System 1 1.2 Human Tactile System 3 1.2.1 Tactile Receptors 5 1.2.2 Tactile Signals 9 1.3 Basic Principle of Tactile Sensor 10 1.3.1 Piezoresistive Tactile Sensor 11 1.3.2 Capacitive Tactile Sensor 13 1.3.3 Triboelectric Tactile Sensor 14 1.3.4 Piezoelectric Tactile Sensor 15 1.4 Basic Principle of Tactile Actuator 16 1.4.1 Kinesthetic Feedback 19 1.4.2 Vibrotactile Feedback 20 1.4.3 Electrical Feedback 21 II. BACKGROUND OF PIEZOELECTRIC EFFECT AND ELECTRICAL STIMULATION 23 2.1 Background 23 2.2 Background of Piezoelectric Effect 25 2.3 Previous Works of Piezoelectric Tactile Sensor 27 2.4 Background of Electrical Stimulation 32 2.5 Previous Works of Electrical Stimulation 34 2.6 Motivation 35 III. THE SENSING PERFORMANCE AND CHARACTERISTICS OF PIEZOELECTRIC SENSOR WITH SLIDING MOTION 39 3.1 Introduction 39 3.2 Sensor Design and fabrication 40 3.3 Basic Performance and Characteristics 42 3.4 Analysis of Piezoelectric Signals with Sliding motion 45 3.5 Surface Topography Measurement 48 IV. IMPROVEMENT OF SENSING CAPABILITY AND 3D STRUCTURE ANALYSIS WITH PIN-MODULES 52 4.1 Introduction 52 4.2 Sensor Design and Fabrication 53 4.3 Analysis of Piezo-Signal for Depth Measurement 57 4.4 Complex Structure Rendering based on Signal Processing 62 4.5 3D-rendering Result of Fabric Samples 67 V. FUNDAMENTAL ANALYSIS OF THE ELECTRICAL SIGNALS OF PIEZOELECTRIC MATERIALS 71 5.1 Introduction 71 5.2 Concept and Simulation 73 5.3 Electrode Design and Fabrication 76 5.4 Signal Comparison between the Unit and Segment Electrodes 77 5.5 Signals processing and General Solution of Piezoelectric signal 83 5.6 Depth Profile Analysis with Artificial Piezoelectric Signals 88 VI. A STUDY OF TACTILE ACTUATOR AND TACTILE SIGNAL PROCESSING BASED ON ELECTRICAL STIMULATION 94 6.1 Introduction 94 6.2 Tactile Actuator System 95 6.3 Tactile Signal Processing with Voltage Mode 98 6.4 Resolution analysis of Electrical Tactile Sensation 101 VII. CONCLUSION 115 VIII. REFERENCES 118DoctordCollectio

Deep Neural Network Classification of Tactile Materials Explored by Tactile Sensor Array With Various Active-Cell Formations

Reducing the input data of tactile sensory systems brings a large degree of freedom to real-world implementations from the perspectives of bandwidth and computational complexity. For this, in this letter, we suggest efficient active-cell formations with a high classification accuracy of tactile materials. By revealing that averaged Kullback-Leibler-divergence and common frequency component power to variance ratio are proportional to the classification accuracy, we showed that those methods can be useful in estimating valid active-cell formations.1

Tactile Sensor Structure Optimized for Sliding Motion with High Resolution Recording of Surface Topography

The demand for tactile devices with human-like exquisiteness has recently been increasing in various fields. Among the various parameters that humans feel through the tactile system, temperature and surface topography are the most important parameters to achieve artificial tactile devices with human level precision. Here, we present a new tactile sensor with high resolution surface topography recording and temperature measurement. Our tactile sensor was designed to have a surface topography sensing part driven by P(VDF-TrFE) and a thermistor part for temperature sensing. Even though the sensor touched the same temperature object, the sliding condition showed different resistance change values. Therefore, the correlation factors of the sliding velocity to temperature were defined with a simple relation function. To achieve high resolution recording, ‘zig-zag’ arrayed tactile sensor can overcome the limitations of simple matrix cell design. The design eliminates empty spaces between sensor cells. By using these systems, a resolution of ~500μm to x-direction was achieved. This tactile sensor has linear sensitivity to pressure with a superior response time of 10ms for dynamic sensing. By integrating the dynamic piezoelectric signals during the sliding motion, we can reconstruct the surface topography of various objects. IEEEFALS

Fundamental insights into the electrical signals of a piezoelectric sensor in a sliding condition

The piezoelectric mechanism represents a promising approach for advanced sensors that measure physical factors such as pressures, strain levels and even temperature responses to various stimuli. However, a conventional analysis of piezoelectric signals is mainly related to the peak output voltage for normal force, meaning that there remain numerous challenges to overcome when analyzing piezoelectric signals over time corresponding to complex or dynamic situations. Here, a fundamental analysis of piezoelectric signals for a dynamic change situation induced by sliding motion and resulting in the partial deformation of a piezoelectric material is introduced. Given that a piezoelectric voltage at a certain time represents the sum value of diploe moments in all piezoelectric material segments, in this respect, some parts are compressed and others are released, and we compared the electric signals between small segments and one unit cell to obtain a clue about the signal process and to confirm the mathematical formula. Based on the results of piezoelectric signal fitting with the exponential and error function, general solutions that can model piezoelectric signals were proposed to create artificial piezoelectric signals which corresponded to each small segment of the piezoelectric material. By comparing the artificial signals and actually measured signals, the general solution was optimized and the induced artificial piezoelectric signals were found to have good reliability and reproducibility. Various depth profiles with sliding motion could be calculated from the combination of artificial signals in the 0.6 mm, 0.9 mm, and 1.2 mm pressing conditions using piezoelectric integral values. In addition, the calculated depth profiles had a resolution of approximately 100 µm, and simultaneously measurable depth profiles were improved with more artificial signals. © 2022 Elsevier LtdFALS

Artificial Tactile Sensor with Pin-type Module for Depth Profile and Surface Topography Detection

Haptic sensors based on piezoelectric sensor arrays with pin-type modules which have high responses and dynamic sensing capabilities were designed and studied for surface topography measurements. Unlike the human finger, most flexible tactile sensor designs do not detect well the depth information of surfaces which change at the ~mm level despite the fact that they have a good sensitivity for pressure or force. To enhance the ability to detect depth information of the surfaces of objects, a piezoelectric sensor combining a pin-type module with excellent monitoring in the depth direction by spring is developed here. Because spike types of piezoelectric signals do not match directly a specific surface topography, a signal processing method that reconstructs the surface topography was studied considering the piezoelectric working principle and spring dynamics. According to the sensor design, it can detect 3mm depth changes with a two-dimensional plane structure at mm-level resolutions. The results showed that that the proposed sensor could measure various shapes and depth profiles precisely via a sliding motion, and the surface topography was reconstructed through highly accurate measurement results, similar to a human.1

Detecting temperature of small object using hybrid tactile sensor array and multi-parameter extraction analysis

An artificial hybrid tactile sensor and a signal process which can detect precisely temperature and pressure were demonstrated. Although a resistive temperature sensor has been used widely due to the easy fabrication process for tactile sensors, it was hard to detect an exact temperature value and the sensor showed the slow response when an object smaller than the dimensions of the sensor structure touched the sensor. To measure the exact temperature for a small object, we conducted a signal process using two major factors, the change of resistance provided by the thermal sensor and the information on the contact dimensions acquired from piezoelectric multiarray sensors. The design of the temperature sensor was simplified by utilizing a single resistor placed on the top layer of the hybrid sensor structure to enhance the sensitivity of thermal detection. Furthermore, using the gradient of resistance change instead of a saturation value can provide more reliable data due to the minimization of thermal conductivity change among various contact situations on sensors and fast detection time. The hybrid sensor system provided area information by which the gradient values were modified, and then the actual temperature value was calculated using the two variables, slope and contact size. As a result, the hybrid sensor successfully classified temperature levels on objects up to 30 times smaller than the resistive sensor dimensions with a very fast response time of below 10 msec.FALS

Electronic Skin to Feel "Pain": Detecting "Prick" and "Hot" Pain Sensations

An artificial tactile system has attracted tremendous interest and intensive study, since it can be applied as a new functional interface between humans and electronic devices. Unfortunately, most previous works focused on improving the sensitivity of sensors. However, humans also respond to psychological feelings for sensations such as pain, softness, or roughness, which are important factors for interacting with others and objects. Here, we present an electronic skin concept that generates a "pain" warning signal, specifically, to sharp "prick" and "hot" sensations. To simplify the sensor structure for these two feelings, a single-body tactile sensor design is proposed. By exploiting "hot" feeling based on the Seebeck effect instead of the pyroelectric property, it is possible to distinguish points registering a "hot" feeling from those generating a "prick" feeling, which is based on the piezoelectric effect. The control of free carrier concentration in nanowire induced the appropriate level of Seebeck current, which enabled the sensor system to be more reliable. The first derivatives of the piezo and Seebeck output signals are the key factors for the signal processing of the "pain" feeling. The main idea can be applied to mimic other psychological tactile feelings. © Copyright 2019, Mary Ann Liebert, Inc.1