Active Tactile Sensing for Texture Perception in Robotic Systems

This thesis presents a comprehensive study of tactile sensing, particularly on the prob- lem of active texture perception. It includes a brief introduction to tactile sensing technology and the neural basis for tactile perception. It follows the literature review of textural percep- tion with tactile sensing. I propose a decoding and perception pipeline to tackle fine-texture classification/identification problems via active touching. Experiments are conducted using a 7DOF robotic arm with a finger-shaped tactile sensor mounted on the end-effector to per- form sliding/rubbing movements on multiple fabrics. Low-dimensional frequency features are extracted from the raw signals to form a perceptive feature space, where tactile signals are mapped and segregated into fabric classes. Fabric classes can be parameterized and sim- plified in the feature space using elliptical equations. Results from experiments of varied control parameters are compared and visualized to show that different exploratory move- ments have an apparent impact on the perceived tactile information. It implies the possibil- ity of optimising the robotic movements to improve the textural classification/identification performance

Self-Supervised Visuo-Tactile Pretraining to Locate and Follow Garment Features

Humans make extensive use of vision and touch as complementary senses, with vision providing global information about the scene and touch measuring local information during manipulation without suffering from occlusions. While prior work demonstrates the efficacy of tactile sensing for precise manipulation of deformables, they typically rely on supervised, human-labeled datasets. We propose Self-Supervised Visuo-Tactile Pretraining (SSVTP), a framework for learning multi-task visuo-tactile representations in a self-supervised manner through cross-modal supervision. We design a mechanism that enables a robot to autonomously collect precisely spatially-aligned visual and tactile image pairs, then train visual and tactile encoders to embed these pairs into a shared latent space using cross-modal contrastive loss. We apply this latent space to downstream perception and control of deformable garments on flat surfaces, and evaluate the flexibility of the learned representations without fine-tuning on 5 tasks: feature classification, contact localization, anomaly detection, feature search from a visual query (e.g., garment feature localization under occlusion), and edge following along cloth edges. The pretrained representations achieve a 73-100% success rate on these 5 tasks.Comment: RSS 2023, site: https://sites.google.com/berkeley.edu/ssvt

Digital fabrication of custom interactive objects with rich materials

As ubiquitous computing is becoming reality, people interact with an increasing number of computer interfaces embedded in physical objects. Today, interaction with those objects largely relies on integrated touchscreens. In contrast, humans are capable of rich interaction with physical objects and their materials through sensory feedback and dexterous manipulation skills. However, developing physical user interfaces that offer versatile interaction and leverage these capabilities is challenging. It requires novel technologies for prototyping interfaces with custom interactivity that support rich materials of everyday objects. Moreover, such technologies need to be accessible to empower a wide audience of researchers, makers, and users. This thesis investigates digital fabrication as a key technology to address these challenges. It contributes four novel design and fabrication approaches for interactive objects with rich materials. The contributions enable easy, accessible, and versatile design and fabrication of interactive objects with custom stretchability, input and output on complex geometries and diverse materials, tactile output on 3D-object geometries, and capabilities of changing their shape and material properties. Together, the contributions of this thesis advance the fields of digital fabrication, rapid prototyping, and ubiquitous computing towards the bigger goal of exploring interactive objects with rich materials as a new generation of physical interfaces.Computer werden zunehmend in Geräten integriert, mit welchen Menschen im Alltag interagieren. Heutzutage basiert diese Interaktion weitgehend auf Touchscreens. Im Kontrast dazu steht die reichhaltige Interaktion mit physischen Objekten und Materialien durch sensorisches Feedback und geschickte Manipulation. Interfaces zu entwerfen, die diese Fähigkeiten nutzen, ist allerdings problematisch. Hierfür sind Technologien zum Prototyping neuer Interfaces mit benutzerdefinierter Interaktivität und Kompatibilität mit vielfältigen Materialien erforderlich. Zudem sollten solche Technologien zugänglich sein, um ein breites Publikum zu erreichen. Diese Dissertation erforscht die digitale Fabrikation als Schlüsseltechnologie, um diese Probleme zu adressieren. Sie trägt vier neue Design- und Fabrikationsansätze für das Prototyping interaktiver Objekte mit reichhaltigen Materialien bei. Diese ermöglichen einfaches, zugängliches und vielseitiges Design und Fabrikation von interaktiven Objekten mit individueller Dehnbarkeit, Ein- und Ausgabe auf komplexen Geometrien und vielfältigen Materialien, taktiler Ausgabe auf 3D-Objektgeometrien und der Fähigkeit ihre Form und Materialeigenschaften zu ändern. Insgesamt trägt diese Dissertation zum Fortschritt der Bereiche der digitalen Fabrikation, des Rapid Prototyping und des Ubiquitous Computing in Richtung des größeren Ziels, der Exploration interaktiver Objekte mit reichhaltigen Materialien als eine neue Generation von physischen Interfaces, bei

Artificial Tactile System and Signal Processing for Haptic applications

Human have the ability to interact with the external environment through five main senses which are vision, hearing, smell, taste and touch. Most of all, the sensation like vision or hearing have been well developed and the use of various applications like TV, Camera, or artificial cochlear have been widely generalized. As the next steps, recently, the tactile sensor to mimic the tactile system of human have been attracted by many groups. Especially, after the development of Apple’s iPhone, the public interest about touch sensing applications have been increased explosively. Other researches for tactile sensing have focused on enhancing the performance of tactile sensor like the sensitivity, stability, response time and so on. As a result, there are some researches that the sensor performance of certain criteria is better than that of human tactile system. However, a human tactile system is not only very sensitive but also complex. In other words, ultimately, the tactile system mimicking the human tactile sensation should detect various parameters such as the pressure, temperature, hardness or roughness and also decide the psychological feeling like the pain by a hot material in touching or the smooth/roughness feeling in sliding the certain material. Therefore, in this thesis, it has been studied for the development of multifunctional tactile sensing system detecting various tactile parameters and deciding the kinds of psychological tactile feeling by measured stimulation. As the first step for the development of tactile system, we have studied the tactile sensor using ZnO nanowire. Therefore, in this chapter, the basic characteristics of ZnO nanowire are investigated to confirm the possibility for the tactile sensor. In addition, structural design factors of sensor units have been studied in order to enhance the sensitivity of ZnO nanowire-based tactile sensor. We have primarily demonstrated the effect of a square pattern array design in a pressure sensor using ZnO nanowires. Nanowires grown on the edge of cells can be bent easily because of growth direction, density of nanowires, and buckling effect. Since smaller square pattern arrays induce a higher circumference to cell area ratio, if one sensor unit consists of many micro-level square pattern arrays, the design enhances the piezoelectric efficiency and the sensitivity. As a result, 20um × 20um cell arrays showed three times higher pressure sensitivity than 250um × 250um cell array structures at a pressure range from 4kPa to 14kPa. The induced piezoelectric voltage with the same pressure level also increased drastically. Therefore, the smaller pattern array design is more appropriate for a high-sensitive pressure sensor than a simple one-body cell design for tactile systems, and it has the advantage of better power efficiency, which is also important for artificial tactile systems. Even if, in previous experiments, the possibility of piezoelectric materials as the tactile sensor and the method for the enhancement of pressure sensitivity are confirmed well, the tactile sensor for mimicking the human tactile sensation should measure various parameters as well as the pressure. However, many studies about ‘smooth-rough’ sensation depend on the machine learning technology with simple tactile sensors rather than developing the sensors that can measure various parameters like surface topography, hardness, quality of materials at the same time. Therefore, after the development of the pressure sensor, specific structures based on PDMS are proposed to measure and analyze above-mentioned parameters related to ‘smooth-rough’ decision, as like fingerprint of human. To find the optimized structure, three kinds of the structure shape (cone, cylinder and dome) are fabricated and the pressure sensitivity according to the shape are also measured. FEM simulation is also carried out to support the experimental result. Our tactile sensor with optimized dome structure (500um height) provides high shear force sensitivity, fast response time, stability, and durability. The high sensitivity about the shear force enables better the tactile sensor to measure the various surface information such as the pitch of pattern, the depth, the sliding velocity, the hardness and so on. In addition, after the study to measure the various surface information by dome structure, the research to measure the other surface information is also followed. In our previous study, we confirmed that the surface topography can be reconstructed by mapping the piezoelectric signals according to the location. In this research, to reduce the number of measurements from dozens to once and minimize the data loss at the empty space between adjacent sensors, the electrode array of Zig-Zag type is applied to the tactile sensor. As a result, with just one measurement, the surface topography of broad region can be successfully reconstructed by our tactile sensor as the high-resolution image. Additionally, the temperature sensor based on the resistive mechanism is fabricated between the Zig-Zag electrode lines to measures the temperature of surface materials when the tactile sensor rubs on the materials in real time. Over the development of the tactile sensing applications, the demand for an artificial system like human tactile sensation have been much more increased. Therefore, in this study, as a surrogate for human tactile sensation, we propose an artificial tactile sensing system based on the developed sensors in previous sections. For this, the piezoelectric tactile signal generated by touching and rubbing the material is transferred to DAQ system connected with our tactile sensor. First, the system decides whether the contacted material is dangerous or not. If dangerous like sharp or hot materials, the warning signal is generated by our artificial tactile system. If not, the sensor connected with the system rubs the materials and detects the roughness of the materials. Especially, the human test data related to ‘soft-rough’ detection is applied to a deep learning structure allowing personalization of the system, because tactile responses vary among humans. This approach could be applied to electronic devices with tactile emotional exchange capabilities, as well as various advanced digital experiences. In this thesis, human-like tactile sensing system based on the piezoelectric effect is successfully confirmed through various experiments. Although there are still some issues that need to be improved, this research is expected to be fundamental results for human-like tactile sensing system detecting a variety of the parameters such as the pressure, temperature, surface morphology, hardness, roughness and so on. In the future, through collaborative research with other fields like brain science, signal processing, we hope that this research can mimic psychological tactile sensations and communicate emotional exchange with external environment like real human skin.YList of Contents Abstract i List of contents iii List of tables vi List of figures vii Ⅰ. INTRODUCTION 1 1.1 Motivation 1 1.2 Various transduction mechanisms for the tactile sensor 5 1.2.1 Capacitive mechanism 5 1.2.2 Resistive mechanism 6 1.2.3 Triboelectric effect 7 1.2.4 Piezoelectric effect 9 1.3 Objectives 12 1.4 Reference 13 II. BASIC CHARACTERISTICS AND THE METHOD FOR ENHANC-ING THE PRESSURE SENSITIVITY OF THE TACTILE SENSOR BASED ON ZnO NANOWIRE 19 2.1 Introduction 19 2.2 Basic characteristics of ZnO nanowire 22 2.3 Device Fabrication 31 2.4 Morphological and Electrical characteristics 33 2.5 Pattern structure for enhanced for pressure sensitivity 38 2.6 Simulation result of piezoelectric effect for pattern structure 42 2.7 Reference 46 III. DOME STRUCTURE TO MEAUSRE THE SURFACE INFOR-MATION 52 3.1 Introduction 52 3.2 Basic characteristics of P(VDF-TrFE) 53 3.3 Device fabrication 61 3.4 Interaction mechanism between dome structure and surface material 63 3.5 Simulation and Experimental result comparing cone, cylinder, and dome structure 64 3.6 Simulation and Experimental result of the sensitivity enhancement ef-fect by dome structure 66 3.7 Depth measurement by tactile sensor with dome structure 72 3.8 Pattern of pitch by multi-array tactile sensor with dome structure 77 3.9 Hardness measurement by the tactile sensor with dome structure 79 3.10 Reference 83 IV. ZIG-ZAG ARRAYED TACTILE SENSOR BASED ON PIEZOE-LECTRIC-RESISTIVE MECHANISM TO DETECT THE SURFACE TOPOG-RAPHY AND TEMPERATURE 87 4.1 Introduction 87 4.2 Device fabrication 88 4.3 Piezoelectric characteristics of fabricated tactile sensor 90 4.4 Surface rendering method by the piezoelectric effect 95 4.5 Surface rendering result of 3D printed materials 96 4.6 Temperature sensing in sliding the high temperature material on Zig-Zag tactile sensor 99 4.7 Reference 103 V. TACTILE SENSING SYSTEM FOR PAIN AND SMOOTH/ROUGH DETECTION 105 5.1 Introduction 105 5.2 Components of the tactile sensing system 107 5.3 Artificial tactile sensing system for generating the pain warning 108 5.4 Artificial tactile sensing system for smooth/rough sensing 112 5.5 Reference 117 VⅠ. CONCLUSION 120DoctordCollectio

Distinguishing sliding from slipping during object pushing

Meier M, Walck G, Haschke R, Ritter H. Distinguishing sliding from slipping during object pushing. In: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE; 2016

Hierarchical tactile sensation integration from prosthetic fingertips enables multi-texture surface recognition\u3csup\u3e†\u3c/sup\u3e

Multifunctional flexible tactile sensors could be useful to improve the control of prosthetic hands. To that end, highly stretchable liquid metal tactile sensors (LMS) were designed, manufactured via photolithography, and incorporated into the fingertips of a prosthetic hand. Three novel contributions were made with the LMS. First, individual fingertips were used to distinguish between different speeds of sliding contact with different surfaces. Second, differences in surface textures were reliably detected during sliding contact. Third, the capacity for hierarchical tactile sensor integration was demonstrated by using four LMS signals simultaneously to distinguish between ten complex multi-textured surfaces. Four different machine learning algorithms were compared for their successful classification capabilities: K-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), and neural network (NN). The time-frequency features of the LMSs were extracted to train and test the machine learning algorithms. The NN generally performed the best at the speed and texture detection with a single finger and had a 99.2 ± 0.8% accuracy to distinguish between ten different multi-textured surfaces using four LMSs from four fingers simultaneously. The capability for hierarchical multi-finger tactile sensation integration could be useful to provide a higher level of intelligence for artificial hands

Study on measurement and evaluation methods on the sense of touch of thick clothing products

信州大学博士（工学）・学位論文・平成24年3月20日授与（甲第555号)・KWON EUICHULThesisKWON EUICHUL. Study on measurement and evaluation methods on the sense of touch of thick clothing products. 信州大学, 2012, 122p, 博士論文doctoral thesi

Microfabricated tactile sensors for biomedical applications: a review

During the last decades, tactile sensors based on different sensing principles have been developed due to the growing interest in robotics and, mainly, in medical applications. Several technological solutions have been employed to design tactile sensors; in particular, solutions based on microfabrication present several attractive features. Microfabrication technologies allow for developing miniaturized sensors with good performance in terms of metrological properties (e.g., accuracy, sensitivity, low power consumption, and frequency response). Small size and good metrological properties heighten the potential role of tactile sensors in medicine, making them especially attractive to be integrated in smart interfaces and microsurgical tools. This paper provides an overview of microfabricated tactile sensors, focusing on the mean principles of sensing, i.e., piezoresistive, piezoelectric and capacitive sensors. These sensors are employed for measuring contact properties, in particular force and pressure, in three main medical fields, i.e., prosthetics and artificial skin, minimal access surgery and smart interfaces for biomechanical analysis. The working principles and the metrological properties of the most promising tactile, microfabricated sensors are analyzed, together with their application in medicine. Finally, the new emerging technologies in these fields are briefly described