From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor

Deep learning has the potential to have the impact on robot touch that it has had on robot vision. Optical tactile sensors act as a bridge between the subjects by allowing techniques from vision to be applied to touch. In this paper, we apply deep learning to an optical biomimetic tactile sensor, the TacTip, which images an array of papillae (pins) inside its sensing surface analogous to structures within human skin. Our main result is that the application of a deep CNN can give reliable edge perception and thus a robust policy for planning contact points to move around object contours. Robustness is demonstrated over several irregular and compliant objects with both tapping and continuous sliding, using a model trained only by tapping onto a disk. These results relied on using techniques to encourage generalization to tasks beyond which the model was trained. We expect this is a generic problem in practical applications of tactile sensing that deep learning will solve. A video demonstrating the approach can be found at https://www.youtube.com/watch?v=QHrGsG9AHtsComment: Accepted in RAL and ICRA 2019. N. Lepora and J. Lloyd contributed equally to this wor

Voronoi Features for Tactile Sensing: Direct Inference of Pressure, Shear, and Contact Locations

There are a wide range of features that tactile contact provides, each with different aspects of information that can be used for object grasping, manipulation, and perception. In this paper inference of some key tactile features, tip displacement, contact location, shear direction and magnitude, is demonstrated by introducing a novel method of transducing a third dimension to the sensor data via Voronoi tessellation. The inferred features are displayed throughout the work in a new visualisation mode derived from the Voronoi tessellation; these visualisations create easier interpretation of data from an optical tactile sensor that measures local shear from displacement of internal pins (the TacTip). The output values of tip displacement and shear magnitude are calibrated to appropriate mechanical units and validate the direction of shear inferred from the sensor. We show that these methods can infer the direction of shear to $\sim$2.3$^{\circ}$ without the need for training a classifier or regressor. The approach demonstrated here will increase the versatility and generality of the sensors and thus allow sensor to be used in more unstructured and unknown environments, as well as improve the use of these tactile sensors in more complex systems such as robot hands.Comment: Presented at ICRA 201

Piezoelectric Transducers Based on Aluminum Nitride and Polyimide for Tactile Applications

The development of micro systems with smart sensing capabilities is paving the way to progresses in the technology for humanoid robotics. The importance of sensory feedback has been recognized the enabler of a high degree of autonomy for robotic systems. In tactile applications, it can be exploited not only to avoid objects slipping during their manipulation but also to allow safe interaction with humans and unknown objects and environments. In order to ensure the minimal deformation of an object during subtle manipulation tasks, information not only on contact forces between the object and fingers but also on contact geometry and contact friction characteristics has to be provided. Touch, unlike other senses, is a critical component that plays a fundamental role in dexterous manipulation capabilities and in the evaluation of objects properties such as type of material, shape, texture, stiffness, which is not easily possible by vision alone. Understanding of unstructured environments is made possible by touch through the determination of stress distribution in the surrounding area of physical contact. To this aim, tactile sensing and pressure detection systems should be integrated as an artificial tactile system. As illustrated in the Chapter I, the role of external stimuli detection in humans is provided by a great number of sensorial receptors: they are specialized endings whose structure and location in the skin determine their specific signal transmission characteristics. Especially, mechanoreceptors are specialized in the conversion of the mechanical deformations caused by force, vibration or slip on skin into electrical nerve impulses which are processed and encoded by the central nervous system. Highly miniaturized systems based on MEMS technology seem to imitate properly the large number of fast responsive mechanoreceptors present in human skin. Moreover, an artificial electronic skin should be lightweight, flexible, soft and wearable and it should be fabricated with compliant materials. In this respect a big challenge of bio-inspired technologies is the efficient application of flexible active materials to convert the mechanical pressure or stress into a usable electric signal (voltage or current). In the emerging field of soft active materials, able of large deformation, piezoelectrics have been recognized as a really promising and attractive material in both sensing and actuation applications. As outlined in Chapter II, there is a wide choice of materials and material forms (ceramics: PZT; polycrystalline films: ZnO, AlN; polymers and copolymers: PVDF, PVDF-TrFe) which are actively piezoelectric and exhibit features more or less attractive. Among them, aluminum nitride is a promising piezoelectric material for flexible technology. It has moderate piezoelectric coefficient, when available in c-axis oriented polycrystalline columnar structure, but, at same time, it exhibits low dielectric constant, high temperature stability, large band gap, large electrical resistivity, high breakdown voltage and low dielectric loss which make it suitable for transducers and high thermal conductivity which implies low thermal drifts. The high chemical stability allows AlN to be used in humid environments. Moreover, all the above properties and its deposition method make AlN compatible with CMOS technology. Exploiting the features of the AlN, three-dimensional AlN dome-shaped cells, embedded between two metal electrodes, are proposed in this thesis. They are fabricated on general purpose Kapton™ substrate, exploiting the flexibility of the polymer and the electrical stability of the semiconductor at the same time. As matter of fact, the crystalline layers release a compressive stress over the polymer, generating three-dimensional structures with reduced stiffness, compared to the semiconductor materials. In Chapter III, a mathematical model to calculate the residual stresses which arise because of mismatch in coefficient of thermal expansion between layers and because of mismatch in lattice constants between the substrate and the epitaxially grown films is adopted. The theoretical equation is then used to evaluate the dependence of geometrical features of the fabricated three-dimensional structures on compressive residual stress. Moreover, FEM simulations and theoretical models analysis are developed in order to qualitative explore the operation principle of curved membranes, which are labelled dome-shaped diaphragm transducers (DSDT), both as sensors and as piezo-actuators and for the related design optimization. For the reliability of the proposed device as a force/pressure sensor and piezo-actuator, an exhaustive electromechanical characterization of the devices is carried out. A complete description of the microfabrication processes is also provided. As shown in Chapter IV, standard microfabrication techniques are employed to fabricate the array of DSDTs. The overall microfabrication process involves deposition of metal and piezoelectric films, photolithography and plasma-based dry and wet etching to pattern thin films with the desired features. The DSDT devices are designed and developed according to FEM and theoretical analysis and following the typical requirements of force/pressure systems for tactile applications. Experimental analyses are also accomplished to extract the relationship between the compressive residual stress due to the aluminum nitride and the geometries of the devices. They reveal different deformations, proving the dependence of the geometrical features of the three-dimensional structures on residual stress. Moreover, electrical characterization is performed to determine capacitance and impedance of the DSDTs and to experimentally calculate the relative dielectric constant of sputtered AlN piezoelectric film. In order to investigate the mechanical behaviour of the curved circular transducers, a characterization of the flexural deflection modes of the DSDT membranes is carried out. The natural frequency of vibrations and the corresponding displacements are measured by a Laser Doppler Vibrometer when a suitable oscillating voltage, with known amplitude, is applied to drive the piezo-DSDTs. Finally, being developed for tactile sensing purpose, the proposed technology is tested in order to explore the electromechanical response of the device when impulsive dynamic and/or long static forces are applied. The study on the impulsive dynamic and long static stimuli detection is then performed by using an ad hoc setup measuring both the applied loading forces and the corresponding generated voltage and capacitance variation. These measurements allow a thorough test of the sensing abilities of the AlN-based DSDT cells. Finally, as stated in Chapter V, the proposed technology exhibits an improved electromechanical coupling with higher mechanical deformation per unit energy compared with the conventional plate structures, when the devices are used as piezo-actuator. On the other hand, it is well suited to realize large area tactile sensors for robotics applications, opening up new perspectives to the development of latest generation biomimetic sensors and allowing the design and the fabrication of miniaturized devices

The TacTip Family : Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies

The authors thank Sam Coupland, Gareth Griffiths, and Samuel Forbes for their help with 3D printing and Jason Welsby for his assistance with electronics. N.L. was supported, in part, by a Leverhulme Trust Research Leadership Award on “A biomimetic forebrain for robot touch” (RL-2016-039), and N.L. and M.E.G. were supported, in part, by an EPSRC grant on Tactile Super-resolution Sensing (EP/M02993X/1). L.C. was supported by the EPSRC Centre for Doctoral Training in Future Autonomous and Robotic Systems (FARSCOPE).Peer reviewedPublisher PD