From pixels to percepts: Highly robust edge perception and contour following using deep learning and an optical biomimetic tactile sensor

Deep learning has the potential to have the impact on robot touch that it has had on robot vision. Optical tactile sensors act as a bridge between the subjects by allowing techniques from vision to be applied to touch. In this paper, we apply deep learning to an optical biomimetic tactile sensor, the TacTip, which images an array of papillae (pins) inside its sensing surface analogous to structures within human skin. Our main result is that the application of a deep CNN can give reliable edge perception and thus a robust policy for planning contact points to move around object contours. Robustness is demonstrated over several irregular and compliant objects with both tapping and continuous sliding, using a model trained only by tapping onto a disk. These results relied on using techniques to encourage generalization to tasks beyond which the model was trained. We expect this is a generic problem in practical applications of tactile sensing that deep learning will solve. A video demonstrating the approach can be found at https://www.youtube.com/watch?v=QHrGsG9AHtsComment: Accepted in RAL and ICRA 2019. N. Lepora and J. Lloyd contributed equally to this wor

Principal Components of Touch

EPSRC Centre for Doctoral Training in Future Autonomous and Robotic Systems (FARSCOPE) at Bristol Robotics Laboratory and Leadership Award from the Leverhulme Trust on `A biomimetic forebrain for robot touch' (RL-2016-39). The data was obtained using 4 different sensors: TacTip-v2, TacTip-v1, iCub fingertip and Tactile Whiskers (BIOTACT Vibrissae). The last three sensors were used to collect data whilst tapping (or whisking) cylinders of different diameters at different lateral displacements (this data was used for [1]). The TacTip-v2 sensor was used to collect data when tapping over a circular stimulus at different sensor radial displacements and orientations. REFERENCES: [1] N. F. Lepora, “Biomimetic active touch with fingertips and whiskers,” IEEE Transactions on Haptics, vol. 9, no. 2, pp. 170–183, April 2016