Improved GelSight Tactile Sensor for Measuring Geometry and Slip

A GelSight sensor uses an elastomeric slab covered with a reflective membrane to measure tactile signals. It measures the 3D geometry and contact force information with high spacial resolution, and successfully helped many challenging robot tasks. A previous sensor, based on a semi-specular membrane, produces high resolution but with limited geometry accuracy. In this paper, we describe a new design of GelSight for robot gripper, using a Lambertian membrane and new illumination system, which gives greatly improved geometric accuracy while retaining the compact size. We demonstrate its use in measuring surface normals and reconstructing height maps using photometric stereo. We also use it for the task of slip detection, using a combination of information about relative motions on the membrane surface and the shear distortions. Using a robotic arm and a set of 37 everyday objects with varied properties, we find that the sensor can detect translational and rotational slip in general cases, and can be used to improve the stability of the grasp.Comment: IEEE/RSJ International Conference on Intelligent Robots and System

Haptic Edge Detection Through Shear

Most tactile sensors are based on the assumption that touch depends on measuring pressure. However, the pressure distribution at the surface of a tactile sensor cannot be acquired directly and must be inferred from the deformation field induced by the touched object in the sensor medium. Currently, there is no consensus as to which components of strain are most informative for tactile sensing. Here, we propose that shape-related tactile information is more suitably recovered from shear strain than normal strain. Based on a contact mechanics analysis, we demonstrate that the elastic behavior of a haptic probe provides a robust edge detection mechanism when shear strain is sensed. We used a jamming-based robot gripper as a tactile sensor to empirically validate that shear strain processing gives accurate edge information that is invariant to changes in pressure, as predicted by the contact mechanics study. This result has implications for the design of effective tactile sensors as well as for the understanding of the early somatosensory processing in mammals

Design, evaluation, and control of nexus: a multiscale additive manufacturing platform with integrated 3D printing and robotic assembly.

Additive manufacturing (AM) technology is an emerging approach to creating three-dimensional (3D) objects and has seen numerous applications in medical implants, transportation, aerospace, energy, consumer products, etc. Compared with manufacturing by forming and machining, additive manufacturing techniques provide more rapid, economical, efficient, reliable, and complex manufacturing processes. However, additive manufacturing also has limitations on print strength and dimensional tolerance, while traditional additive manufacturing hardware platforms for 3D printing have limited flexibility. In particular, part geometry and materials are limited to most 3D printing hardware. In addition, for multiscale and complex products, samples must be printed, fabricated, and transferred among different additive manufacturing platforms in different locations, which leads to high cost, long process time, and low yield of products. This thesis investigates methods to design, evaluate, and control the NeXus, which is a novel custom robotic platform for multiscale additive manufacturing with integrated 3D printing and robotic assembly. NeXus can be used to prototype miniature devices and systems, such as wearable MEMS sensor fabrics, microrobots for wafer-scale microfactories, tactile robot skins, next generation energy storage (solar cells), nanostructure plasmonic devices, and biosensors. The NeXus has the flexibility to fixture, position, transport, and assemble components across a wide spectrum of length scales (Macro-Meso-Micro-Nano, 1m to 100nm) and provides unparalleled additive process capabilities such as 3D printing through both aerosol jetting and ultrasonic bonding and forming, thin-film photonic sintering, fiber loom weaving, and in-situ Micro-Electro-Mechanical System (MEMS) packaging and interconnect formation. The NeXus system has a footprint of around 4m x 3.5m x 2.4m (X-Y-Z) and includes two industrial robotic arms, precision positioners, multiple manipulation tools, and additive manufacturing processes and packaging capabilities. The design of the NeXus platform adopted the Lean Robotic Micromanufacturing (LRM) design principles and simulation tools to mitigate development risks. The NeXus has more than 50 degrees of freedom (DOF) from different instruments, precise evaluation of the custom robots and positioners is indispensable before employing them in complex and multiscale applications. The integration and control of multi-functional instruments is also a challenge in the NeXus system due to different communication protocols and compatibility. Thus, the NeXus system is controlled by National Instruments (NI) LabVIEW real-time operating system (RTOS) with NI PXI controller and a LabVIEW State Machine User Interface (SMUI) and was programmed considering the synchronization of various instruments and sequencing of additive manufacturing processes for different tasks. The operation sequences of each robot along with relevant tools must be organized in safe mode to avoid crashes and damage to tools during robots’ motions. This thesis also describes two demonstrators that are realized by the NeXus system in detail: skin tactile sensor arrays and electronic textiles. The fabrication process of the skin tactile sensor uses the automated manufacturing line in the NeXus with pattern design, precise calibration, synchronization of an Aerosol Jet printer, and a custom positioner. The fabrication process for electronic textiles is a combination of MEMS fabrication techniques in the cleanroom and the collaboration of multiple NeXus robots including two industrial robotic arms and a custom high-precision positioner for the deterministic alignment process

Proprioceptive Learning with Soft Polyhedral Networks

Proprioception is the "sixth sense" that detects limb postures with motor neurons. It requires a natural integration between the musculoskeletal systems and sensory receptors, which is challenging among modern robots that aim for lightweight, adaptive, and sensitive designs at a low cost. Here, we present the Soft Polyhedral Network with an embedded vision for physical interactions, capable of adaptive kinesthesia and viscoelastic proprioception by learning kinetic features. This design enables passive adaptations to omni-directional interactions, visually captured by a miniature high-speed motion tracking system embedded inside for proprioceptive learning. The results show that the soft network can infer real-time 6D forces and torques with accuracies of 0.25/0.24/0.35 N and 0.025/0.034/0.006 Nm in dynamic interactions. We also incorporate viscoelasticity in proprioception during static adaptation by adding a creep and relaxation modifier to refine the predicted results. The proposed soft network combines simplicity in design, omni-adaptation, and proprioceptive sensing with high accuracy, making it a versatile solution for robotics at a low cost with more than 1 million use cycles for tasks such as sensitive and competitive grasping, and touch-based geometry reconstruction. This study offers new insights into vision-based proprioception for soft robots in adaptive grasping, soft manipulation, and human-robot interaction.Comment: 20 pages, 10 figures, 2 tables, submitted to the International Journal of Robotics Research for revie

Objekt-Manipulation und Steuerung der Greifkraft durch Verwendung von Taktilen Sensoren

This dissertation describes a new type of tactile sensor and an improved version of the dynamic tactile sensing approach that can provide a regularly updated and accurate estimate of minimum applied forces for use in the control of gripper manipulation. The pre-slip sensing algorithm is proposed and implemented into two-finger robot gripper. An algorithm that can discriminate between types of contact surface and recognize objects at the contact stage is also proposed. A technique for recognizing objects using tactile sensor arrays, and a method based on the quadric surface parameter for classifying grasped objects is described. Tactile arrays can recognize surface types on contact, making it possible for a tactile system to recognize translation, rotation, and scaling of an object independently.Diese Dissertation beschreibt eine neue Art von taktilen Sensoren und einen verbesserten Ansatz zur dynamischen Erfassung von taktilen daten, der in regelmäßigen Zeitabständen eine genaue Bewertung der minimalen Greifkraft liefert, die zur Steuerung des Greifers nötig ist. Ein Berechnungsverfahren zur Voraussage des Schlupfs, das in einen Zwei-Finger-Greifarm eines Roboters eingebaut wurde, wird vorgestellt. Auch ein Algorithmus zur Unterscheidung von verschiedenen Oberflächenarten und zur Erkennung von Objektformen bei der Berührung wird vorgestellt. Ein Verfahren zur Objekterkennung mit Hilfe einer Matrix aus taktilen Sensoren und eine Methode zur Klassifikation ergriffener Objekte, basierend auf den Daten einer rechteckigen Oberfläche, werden beschrieben. Mit Hilfe dieser Matrix können unter schiedliche Arten von Oberflächen bei Berührung erkannt werden, was es für das Tastsystem möglich macht, Verschiebung, Drehung und Größe eines Objektes unabhängig voneinander zu erkennen