Data-driven Tactile Sensing using Spatially Overlapping Signals

Providing robots with distributed, robust and accurate tactile feedback is a fundamental problem in robotics because of the large number of tasks that require physical interaction with objects. Tactile sensors can provide robots with information about the location of each point of contact with the manipulated object, an estimation of the contact forces applied (normal and shear) and even slip detection. Despite significant advances in touch and force transduction, tactile sensing is still far from ubiquitous in robotic manipulation. Existing methods for building touch sensors have proven difficult to integrate into robot fingers due to multiple challenges, including difficulty in covering multicurved surfaces, high wire count, or packaging constrains preventing their use in dexterous hands. In this dissertation, we focus on the development of soft tactile systems that can be deployed over complex, three-dimensional surfaces with a low wire count and using easily accessible manufacturing methods. To this effect, we present a general methodology called spatially overlapping signals. The key idea behind our method is to embed multiple sensing terminals in a volume of soft material which can be deployed over arbitrary, non-developable surfaces. Unlike a traditional taxel, these sensing terminals are not capable of measuring strain on their own. Instead, we take measurements across pairs of sensing terminals. Applying strain in the receptive field of this terminal pair should measurably affect the signal associated with it. As we embed multiple sensing terminals in this soft material, a significant overlap of these receptive fields occurs across the whole active sensing area, providing us with a very rich dataset characterizing the contact event. The use of an all-pairs approach, where all possible combinations of sensing terminals pairs are used, maximizes the number of signals extracted while reducing the total number of wires for the overall sensor, which in turn facilitates its integration. Building an analytical model for how this rich signal set relates to various contacts events can be very challenging. Further, any such model would depend on knowing the exact locations of the terminals in the sensor, thus requiring very precise manufacturing. Instead, we build forward models of our sensors from data. We collect training data using a dataset of controlled indentations of known characteristics, directly learning the mapping between our signals and the variables characterizing a contact event. This approach allows for accessible, cheap manufacturing while enabling extensive coverage of curved surfaces. The concept of spatially overlapping signals can be realized using various transduction methods; we demonstrate sensors using piezoresistance, pressure transducers and optics. With piezoresistivity we measure resistance values across various electrodes embedded in a carbon nanotubes infused elastomer to determine the location of touch. Using commercially available pressure transducers embedded in various configurations inside a soft volume of rubber, we show its possible to localize contacts across a curved surface. Finally, using optics, we measure light transport between LEDs and photodiodes inside a clear elastomer which makes up our sensor. Our optical sensors are able to detect both the location and depth of an indentation very accurately on both planar and multicurved surfaces. Our Distributed Interleaved Signals for Contact via Optics or D.I.S.C.O Finger is the culmination of this methodology: a fully integrated, sensorized robot finger, with a low wire count and designed for easy integration into dexterous manipulators. Our DISCO Finger can generally determine contact location with sub-millimeter accuracy, and contact force to within 10% (and often with 5%) of the true value without the need for analytical models. While our data-driven method requires training data representative of the final operational conditions that the system will encounter, we show our finger can be robust to novel contact scenarios where the shape of the indenter has not been seen during training. Moreover, the forward model that predicts contact locations and applied normal force can be transfered to new fingers with minimal loss of performance, eliminating the need to collect training data for each individual finger. We believe that rich tactile information, in a highly functional form with limited blind spots and a simple integration path into complete systems, like we demonstrate in this dissertation, will prove to be an important enabler for data-driven complex robotic motor skills, such as dexterous manipulation

Micro-patterned graphene-based sensing skins for human physiological monitoring.

Ultrathin, flexible, conformal, and skin-like electronic transducers are emerging as promising candidates for noninvasive and nonintrusive human health monitoring. In this work, a wearable sensing membrane is developed by patterning a graphene-based solution onto ultrathin medical tape, which can then be attached to the skin for monitoring human physiological parameters and physical activity. Here, the sensor is validated for monitoring finger bending/movements and for recognizing hand motion patterns, thereby demonstrating its future potential for evaluating athletic performance, physical therapy, and designing next-generation human-machine interfaces. Furthermore, this study also quantifies the sensor's ability to monitor eye blinking and radial pulse in real-time, which can find broader applications for the healthcare sector. Overall, the printed graphene-based sensing skin is highly conformable, flexible, lightweight, nonintrusive, mechanically robust, and is characterized by high strain sensitivity

The state of carbon and the piezoresistive effect in silicon oxycarbide ceramics

The present work reports on the morphology of carbon, the electrical properties and the piezoresistve effect in polymer-derived silicon oxycarbides (SiOC/C nanocomposites) within carbon concentrations of 1 to 45 vol.%. The nanocomposites have been prepared by pyrolysis of poly-organosilicon precursors or preceramic polymers and a subsequent densification step using spark plasma sintering (1000 < T < 1800 °C; P = 50 MPa, Argon). The obtained samples are characterized by the means of spectroscopic (Raman, TGA-FTIR, XRD, XPS) and electrical (dc conductivity, impedance, Hall effect) investigations. The SiOC/C composites consist of a glassy matrix (silica, SiOxC4-x), silicon carbide and segregated carbon and is simplified as a two phase system (glass/carbon) for the description of the electrical and piezoresistive properties. The state of carbon within SiOC/C depends on the carbon content of the precursor and the thermal treatment. According to UV Raman the microstructure of carbon changes from a disordered (amorphous) to nano-crystalline state within 1000 < T < 1800 °C. The progressive ordering or graphitization, respectively, is illustrated by the increasing lateral crystal size (7.5 < La < 20 nm) and the corresponding decrease of the density of defects as derived from the intensity ratio AD/AG of the Raman D- and G-band. Vacancies have been identified as main type of defects. The electrical and piezoresitive properties of SiOC/C mainly depend on the state of carbon with the exception of samples with very low carbon content (C < 1 vol.%). For samples treated at T = 1600 °C the resistivity decreases by several orders of magnitude at a critical carbon concentration of about 6 vol.%. For samples treated at 1400 and 1100 °C the percolation threshold shifts from 8 to 20 vol.% of carbon because of the lower degree of graphitization. Beyond the percolation threshold weakly activated transport (Ea < 0.1 eV) occurs in conjugated sp² bonds within a continuous 3D network. The charge carrier mobility (μ ≈ 3 cm²/Vs) and density (N = 1018 – 1020 cm-³) of carriers are comparable to those of glassy carbon. A change from band-like transport to conduction in localized states at the percolation threshold is indicated by an increase in Ea ≈ 0.3 eV and presumably arises from electron/hole confinement when the localization length approaches a lateral crystal size of La ≲ 10 nm (La decreases from about 8 to 4 nm within 1 to 10 vol.% carbon). SiOC/C changes its resistivity with strain and, accordingly, the piezoresistive effect also relies on the carbon content and its degree of graphitization. Within 1 to 6 vol.% the gauge factor (GF) increases up to the percolation threshold (GF ≈ 45) and appears to arise from charge carrier tunneling. The extremely high GF values (> 1000) at the percolation threshold suggest either divergence of the GF vs. carbon concentration function or local strain concentration. Above the percolation threshold GF is related to the density of defects within the continuous carbon network. A strain-induced increase of the density of states near EF is identified as origin of the piezoresistive effect, a change in mobility is unable to explain the experimental decrease of the resistance with applied strain. The obtain results on the piezoresistivity favor the development of high-temperature strain sensors able to detect static and dynamic excitation. As a proof of concept a demonstrator to sense the natural frequency of a planetary gear has been developed

Human-Machine Interfaces using Distributed Sensing and Stimulation Systems

As the technology moves towards more natural human-machine interfaces (e.g. bionic limbs, teleoperation, virtual reality), it is necessary to develop a sensory feedback system in order to foster embodiment and achieve better immersion in the control system. Contemporary feedback interfaces presented in research use few sensors and stimulation units to feedback at most two discrete feedback variables (e.g. grasping force and aperture), whereas the human sense of touch relies on a distributed network of mechanoreceptors providing a wide bandwidth of information. To provide this type of feedback, it is necessary to develop a distributed sensing system that could extract a wide range of information during the interaction between the robot and the environment. In addition, a distributed feedback interface is needed to deliver such information to the user. This thesis proposes the development of a distributed sensing system (e-skin) to acquire tactile sensation, a first integration of distributed sensing system on a robotic hand, the development of a sensory feedback system that compromises the distributed sensing system and a distributed stimulation system, and finally the implementation of deep learning methods for the classification of tactile data. It\u2019s core focus addresses the development and testing of a sensory feedback system, based on the latest distributed sensing and stimulation techniques. To this end, the thesis is comprised of two introductory chapters that describe the state of art in the field, the objectives, and the used methodology and contributions; as well as six studies that tackled the development of human-machine interfaces

Neuromorphic Computing Systems for Tactile Sensing Perception

Touch sensing plays an important role in humans daily life. Tasks like exploring, grasping and manipulating objects deeply rely on it. As such, Robots and hand prosthesis endowed with the sense of touch can better and more easily manipulate objects, and physically collaborate with other agents. Towards this goal, information about touched objects and surfaces has to be inferred from raw data coming from the sensors. The orientation of edges, which is employed as a pre-processing stage in both artificial vision and touch, is a key indication for object discrimination. Inspired on the encoding of edges in human first-order tactile afferents, we developed a biologically inspired, spiking models architecture that mimics human tactile perception with computational primitives that are implementable on low-power subthreshold neuromorphic hardware. The network architecture uses three layers of Leaky Integrate and Fire neurons to distinguish different edge orientations of a bar pressed on the artificial skin of the iCub robot. We demonstrated that the network architecture can learn the appropriate connectivity through unsupervised spike-based learning, and that the number and spatial distribution of sensitive areas within receptive fields are important in edge orientation discrimination. The unconstrained and random structure of the connectivity among layers can produce unbalanced activity in the output neurons, which are driven by a variable amount of synaptic inputs. We explored two different mechanisms of synaptic normalization (weights normalization and homeostasis), defining how this can be useful during the learning phase and inference phase. The network is successfully able to discriminate between 35 orientations of 36 (0 degree to 180 degree with 5 degree step increments) with homeostasis and weights normalization mechanism. Besides edge orientation discrimination, we modified the network architecture to be able to classify six different touch modalities (e.g. poke, press, grab, squeeze, push, and rolling a wheel). We demonstrated the ability of the network to learn appropriate connectivity patterns for the classification, achieving a total accuracy of 88.3 %. Furthermore, another application scenario on the tactile object shapes recognition has been considered because of its importance in robotic manipulation. We illustrated that the network architecture with 2 layers of spiking neurons was able to discriminate the tactile object shapes with accuracy 100 %, after integrating to it an array of 160 piezoresistive tactile sensors where the object shapes are applied

Conductivity-Based Nanocomposite Structural Health Monitoring via Electrical Impedance Tomography.

Nanocomposites have incredible potential when integrated as matrices in fiber-reinforced composites for transformative conductivity-based structural health monitoring (SHM). Key to this potential is the dependence of nanocomposite conductivity on well-connected nanofiller networks. Damage that severs the network or strain that affects the connectivity will manifest as a conductivity change. These damage or strain-induced conductivity changes can then be detected and spatially located by electrical impedance tomography (EIT). The nanofiller network therefore acts as an integrated sensor network giving unprecedented insight into the mechanical state of the structure. Despite the potential of combining nanocomposite matrices with EIT, important limitations exist. EIT, for example, requires large electrode arrays that are too unwieldy to be practically implemented on in-service structures. EIT also tends to be insensitive to small, highly localized conductivity losses as is expected from common modes fiber-reinforced composite damage such as matrix cracking and delamination. Furthermore, there are gaps in the fundamental understanding of nanocomposite conductivity. This thesis advances the state of the art by addressing the aforementioned limitations of EIT for conductivity-based SHM. This is done by insightfully leveraging the unique properties of nanocomposite conductivity to circumvent EIT's limitations. First, nanocomposite conductive properties are studied. This results in fundamental contributions to the understanding of nanocomposite piezoresistivity, the influence of nanofiller alignment on transverse percolation and conductivity, and conductivity evolution due to electrical loading. Next, the potential of EIT for conductivity-based health monitoring is studied and demonstrated for damage detection in carbon nanofiber (CNF)/epoxy and glass fiber/epoxy laminates manufactured with carbon black (CB) filler and for strain detection in CNF/polyurethane (PU). Lastly, the previously developed insights into nanocomposite conductive properties and damage detection via EIT are combined to greatly enhance EIT for SHM. This is done by first exploring how the sensitivity of EIT to delamination can be enhanced through nanofiller alignment and tailoring. A method of coupling the EIT image reconstruction process with known conductivity changes such as those induced by straining piezoresistive nanocomposites is developed and presented. This approach will tremendously bolster the image quality of EIT or, synonymously, significantly abate the number of electrodes required by EIT.PhDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111613/1/ttallman_1.pd