A Multi-Modal Sensing Glove for Human Manual-Interaction Studies

We present an integrated sensing glove that combines two of the most visionary wearable sensing technologies to provide both hand posture sensing and tactile pressure sensing in a unique, lightweight, and stretchable device. Namely, hand posture reconstruction employs Knitted Piezoresistive Fabrics that allows us to measure bending. From only five of these sensors (one for each finger) the full hand pose of a 19 degrees of freedom (DOF) hand model is reconstructed leveraging optimal sensor placement and estimation techniques. To this end, we exploit a-priori information of synergistic coordination patterns in grasping tasks. Tactile sensing employs a piezoresistive fabric allowing us to measure normal forces in more than 50 taxels spread over the palmar surface of the glove. We describe both sensing technologies, report on the software integration of both modalities, and describe a preliminary evaluation experiment analyzing hand postures and force patterns during grasping. Results of the reconstruction are promising and encourage us to push further our approach with potential applications in neuroscience, virtual reality, robotics and tele-operation

Ultra conformable and multimodal tactile sensors based on organic field-effect transistors

Cognitive psychology is the branch of psychology related to all the processes by which sensory input is transformed, processed and used. Academic and industrial research has always invested time and resources to develop devices capable to simulate the behavior of the organs where the perceptions are located. In recent years, in fact, there have been numerous discoveries related to new materials, and new devices, capable of reproducing, in a reliable manner, the sensory behavior of humans. Particular interest in scientific research has been aimed at understanding and reproducing of man's tactile sensations. It is known that, through the receptors of the skin, it is possible to detect sensations such as pain, changes in pressure and/or temperature. The development of tactile sensor technology had a significant increase in the last years of 1970s, thanks to the important surveys of Stojiljkovic, Harmon and Lumelsky who presented the firsts prototype of sensors for artificial skin applications, and summarized the main characteristics and requirements of tactile sensors. Recently, organic electronics has been deeply investigated as technology for the fabrication of tactile sensors using biocompatible materials, which can be deposited and processed on ultra flexible and ultra conformable substrates. In general, the most attractive property of these materials is mainly related to their high mechanical flexibility, which is mandatory for artificial skin applications. The main object of this PhD research activity was the development and optimization of an innovative technology for the realization of physical sensors able to detect pressure and temperature variations, which can be applied in the field of biomedical engineering and biorobotics. By exploiting the particular characteristics of the employed materials, such as mechanical flexibility, the proposed sensors are very suitable to be integrated with flexible structures (for example plastics) as a pressure and temperature sensor, and therefore, ideal for the realization of an artificial skin like. In Chapter 1, the basics of humans somatosensory system will be introduced: after a brief description of tactile thermoreceptors, mechanoreceptors and nociceptors, a definition of electronic skin and its characteristics will be provided. In Chapter 2, a wide analysis of the state of the art will be reported. Several and different examples of tactile sensor (in inorganic and organic technology) will be presented, underlining advantages and disadvantages for each approach. In Chapter 3, the firsts experimental results, obtained in the first part of my PhD program, will be presented. All the steps of the fabrication process of the devices will be described, as well as the measurement setup used for the electrical characterization of the sensors. In Chapter 4, the sensor structure optimization will be presented. It will be demonstrated how the presented devices are able to sense simultaneously thermal and mechanical stimuli. Moreover, it will be demonstrated that, thanks to an alternative and innovative fabrication process, the sensors can be transferred directly on skin, thus proving the suitability of the proposed sensor architecture for tactile applications

Functional mimicry of Ruffini receptors with fibre Bragg gratings and deep neural networks enables a bio-inspired large-area tactile-sensitive skin

Collaborative robots are expected to physically interact with humans in daily living and the workplace, including industrial and healthcare settings. A key related enabling technology is tactile sensing, which currently requires addressing the outstanding scientific challenge to simultaneously detect contact location and intensity by means of soft conformable artificial skins adapting over large areas to the complex curved geometries of robot embodiments. In this work, the development of a large-area sensitive soft skin with a curved geometry is presented, allowing for robot total-body coverage through modular patches. The biomimetic skin consists of a soft polymeric matrix, resembling a human forearm, embedded with photonic fibre Bragg grating transducers, which partially mimics Ruffini mechanoreceptor functionality with diffuse, overlapping receptive fields. A convolutional neural network deep learning algorithm and a multigrid neuron integration process were implemented to decode the fibre Bragg grating sensor outputs for inference of contact force magnitude and localization through the skin surface. Results of 35 mN (interquartile range 56 mN) and 3.2 mm (interquartile range 2.3 mm) median errors were achieved for force and localization predictions, respectively. Demonstrations with an anthropomorphic arm pave the way towards artificial intelligence based integrated skins enabling safe human–robot cooperation via machine intelligence

Multi-fingered haptic palpation utilizing granular jamming stiffness feedback actuators

This paper describes a multi-fingered haptic palpation method using stiffness feedback actuators for simulating tissue palpation procedures in traditional and in robot-assisted minimally invasive surgery. Soft tissue stiffness is simulated by changing the stiffness property of the actuator during palpation. For the first time, granular jamming and pneumatic air actuation are combined to realize stiffness modulation. The stiffness feedback actuator is validated by stiffness measurements in indentation tests and through stiffness discrimination based on a user study. According to the indentation test results, the introduction of a pneumatic chamber to granular jamming can amplify the stiffness variation range and reduce hysteresis of the actuator. The advantage of multi-fingered palpation using the proposed actuators is proven by the comparison of the results of the stiffness discrimination performance using two-fingered (sensitivity: 82.2%, specificity: 88.9%, positive predicative value: 80.0%, accuracy: 85.4%, time: 4.84 s) and single-fingered (sensitivity: 76.4%, specificity: 85.7%, positive predicative value: 75.3%, accuracy: 81.8%, time: 7.48 s) stiffness feedback

Stretchable metallization technologies for skin-like transducers

The skin is not only the largest human organ, capable of accomplishing distributed and multimodal sensing functions. Replicating the versatility of skin artificially is a significant challenge, not only in terms of signal processing but also in mechanics. Stretchable electronics are an approach designed to cover human and artificial limbs and provide wearable sensing capabilities: motion sensors distributed on the hand of neurologically impaired patients could help therapists quantify their abilities; prostheses equipped with multiple tactile sensors could enable amputees to naturally adjust their grasp force. Skin-like electronic systems have specific requirements: they must mechanically adapt to the deformations imposed by the body they equip with minimal impediment to its natural movements, while also providing sufficient electrical performance for sensor transduction and passing electrical signals and power. A metallization ensuring stable conductivity under large strains is a prerequisite to designing and assembling wearable circuits that are integrated with several types of sensors. In this work, two innovative metallization processes have been developed to enable scalable integration of multiple sensing modalities in stretchable circuits. First, stretchable micro-cracked gold (Au) thin films were interfaced with gallium indium eutectic (EGaIn) liquid metal wires. The Au films, thermally evaporated on silicone elastomer substrates, combined high sheet resistance (9 to 30 Ohm/sq) and high sensitivity to strain up to 50%. The EGaIn wires drawn using a micro-plotting setup had a low gauge factor (2) and a low sheet resistance (5 mOhm/sq). Second, a novel physical vapor deposition method to deposit of thin gallium-based biphasic (solid-liquid) films over large areas was achieved. The obtained conductors combined a low sheet resistance (0.5 Ohm/sq), a low gauge factor (~1 up to 80% strain), and a failure strain of more than 400%. They could be patterned down to 10 µm critical dimensions. Skin-like sensors for the hand were assembled using the two processes and their capabilities were demonstrated. Thin (0.5 mm) silicone strips integrating EGaIN wires and micro-cracked Au strain gauges were mounted on gloves to encode the position of a biomimetic robotic finger and a human finger. In combination with soft pressure sensors, they enabled precise grasp analysis over a limited range of motion. Then, biphasic films were micro-patterned on silicone to assemble 50 µm thin epidermal strain gauges. The strain gauges were attached on a user's finger and accurately encoded fine grasping tasks covering most of the human hand range of motion. The biphasic films were also used to power wireless MEMS pressure sensors integrated in a rubber scaffold. The device was mounted on a prosthetic hand to encode normal forces in the 0 N to 20 N range with excellent linearity. The epidermal strain sensors are currently being used to quantify the tremors of patients with Parkinson's disease. In the future, the unique properties of the biphasic films could enable advanced artificial skins integrating a high density of soft transducers and traditional high-performance circuits

Proposal of the Tactile Glove Device

This project aims to develop a tactile glove device and a virtual environment inserted in the context of tactile internet. The tactile glove allows a human operator to interact remotely with objects from a 3D environment through tactile feedback or tactile sensation. In other words, the human operator is able to feel the contour and texture from virtual objects. Applications such as remote diagnostics, games, remote analysis of materials, and others in which objects could be virtualized can be significantly improved using this kind of device. These gloves have been an essential device in all research on the internet next generation called “Tactile Internet”, in which this project is inserted. Unlike the works presented in the literature, the novelty of this work is related to architecture, and tactile devices developed. They are within the 10 ms round trip latency limits required in a tactile internet environment. Details of hardware and software designs of a tactile glove, as well as the virtual environment, are described. Results and comparative analysis about round trip latency time in the tactile internet environment is developed

Fluidic Fabric Muscle Sheets for Wearable and Soft Robotics

Conformable robotic systems are attractive for applications in which they can be used to actuate structures with large surface areas, to provide forces through wearable garments, or to realize autonomous robotic systems. We present a new family of soft actuators that we refer to as Fluidic Fabric Muscle Sheets (FFMS). They are composite fabric structures that integrate fluidic transmissions based on arrays of elastic tubes. These sheet-like actuators can strain, squeeze, bend, and conform to hard or soft objects of arbitrary shapes or sizes, including the human body. We show how to design and fabricate FFMS actuators via facile apparel engineering methods, including computerized sewing techniques. Together, these determine the distributions of stresses and strains that can be generated by the FFMS. We present a simple mathematical model that proves effective for predicting their performance. FFMS can operate at frequencies of 5 Hertz or more, achieve engineering strains exceeding 100%, and exert forces greater than 115 times their own weight. They can be safely used in intimate contact with the human body even when delivering stresses exceeding 10$^\text{6}$ Pascals. We demonstrate their versatility for actuating a variety of bodies or structures, and in configurations that perform multi-axis actuation, including bending and shape change. As we also show, FFMS can be used to exert forces on body tissues for wearable and biomedical applications. We demonstrate several potential use cases, including a miniature steerable robot, a glove for grasp assistance, garments for applying compression to the extremities, and devices for actuating small body regions or tissues via localized skin stretch.Comment: 32 pages, 10 figure