Optical skin: Sensor-integration-free multimodal flexible sensing

The biological skin enables animals to sense various stimuli. Extensive efforts have been made recently to develop smart skin-like sensors to extend the capabilities of biological skins; however, simultaneous sensing of several types of stimuli in a large area remains challenging because this requires large-scale sensor integration with numerous wire connections. We propose a simple, highly sensitive, and multimodal sensing approach, which does not require integrating multiple sensors. The proposed approach is based on an optical interference technique, which can encode the information of various stimuli as a spatial pattern. In contrast to the existing approach, the proposed approach, combined with a deep neural network, enables us to freely select the sensing mode according to our purpose. As a key example, we demonstrate simultaneous sensing mode of three different physical quantities, contact force, contact location, and temperature, using a single soft material without requiring complex integration. Another unique property of the proposed approach is spatially continuous sensing with ultrahigh resolution of few tens of micrometers, which enables identifying the shape of the object in contact. Furthermore, we present a haptic soft device for a human-machine interface. The proposed approach encourages the development of high-performance optical skins.Comment: 13 pages, 11 figure

Functional mimicry of Ruffini receptors with fibre Bragg gratings and deep neural networks enables a bio-inspired large-area tactile-sensitive skin

Collaborative robots are expected to physically interact with humans in daily living and the workplace, including industrial and healthcare settings. A key related enabling technology is tactile sensing, which currently requires addressing the outstanding scientific challenge to simultaneously detect contact location and intensity by means of soft conformable artificial skins adapting over large areas to the complex curved geometries of robot embodiments. In this work, the development of a large-area sensitive soft skin with a curved geometry is presented, allowing for robot total-body coverage through modular patches. The biomimetic skin consists of a soft polymeric matrix, resembling a human forearm, embedded with photonic fibre Bragg grating transducers, which partially mimics Ruffini mechanoreceptor functionality with diffuse, overlapping receptive fields. A convolutional neural network deep learning algorithm and a multigrid neuron integration process were implemented to decode the fibre Bragg grating sensor outputs for inference of contact force magnitude and localization through the skin surface. Results of 35 mN (interquartile range 56 mN) and 3.2 mm (interquartile range 2.3 mm) median errors were achieved for force and localization predictions, respectively. Demonstrations with an anthropomorphic arm pave the way towards artificial intelligence based integrated skins enabling safe human–robot cooperation via machine intelligence

Empowering and assisting natural human mobility: The simbiosis walker

This paper presents the complete development of the Simbiosis Smart Walker. The device is equipped with a set of sensor subsystems to acquire user-machine interaction forces and the temporal evolution of user's feet during gait. The authors present an adaptive filtering technique used for the identification and separation of different components found on the human-machine interaction forces. This technique allowed isolating the components related with the navigational commands and developing a Fuzzy logic controller to guide the device. The Smart Walker was clinically validated at the Spinal Cord Injury Hospital of Toledo - Spain, presenting great acceptability by spinal chord injury patients and clinical staf

An Embedded, Multi-Modal Sensor System for Scalable Robotic and Prosthetic Hand Fingers

Grasping and manipulation with anthropomorphic robotic and prosthetic hands presents a scientific challenge regarding mechanical design, sensor system, and control. Apart from the mechanical design of such hands, embedding sensors needed for closed-loop control of grasping tasks remains a hard problem due to limited space and required high level of integration of different components. In this paper we present a scalable design model of artificial fingers, which combines mechanical design and embedded electronics with a sophisticated multi-modal sensor system consisting of sensors for sensing normal and shear force, distance, acceleration, temperature, and joint angles. The design is fully parametric, allowing automated scaling of the fingers to arbitrary dimensions in the human hand spectrum. To this end, the electronic parts are composed of interchangeable modules that facilitate the echanical scaling of the fingers and are fully enclosed by the mechanical parts of the finger. The resulting design model allows deriving freely scalable and multimodally sensorised fingers for robotic and prosthetic hands. Four physical demonstrators are assembled and tested to evaluate the approach

An Embedded, Multi-Modal Sensor System for Scalable Robotic and Prosthetic Hand Fingers

Grasping and manipulation with anthropomorphic robotic and prosthetic hands presents a scientific challenge regarding mechanical design, sensor system, and control. Apart from the mechanical design of such hands, embedding sensors needed for closed-loop control of grasping tasks remains a hard problem due to limited space and required high level of integration of different components. In this paper we present a scalable design model of artificial fingers, which combines mechanical design and embedded electronics with a sophisticated multi-modal sensor system consisting of sensors for sensing normal and shear force, distance, acceleration, temperature, and joint angles. The design is fully parametric, allowing automated scaling of the fingers to arbitrary dimensions in the human hand spectrum. To this end, the electronic parts are composed of interchangeable modules that facilitate the echanical scaling of the fingers and are fully enclosed by the mechanical parts of the finger. The resulting design model allows deriving freely scalable and multimodally sensorised fingers for robotic and prosthetic hands. Four physical demonstrators are assembled and tested to evaluate the approach

Development and Characterization of highly flexible and conformable electronic devices for wearable applications

As shown in the story, humanity has tried to develop objects, tools, and devices that could first help to survive in a difficult environment and then improve everyday life. The idea of creating objects that can be worn to restore or improve human abilities or to help during daily routine has fueled technological development and research since the beginning of technological advancement. Wearable technology goes back hundreds of years, and one of the first examples was the invention of glasses to restore the sight, or the wristwatch when big watches were reduced to something that people could take with them anywhere. However, it could be considered that, only when the computer age was established, wearable electronic devices were developed and started to spread out and get into the market. Wearable electronics are a category of technological devices that can be transferred into clothes or directly in touch with the body, typically as accessories or clothing, and these devices can be designed to provide different functionalities, such as notification sending, communication abilities, health and fitness monitoring, and even augmented or virtual reality experiences. In recent years, organic electronics have been deeply investigated as a technology platform to develop devices using biocompatible materials that can be deposited and processed on flexible and even ultra-flexible substrates. The high mechanical flexibility of such materials leads to a new category of devices going beyond wearable devices to more-than-wearable applications. In this context, epidermal electronics is a closely related field that focuses on developing electronic devices that can be directly attached to the skin with a minimally invasive, comfortable, and possibly enabling long-term application. The main object of this Ph.D. research activity is the development and optimization of a technology for the realization of wearable and more-than-wearable devices, able to meet all the new needs in this field, such as the low-cost production process and the mechanical flexibility of the devices and deposition over large areas on unconventional substrates, exploiting all the features and advantages of the organic electronic field, but also finding some solution to overcome the disadvantages of this technology. In this work, different application fields were studied, such as health monitoring through biopotential acquisitions, the development, and optimization of multimodal physical sensors able to detect simultaneously pressure and temperature for tactile and artificial skin applications, and the development of flexible high-performing transistors as a building block for the future of wearable and electronic-skin applications

A Novel Broadband Forcecardiography Sensor for Simultaneous Monitoring of Respiration, Infrasonic Cardiac Vibrations and Heart Sounds

The precordial mechanical vibrations generated by cardiac contractions have a rich frequency spectrum. While the lowest frequencies can be palpated, the higher infrasonic frequencies are usually captured by the seismocardiogram (SCG) signal and the audible ones correspond to heart sounds. Forcecardiography (FCG) is a non-invasive technique that measures these vibrations via force sensing resistors (FSR). This study presents a new piezoelectric sensor able to record all heart vibrations simultaneously, as well as a respiration signal. The new sensor was compared to the FSR-based one to assess its suitability for FCG. An electrocardiogram (ECG) lead and a signal from an electro-resistive respiration band (ERB) were synchronously acquired as references on six healthy volunteers (4 males, 2 females) at rest. The raw signals from the piezoelectric and the FSR-based sensors turned out to be very similar. The raw signals were divided into four components: Forcerespirogram (FRG), Low-Frequency FCG (LF-FCG), High-Frequency FCG (HF-FCG) and heart sounds (HS-FCG). A beat-by-beat comparison of FCG and ECG signals was carried out by means of regression, correlation and Bland–Altman analyses, and similarly for respiration signals (FRG and ERB). The results showed that the infrasonic FCG components are strongly related to the cardiac cycle (R2 > 0.999, null bias and Limits of Agreement (LoA) of ± 4.9 ms for HF-FCG; R2 > 0.99, null bias and LoA of ± 26.9 ms for LF-FCG) and the FRG inter-breath intervals are consistent with ERB ones (R2 > 0.99, non-significant bias and LoA of ± 0.46 s). Furthermore, the piezoelectric sensor was tested against an accelerometer and an electronic stethoscope: synchronous acquisitions were performed to quantify the similarity between the signals. ECG-triggered ensemble averages (synchronized with R-peaks) of HF-FCG and SCG showed a correlation greater than 0.81, while those of HS-FCG and PCG scored a correlation greater than 0.85. The piezoelectric sensor demonstrated superior performances as compared to the FSR, providing more accurate, beat-by-beat measurements. This is the first time that a single piezoelectric sensor demonstrated the ability to simultaneously capture respiration, heart sounds, an SCG-like signal (i.e., HF-FCG) and the LF-FCG signal, which may provide information on ventricular emptying and filling events. According to these preliminary results the novel piezoelectric FCG sensor stands as a promising device for accurate, unobtrusive, long-term monitoring of cardiorespiratory functions and paves the way for a wide range of potential applications, both in the research and clinical fields. However, these results should be confirmed by further analyses on a larger cohort of subjects, possibly including also pathological patients