Embedded Electronic System Based on Dedicated Hardware DSPs for Electronic Skin Implementation

The effort to develop an electronic skin is highly motivated by many application domains namely robotics, biomedical instrumentations, and replacement prosthetic devices. Several e-skin systems have been proposed recently and have demonstrated the need of an embedded electronic system for tactile data processing either to mimic the human skin or to respond to the application demands. Processing tactile data requires efficient methods to extract meaningful information from raw sensors data. In this framework, our goal is the development of a dedicated embedded electronic system for electronic skin. The embedded electronic system has to acquire the tactile data, process and extract structured information. Machine Learning (ML) represents an effective method for data analysis in many domains: it has recently demonstrated its effectiveness in processing tactile sensors data. This paper presents an embedded electronic system based on dedicated hardware implementation for electronic skin systems. It provides a Tensorial kernel function implementation for machine learning based on Tensorial kernel approach. Results assess the time latency and the hardware complexity for real time functionality. The implementation results highlight the high amount of power consumption needed for the input touch modalities classification task. Conclusions and future perspectives are also presented

Embedded Electronic Systems for Electronic Skin Applications

The advances in sensor devices are potentially providing new solutions to many applications including prosthetics and robotics. Endowing upper limb prosthesis with tactile sensors (electronic/sensitive skin) can be used to provide tactile sensory feedback to the amputees. In this regard, the prosthetic device is meant to be equipped with tactile sensing system allowing the user limb to receive tactile feedback about objects and contact surfaces. Thus, embedding tactile sensing system is required for wearable sensors that should cover wide areas of the prosthetics. However, embedding sensing system involves set of challenges in terms of power consumption, data processing, real-time response and design scalability (e-skin may include large number of tactile sensors). The tactile sensing system is constituted of: (i) a tactile sensor array, (ii) an interface electronic circuit, (iii) an embedded processing unit, and (iv) a communication interface to transmit tactile data. The objective of the thesis is to develop an efficient embedded tactile sensing system targeting e-skin application (e.g. prosthetic) by: 1) developing a low power and miniaturized interface electronics circuit, operating in real-time; 2) proposing an efficient algorithm for embedded tactile data processing, affecting the system time latency and power consumption; 3) implementing an efficient communication channel/interface, suitable for large amount of data generated from large number of sensors. Most of the interface electronics for tactile sensing system proposed in the literature are composed of signal conditioning and commercial data acquisition devices (i.e. DAQ). However, these devices are bulky (PC-based) and thus not suitable for portable prosthetics from the size, power consumption and scalability point of view. Regarding the tactile data processing, some works have exploited machine learning methods for extracting meaningful information from tactile data. However, embedding these algorithms poses some challenges because of 1) the high amount of data to be processed significantly affecting the real time functionality, and 2) the complex processing tasks imposing burden in terms of power consumption. On the other hand, the literature shows lack in studies addressing data transfer in tactile sensing system. Thus, dealing with large number of sensors will pose challenges on the communication bandwidth and reliability. Therefore, this thesis exploits three approaches: 1) Developing a low power and miniaturized Interface Electronics (IE), capable of interfacing and acquiring signals from large number of tactile sensors in real-time. We developed a portable IE system based on a low power arm microcontroller and a DDC232 A/D converter, that handles an array of 32 tactile sensors. Upon touch applied to the sensors, the IE acquires and pre-process the sensor signals at low power consumption achieving a battery lifetime of about 22 hours. Then we assessed the functionality of the IE by carrying out Electrical and electromechanical characterization experiments to monitor the response of the interface electronics with PVDF-based piezoelectric sensors. The results of electrical and electromechanical tests validate the correct functionality of the proposed system. In addition, we implemented filtering methods on the IE that reduced the effect of noise in the system. Furthermore, we evaluated our proposed IE by integrating it in tactile sensory feedback system, showing effective deliver of tactile data to the user. The proposed system overcomes similar state of art solutions dealing with higher number of input channels and maintaining real time functionality. 2) Optimizing and implementing a tensorial-based machine learning algorithm for touch modality classification on embedded Zynq System-on-chip (SoC). The algorithm is based on Support Vector Machine classifier to discriminate between three input touch modality classes \u201cbrushing\u201d, \u201crolling\u201d and \u201csliding\u201d. We introduced an efficient algorithm minimizing the hardware implementation complexity in terms of number of operations and memory storage which directly affect time latency and power consumption. With respect to the original algorithm, the proposed approach \u2013 implemented on Zynq SoC \u2013 achieved reduction in the number of operations per inference from 545 M-ops to 18 M-ops and the memory storage from 52.2 KB to 1.7 KB. Moreover, the proposed method speeds up the inference time by a factor of 43 7 at a cost of only 2% loss in accuracy, enabling the algorithm to run on embedded processing unit and to extract tactile information in real-time. 3) Implementing a robust and efficient data transfer channel to transfer aggregated data at high transmission data rate and low power consumption. In this approach, we proposed and demonstrated a tactile sensory feedback system based on an optical communication link for prosthetic applications. The optical link features a low power and wide transmission bandwidth, which makes the feedback system suitable for large number of tactile sensors. The low power transmission is due to the employed UWB-based optical modulation. We implemented a system prototype, consisting of digital transmitter and receiver boards and acquisition circuits to interface 32 piezoelectric sensors. Then we evaluated the system performance by measuring, processing and transmitting data of the 32 piezoelectric sensors at 100 Mbps data rate through the optical link, at 50 pJ/bit communication energy consumption. Experimental results have validated the functionality and demonstrated the real time operation of the proposed sensory feedback system

Piezoelectric Transducers Based on Aluminum Nitride and Polyimide for Tactile Applications

The development of micro systems with smart sensing capabilities is paving the way to progresses in the technology for humanoid robotics. The importance of sensory feedback has been recognized the enabler of a high degree of autonomy for robotic systems. In tactile applications, it can be exploited not only to avoid objects slipping during their manipulation but also to allow safe interaction with humans and unknown objects and environments. In order to ensure the minimal deformation of an object during subtle manipulation tasks, information not only on contact forces between the object and fingers but also on contact geometry and contact friction characteristics has to be provided. Touch, unlike other senses, is a critical component that plays a fundamental role in dexterous manipulation capabilities and in the evaluation of objects properties such as type of material, shape, texture, stiffness, which is not easily possible by vision alone. Understanding of unstructured environments is made possible by touch through the determination of stress distribution in the surrounding area of physical contact. To this aim, tactile sensing and pressure detection systems should be integrated as an artificial tactile system. As illustrated in the Chapter I, the role of external stimuli detection in humans is provided by a great number of sensorial receptors: they are specialized endings whose structure and location in the skin determine their specific signal transmission characteristics. Especially, mechanoreceptors are specialized in the conversion of the mechanical deformations caused by force, vibration or slip on skin into electrical nerve impulses which are processed and encoded by the central nervous system. Highly miniaturized systems based on MEMS technology seem to imitate properly the large number of fast responsive mechanoreceptors present in human skin. Moreover, an artificial electronic skin should be lightweight, flexible, soft and wearable and it should be fabricated with compliant materials. In this respect a big challenge of bio-inspired technologies is the efficient application of flexible active materials to convert the mechanical pressure or stress into a usable electric signal (voltage or current). In the emerging field of soft active materials, able of large deformation, piezoelectrics have been recognized as a really promising and attractive material in both sensing and actuation applications. As outlined in Chapter II, there is a wide choice of materials and material forms (ceramics: PZT; polycrystalline films: ZnO, AlN; polymers and copolymers: PVDF, PVDF-TrFe) which are actively piezoelectric and exhibit features more or less attractive. Among them, aluminum nitride is a promising piezoelectric material for flexible technology. It has moderate piezoelectric coefficient, when available in c-axis oriented polycrystalline columnar structure, but, at same time, it exhibits low dielectric constant, high temperature stability, large band gap, large electrical resistivity, high breakdown voltage and low dielectric loss which make it suitable for transducers and high thermal conductivity which implies low thermal drifts. The high chemical stability allows AlN to be used in humid environments. Moreover, all the above properties and its deposition method make AlN compatible with CMOS technology. Exploiting the features of the AlN, three-dimensional AlN dome-shaped cells, embedded between two metal electrodes, are proposed in this thesis. They are fabricated on general purpose Kapton™ substrate, exploiting the flexibility of the polymer and the electrical stability of the semiconductor at the same time. As matter of fact, the crystalline layers release a compressive stress over the polymer, generating three-dimensional structures with reduced stiffness, compared to the semiconductor materials. In Chapter III, a mathematical model to calculate the residual stresses which arise because of mismatch in coefficient of thermal expansion between layers and because of mismatch in lattice constants between the substrate and the epitaxially grown films is adopted. The theoretical equation is then used to evaluate the dependence of geometrical features of the fabricated three-dimensional structures on compressive residual stress. Moreover, FEM simulations and theoretical models analysis are developed in order to qualitative explore the operation principle of curved membranes, which are labelled dome-shaped diaphragm transducers (DSDT), both as sensors and as piezo-actuators and for the related design optimization. For the reliability of the proposed device as a force/pressure sensor and piezo-actuator, an exhaustive electromechanical characterization of the devices is carried out. A complete description of the microfabrication processes is also provided. As shown in Chapter IV, standard microfabrication techniques are employed to fabricate the array of DSDTs. The overall microfabrication process involves deposition of metal and piezoelectric films, photolithography and plasma-based dry and wet etching to pattern thin films with the desired features. The DSDT devices are designed and developed according to FEM and theoretical analysis and following the typical requirements of force/pressure systems for tactile applications. Experimental analyses are also accomplished to extract the relationship between the compressive residual stress due to the aluminum nitride and the geometries of the devices. They reveal different deformations, proving the dependence of the geometrical features of the three-dimensional structures on residual stress. Moreover, electrical characterization is performed to determine capacitance and impedance of the DSDTs and to experimentally calculate the relative dielectric constant of sputtered AlN piezoelectric film. In order to investigate the mechanical behaviour of the curved circular transducers, a characterization of the flexural deflection modes of the DSDT membranes is carried out. The natural frequency of vibrations and the corresponding displacements are measured by a Laser Doppler Vibrometer when a suitable oscillating voltage, with known amplitude, is applied to drive the piezo-DSDTs. Finally, being developed for tactile sensing purpose, the proposed technology is tested in order to explore the electromechanical response of the device when impulsive dynamic and/or long static forces are applied. The study on the impulsive dynamic and long static stimuli detection is then performed by using an ad hoc setup measuring both the applied loading forces and the corresponding generated voltage and capacitance variation. These measurements allow a thorough test of the sensing abilities of the AlN-based DSDT cells. Finally, as stated in Chapter V, the proposed technology exhibits an improved electromechanical coupling with higher mechanical deformation per unit energy compared with the conventional plate structures, when the devices are used as piezo-actuator. On the other hand, it is well suited to realize large area tactile sensors for robotics applications, opening up new perspectives to the development of latest generation biomimetic sensors and allowing the design and the fabrication of miniaturized devices

Energy-efficient embedded machine learning algorithms for smart sensing systems

Embedded autonomous electronic systems are required in numerous application domains such as Internet of Things (IoT), wearable devices, and biomedical systems. Embedded electronic systems usually host sensors, and each sensor hosts multiple input channels (e.g., tactile, vision), tightly coupled to the electronic computing unit (ECU). The ECU extracts information by often employing sophisticated methods, e.g., Machine Learning. However, embedding Machine Learning algorithms poses essential challenges in terms of hardware resources and energy consumption because of: 1) the high amount of data to be processed; 2) computationally demanding methods. Leveraging on the trade-off between quality requirements versus computational complexity and time latency could reduce the system complexity without affecting the performance. The objectives of the thesis are to develop: 1) energy-efficient arithmetic circuits outperforming state of the art solutions for embedded machine learning algorithms, 2) an energy-efficient embedded electronic system for the \u201celectronic-skin\u201d (e-skin) application. As such, this thesis exploits two main approaches: Approximate Computing: In recent years, the approximate computing paradigm became a significant major field of research since it is able to enhance the energy efficiency and performance of digital systems. \u201cApproximate Computing\u201d(AC) turned out to be a practical approach to trade accuracy for better power, latency, and size . AC targets error-resilient applications and offers promising benefits by conserving some resources. Usually, approximate results are acceptable for many applications, e.g., tactile data processing,image processing , and data mining ; thus, it is highly recommended to take advantage of energy reduction with minimal variation in performance . In our work, we developed two approximate multipliers: 1) the first one is called \u201cMETA\u201d multiplier and is based on the Error Tolerant Adder (ETA), 2) the second one is called \u201cApproximate Baugh-Wooley(BW)\u201d multiplier where the approximations are implemented in the generation of the partial products. We showed that the proposed approximate arithmetic circuits could achieve a relevant reduction in power consumption and time delay around 80.4% and 24%, respectively, with respect to the exact BW multiplier. Next, to prove the feasibility of AC in real world applications, we explored the approximate multipliers on a case study as the e-skin application. The e-skin application is defined as multiple sensing components, including 1) structural materials, 2) signal processing, 3) data acquisition, and 4) data processing. Particularly, processing the originated data from the e-skin into low or high-level information is the main problem to be addressed by the embedded electronic system. Many studies have shown that Machine Learning is a promising approach in processing tactile data when classifying input touch modalities. In our work, we proposed a methodology for evaluating the behavior of the system when introducing approximate arithmetic circuits in the main stages (i.e., signal and data processing stages) of the system. Based on the proposed methodology, we first implemented the approximate multipliers on the low-pass Finite Impulse Response (FIR) filter in the signal processing stage of the application. We noticed that the FIR filter based on (Approx-BW) outperforms state of the art solutions, while respecting the tradeoff between accuracy and power consumption, with an SNR degradation of 1.39dB. Second, we implemented approximate adders and multipliers respectively into the Coordinate Rotational Digital Computer (CORDIC) and the Singular Value Decomposition (SVD) circuits; since CORDIC and SVD take a significant part of the computationally expensive Machine Learning algorithms employed in tactile data processing. We showed benefits of up to 21% and 19% in power reduction at the cost of less than 5% accuracy loss for CORDIC and SVD circuits when scaling the number of approximated bits. 2) Parallel Computing Platforms (PCP): Exploiting parallel architectures for near-threshold computing based on multi-core clusters is a promising approach to improve the performance of smart sensing systems. In our work, we exploited a novel computing platform embedding a Parallel Ultra Low Power processor (PULP), called \u201cMr. Wolf,\u201d for the implementation of Machine Learning (ML) algorithms for touch modalities classification. First, we tested the ML algorithms at the software level; for RGB images as a case study and tactile dataset, we achieved accuracy respectively equal to 97% and 83.5%. After validating the effectiveness of the ML algorithm at the software level, we performed the on-board classification of two touch modalities, demonstrating the promising use of Mr. Wolf for smart sensing systems. Moreover, we proposed a memory management strategy for storing the needed amount of trained tensors (i.e., 50 trained tensors for each class) in the on-chip memory. We evaluated the execution cycles for Mr. Wolf using a single core, 2 cores, and 3 cores, taking advantage of the benefits of the parallelization. We presented a comparison with the popular low power ARM Cortex-M4F microcontroller employed, usually for battery-operated devices. We showed that the ML algorithm on the proposed platform runs 3.7 times faster than ARM Cortex M4F (STM32F40), consuming only 28 mW. The proposed platform achieves 15 7 better energy efficiency than the classification done on the STM32F40, consuming 81mJ per classification and 150 pJ per operation

Development of composite piezoelectric materials for tactile sensing

This thesis has dealt with the preparation and the characterization of piezoelectric 0-3 composite materials. The technological aim is to evaluate a material potentially suitable for the development of a sensitive skin for human robotics. As secondary objectives this material should be cheap (relatively) and easy processable in order to make it possibly adaptable for industrial production. For these reasons, 0-3 composites were prepared and characterized. Raw materials were selected among the most commonly used for piezoelectric applications. PVDF is the most widely used ferroelectric polymer. It was used along with two of its copolymers: PVDF-HeFP, developed to have improved flexibility but no piezoelectricity and PVDF-TrFE, developed to obtain the crystalline piezoelectric phase whatever the process. PMMA was also studied. Barium titanate submicron-powder was chosen as piezo-active filler. Composites were prepared with increasing volume percentages of fillers. Two different processing methods were explored in order to evaluate their effect on the microstructure and the piezoelectric respons

Embedded Artificial Intelligence for Tactile Sensing

Electronic tactile sensing becomes an active research field whether for prosthetic applications, robotics, virtual reality or post stroke patients rehabilitation. To achieve such sensing, an array of sensors is used to retrieve human-skin like information, which is called Electronic skin (E-skin). Humans through their skins, are able to collect different types of information e.g. pressure, temperature, texture, etc. which are then passed to the nervous system, and finally to the brain in order to extract high level information from these sensory data. In order to make E-skin capable of such task, data acquired from E-skin should be filtered, processed, and then conveyed to the user (or robot). Processing these sensory information, should occur in real-time, taking in consideration the power limitation in such applications, especially prosthetic applications. The power consumption itself is related to different factors, one factor is the complexity of the algorithm e.g. number of FLOPs, and another is the memory consumption. In this thesis, I will focus on the processing of real tactile information, by 1)exploring different algorithms and methods for tactile data classification, 2)data organization and preprocessing of such tactile data and 3)hardware implementation. More precisely the focus will be on deep learning algorithms for tactile data processing mainly CNNs and RNNs, with energy-efficient embedded implementations. The proposed solution has proved less memory, FLOPs, and latency compared to the state of art (including tensorial SVM), applied to real tactile sensors data. Keywords: E-skin, tactile data processing, deep learning, CNN, RNN, LSTM, GRU, embedded, energy-efficient algorithms, edge computing, artificial intelligence

Human-Machine Interfaces using Distributed Sensing and Stimulation Systems

As the technology moves towards more natural human-machine interfaces (e.g. bionic limbs, teleoperation, virtual reality), it is necessary to develop a sensory feedback system in order to foster embodiment and achieve better immersion in the control system. Contemporary feedback interfaces presented in research use few sensors and stimulation units to feedback at most two discrete feedback variables (e.g. grasping force and aperture), whereas the human sense of touch relies on a distributed network of mechanoreceptors providing a wide bandwidth of information. To provide this type of feedback, it is necessary to develop a distributed sensing system that could extract a wide range of information during the interaction between the robot and the environment. In addition, a distributed feedback interface is needed to deliver such information to the user. This thesis proposes the development of a distributed sensing system (e-skin) to acquire tactile sensation, a first integration of distributed sensing system on a robotic hand, the development of a sensory feedback system that compromises the distributed sensing system and a distributed stimulation system, and finally the implementation of deep learning methods for the classification of tactile data. It\u2019s core focus addresses the development and testing of a sensory feedback system, based on the latest distributed sensing and stimulation techniques. To this end, the thesis is comprised of two introductory chapters that describe the state of art in the field, the objectives, and the used methodology and contributions; as well as six studies that tackled the development of human-machine interfaces

SUAS: A Novel Soft Underwater Artificial Skin with Capacitive Transducers and Hyperelastic Membrane

The paper presents physical modeling, design, simulations, and experimentation on a novel Soft Underwater Artificial Skin (SUAS) used as tactile sensor. The SUAS functions as an electrostatic capacitive sensor, and it is composed of a hyperelastic membrane used as external cover and oil inside it used to compensate the marine pressure. Simulation has been performed studying and modeling the behavior of the external interface of the SUAS in contact with external concentrated loads in marine environment. Experiments on the external and internal components of the SUAS have been done using two different conductive layers in oil. A first prototype has been realized using a 3D printer. The results of the paper underline how the soft materials permit better adhesion of the conductive layer to the transducers of the SUAS obtaining higher capacitance. The results here presented confirmed the first hypotheses presented in a last work and opened new ways in the large-scale underwater tactile sensor design and development. The investigations are performed in collaboration with a national Italian project named MARIS, regarding the possible extension to the underwater field of the technologies developed within the European project ROBOSKIN

Novel Tactile-SIFT Descriptor for Object Shape Recognition

Using a tactile array sensor to recognize an object often requires multiple touches at different positions. This process is prone to move or rotate the object, which inevitably increases difficulty in object recognition. To cope with the unknown object movement, this paper proposes a new tactile-SIFT descriptor to extract features in view of gradients in the tactile image to represent objects, to allow the features being invariant to object translation and rotation. The tactile-SIFT segments a tactile image into overlapping subpatches, each of which is represented using a dn-dimensional gradient vector, similar to the classic SIFT descriptor. Tactile-SIFT descriptors obtained from multiple touches form a dictionary of k words, and the bag-of-words method is then used to identify objects. The proposed method has been validated by classifying 18 real objects with data from an off-the-shelf tactile sensor. The parameters of the tactile-SIFT descriptor, including the dimension size dn and the number of subpatches sp, are studied. It is found that the optimal performance is obtained using an 8-D descriptor with three subpatches, taking both the classification accuracy and time efficiency into consideration. By employing tactile-SIFT, a recognition rate of 91.33% has been achieved with a dictionary size of 50 clusters using only 15 touches