Spiking Neural Network Based on Threshold Encoding For Texture Recognition

This paper presents a neuromorphic computing model that classifies material textures using a neural coding scheme based on threshold encoding. The proposed threshold encoding converts raw tactile data of each texture into an eventbased data highlighting the spatio-temporal features needed to recognize human touch. Achieved results show that the model can categorize the input tactile signals into their corresponding material textures with high accuracy and fast inference. This work paves the way toward employing the proposed encoding method in more complex tactile based applications from the theoretical and hardware implementation aspects

cuneate spiking neural network learning to classify naturalistic texture stimuli under varying sensing conditions

Abstract We implemented a functional neuronal network that was able to learn and discriminate haptic features from biomimetic tactile sensor inputs using a two-layer spiking neuron model and homeostatic synaptic learning mechanism. The first order neuron model was used to emulate biological tactile afferents and the second order neuron model was used to emulate biological cuneate neurons. We have evaluated 10 naturalistic textures using a passive touch protocol, under varying sensing conditions. Tactile sensor data acquired with five textures under five sensing conditions were used for a synaptic learning process, to tune the synaptic weights between tactile afferents and cuneate neurons. Using post-learning synaptic weights, we evaluated the individual and population cuneate neuron responses by decoding across 10 stimuli, under varying sensing conditions. This resulted in a high decoding performance. We further validated the decoding performance across stimuli, irrespective of sensing velocities using a set of 25 cuneate neuron responses. This resulted in a median decoding performance of 96% across the set of cuneate neurons. Being able to learn and perform generalized discrimination across tactile stimuli, makes this functional spiking tactile system effective and suitable for further robotic applications

Neuromorphic Tactile Sensing System for Textural Features Classification

Artificial tactile sensing systems have gained significant attention in recent years due to their potential to enhance human-machine interaction. Numerous initiatives have been introduced to shift the computational paradigms of these systems towards a more biologically inspired approach, by incorporating neuromorphic computing methods. Despite the significant advances made by these systems, dependence on complex offline methods for classification (i.e. hand-crafted encoding features), remains a limitation for their real-time deployment. In this work, we present a neuromorphic tactile PVDF-based sensing system for textural features classification, that employs raw signals directly for classification. We first converted raw signals into spikes and then trained recurrent spiking neural networks (RSNNs) using Backpropagation Through Time (BPTT) with surrogate gradients to perform classification. We proposed an optimization method based on tuning the refractory period of the encoding neurons, to explore a potential trade-off between the computational cost and classification accuracy of the RSNN. The proposed method effectively identified two RSNNs with refractory period configurations that achieved a trade-off between the two evaluation metrics. Following this, we reduced the inference time steps of the selected RSNN during inference using a rate-coding based method. This method succeeded in saving around 26.6% out of the total original time steps. In summary, the proposed system paves the way for establishing an end-to-end neuromorphic approach for tactile textural features classification, through deploying the selected RSNNs on a dedicated neuromorphic hardware device for real-time inferences

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies

Neuroprosthetic Haptic Interface and Haptic Stimulation: Neuromorphic Microtransduction and EEG Alpha Variations

According to the recent studies on the psychophysiology of touch, a haptic effector designed in a neuromorphic way was projected, designing an electronic card as to be able to deliver variable signals over time and in intensity. The two-dimensional arrays of micro-actuators were made either with planar geometry or with three-dimensional, semi-spherical or “dome” geometry. Subsequently, on both the behavioral and the electrophysiological level the haptic sensation received by the effector was evaluated on 6 subjects and compared to real stimulations of different grains (Paper). During the various stimulations the subject was in a state of Resting State (RS). Each stimulation had a frequency range ranging from 2 to 500 Hz on 2 and 5 s. Analysis of behavioral responses and the alpha rhythm in RS showed significant differences for low frequencies vs Paper. RS highlighted differences in ROIs on the various frequency distributions, especially low frequencies in Frontal ROI. This pilot study indicates that the best frequencies for a haptic simulation are between a range from 20 Hz to 166 Hz

Neuromorphic tactile sensor array based on fiber Bragg gratings to encode object qualities

Emulating the sense of touch is fundamental to endow robotic systems with perception abilities. This work presents an unprecedented mechanoreceptor-like neuromorphic tactile sensor implemented with fiber optic sensing technologies. A robotic gripper was sensorized using soft and flexible tactile sensors based on Fiber Bragg Grating (FBG) transducers and a neuro-bio-inspired model to extract tactile features. The FBGs connected to the neuron model emulated biological mechanoreceptors in encoding tactile information by means of spikes. This conversion of inflowing tactile information into event-based spikes has an advantage of reduced bandwidth requirements to allow communication between sensing and computational subsystems of robots. The outputs of the sensor were converted into spiking on-off events by means of an architecture implemented in a Field Programmable Gate Array (FPGA) and applied to robotic manipulation tasks to evaluate the effectiveness of such information encoding strategy. Different tasks were performed with the objective to grant fine manipulation abilities using the features extracted from the grasped objects (i.e., size and hardness). This is envisioned to be a futuristic sensor technology combining two promising technologies: optical and neuromorphic sensing

Human-Inspired Neurorobotic System for Classifying Surface Textures by Touch

© 2016 IEEE. Giving robots the ability to classify surface textures requires appropriate sensors and algorithms. Inspired by the biology of human tactile perception, we implement a neurorobotic texture classifier with a recurrent spiking neural network, using a novel semisupervised approach for classifying dynamic stimuli. Input to the network is supplied by accelerometers mounted on a robotic arm. The sensor data are encoded by a heterogeneous population of neurons, modeled to match the spiking activity of mechanoreceptor cells. This activity is convolved by a hidden layer using bandpass filters to extract nonlinear frequency information from the spike trains. The resulting high-dimensional feature representation is then continuously classified using a neurally implemented support vector machine. We demonstrate that our system classifies 18 metal surface textures scanned in two opposite directions at a constant velocity. We also demonstrate that our approach significantly improves upon a baseline model that does not use the described feature extraction. This method can be performed in real-time using neuromorphic hardware, and can be extended to other applications that process dynamic stimuli online