Haptic Glove and Platform with Gestural Control For Neuromorphic Tactile Sensory Feedback In Medical Telepresence

Advancements in the study of the human sense of touch are fueling the field of haptics. This is paving the way for augmenting sensory perception during object palpation in tele-surgery and reproducing the sensed information through tactile feedback. Here, we present a novel tele-palpation apparatus that enables the user to detect nodules with various distinct stiffness buried in an ad-hoc polymeric phantom. The contact force measured by the platform was encoded using a neuromorphic model and reproduced on the index fingertip of a remote user through a haptic glove embedding a piezoelectric disk. We assessed the effectiveness of this feedback in allowing nodule identification under two experimental conditions of real-time telepresence: In Line of Sight (ILS), where the platform was placed in the visible range of a user; and the more demanding Not In Line of Sight (NILS), with the platform and the user being 50 km apart. We found that the entailed percentage of identification was higher for stiffer inclusions with respect to the softer ones (average of 74% within the duration of the task), in both telepresence conditions evaluated. These promising results call for further exploration of tactile augmentation technology for telepresence in medical interventions

Neuromorphic tactile sensor array based on fiber Bragg gratings to encode object qualities

Emulating the sense of touch is fundamental to endow robotic systems with perception abilities. This work presents an unprecedented mechanoreceptor-like neuromorphic tactile sensor implemented with fiber optic sensing technologies. A robotic gripper was sensorized using soft and flexible tactile sensors based on Fiber Bragg Grating (FBG) transducers and a neuro-bio-inspired model to extract tactile features. The FBGs connected to the neuron model emulated biological mechanoreceptors in encoding tactile information by means of spikes. This conversion of inflowing tactile information into event-based spikes has an advantage of reduced bandwidth requirements to allow communication between sensing and computational subsystems of robots. The outputs of the sensor were converted into spiking on-off events by means of an architecture implemented in a Field Programmable Gate Array (FPGA) and applied to robotic manipulation tasks to evaluate the effectiveness of such information encoding strategy. Different tasks were performed with the objective to grant fine manipulation abilities using the features extracted from the grasped objects (i.e., size and hardness). This is envisioned to be a futuristic sensor technology combining two promising technologies: optical and neuromorphic sensing

Assessment of intuitiveness and comfort of wearable haptic feedback strategies for assisting level and stair walking

Nowadays, lower-limb prostheses are reaching real-world usability especially on ground-level walking. However, some key tasks such as stair walking are still quite demanding. Providing haptic feedback about the foot placement on the steps might reduce the cognitive load of the task, compensating for increased dependency on vision and lessen the risk of falling. Experiments on intact subjects can be useful to define the feedback strategies prior to clinical trials, but effective methods to assess the efficacy of the strategies are few and usually rely on the emulation of the disability condition. The present study reports on the design and testing of a wearable haptic feedback system in a protocol involving intact subjects to assess candidate strategies to be adopted in clinical trials. The system integrated a sensorized insole wirelessly connected to a textile waist belt equipped with three vibrating motors. Three stimulation strategies for mapping the insole pressure data to vibrotactile feedback were implemented and compared in terms of intuitiveness and comfort perceived during level and stair walking. The strategies were ranked using a relative rating approach, which highlighted the differences between them and suggested guidelines for their improvement. The feedback evaluation procedure proposed could facilitate the selection and improvement of haptic feedback strategies prior to clinical testing

Neuromorphic tactile sensor array based on fiber Bragg gratings to encode object qualities

Emulating the sense of touch is fundamental to endow robotic systems with perception abilities. This work presents an unprecedented mechanoreceptor-like neuromorphic tactile sensor implemented with fiber optic sensing technologies. A robotic gripper was sensorized using soft and flexible tactile sensors based on Fiber Bragg Grating (FBG) transducers and a neuro-bio-inspired model to extract tactile features. The FBGs connected to the neuron model emulated biological mechanoreceptors in encoding tactile information by means of spikes. This conversion of inflowing tactile information into event-based spikes has an advantage of reduced bandwidth requirements to allow communication between sensing and computational subsystems of robots. The outputs of the sensor were converted into spiking on-off events by means of an architecture implemented in a Field Programmable Gate Array (FPGA) and applied to robotic manipulation tasks to evaluate the effectiveness of such information encoding strategy. Different tasks were performed with the objective to grant fine manipulation abilities using the features extracted from the grasped objects (i.e., size and hardness). This is envisioned to be a futuristic sensor technology combining two promising technologies: optical and neuromorphic sensing

A mechatronic platform for computer aided detection of nodules in anatomopathological analyses via stiffness and ultrasound measurements

This study presents a platform for ex-vivo detection of cancer nodules, addressing automation of medical diagnoses in surgery and associated histological analyses. The proposed approach takes advantage of the property of cancer to alter the mechanical and acoustical properties of tissues, because of changes in stiffness and density. A force sensor and an ultrasound probe were combined to detect such alterations during force-regulated indentations. To explore the specimens, regardless of their orientation and shape, a scanned area of the test sample was defined using shape recognition applying optical background subtraction to the images captured by a camera. The motorized platform was validated using seven phantom tissues, simulating the mechanical and acoustical properties of ex-vivo diseased tissues, including stiffer nodules that can be encountered in pathological conditions during histological analyses. Results demonstrated the platform’s ability to automatically explore and identify the inclusions in the phantom. Overall, the system was able to correctly identify up to 90.3% of the inclusions by means of stiffness in combination with ultrasound measurements, paving pathways towards robotic palpation during intraoperative examinations

Towards telementoring for needle insertion

This thesis presents a telementoring haptic system that aims to achieve online, bidirectional interaction between a medical mentor and a medical trainee during needle insertion training. The system can be seen as the integration of two modes of haptic feedback: (i) kinetic feedback which informs the mentor of the trainee's insertion forces and (ii) tactile feedback which guides a trainee along an insertion path according to the mentor's recommended motions. In our first experiment, addressing (i), we evaluated five forms of feedback with respect to trainee-force classification determined by the mentor. Our system adopts haptic plus visual feedback as it yielded the highest median accuracy (100\%). In the second experiment, addressing (ii), we tested the efficacy of a tactile guidance sleeve in comparison to traditional verbal guidance. The current design of the sleeve was shown to be less intuitive than verbal guidance. We formed educated hypotheses about possible improvements to the sleeve and discuss lessons learned about tactile feedback applied to guidance in medical tasks.Mechanical Engineerin

Tactile sensing and control of robotic manipulator integrating fiber Bragg grating strain-sensor

Tactile sensing is an instrumental modality of robotic manipulation, as it provides information that is not accessible via remote sensors such as cameras or lidars. Touch is particularly crucial in unstructured environments, where the robot’s internal representation of manipulated objects is uncertain. In this study we present the sensorization of an existing artificial hand, with the aim to achieve fine control of robotic limbs and perception of object’s physical properties. Tactile feedback is conveyed by means of a soft sensor integrated at the fingertip of a robotic hand. The sensor consists of an optical fiber, housing Fiber Bragg Gratings (FBGs) transducers, embedded into a soft polymeric material integrated on a rigid hand. Through several tasks involving grasps of different objects in various conditions, the ability of the system to acquire information is assessed. Results show that a classifier based on the sensor outputs of the robotic hand is capable of accurately detecting both size and rigidity of the operated objects (99.36 and 100% accuracy, respectively). Furthermore, the outputs provide evidence of the ability to grab fragile objects without breakage or slippage e and to perform dynamic manipulative tasks, that involve the adaptation of fingers position based on the grasped objects’ condition