Distributed Sensing and Stimulation Systems Towards Sense of Touch Restoration in Prosthetics

Modern prostheses aim at restoring the functional and aesthetic characteristics of the lost limb. To foster prosthesis embodiment and functionality, it is necessary to restitute both volitional control and sensory feedback. Contemporary feedback interfaces presented in research use few sensors and stimulation units to feedback at most two discrete feedback variables (e.g. grasping force and aperture), whereas the human sense of touch relies on a distributed network of mechanoreceptors providing high-fidelity spatial information. To provide this type of feedback in prosthetics, it is necessary to sense tactile information from artificial skin placed on the prosthesis and transmit tactile feedback above the amputation in order to map the interaction between the prosthesis and the environment. This thesis proposes the integration of distributed sensing systems (e-skin) to acquire tactile sensation, and non-invasive multichannel electrotactile feedback and virtual reality to deliver high-bandwidth information to the user. Its core focus addresses the development and testing of close-loop sensory feedback human-machine interface, based on the latest distributed sensing and stimulation techniques for restoring the sense of touch in prosthetics. To this end, the thesis is comprised of two introductory chapters that describe the state of art in the field, the objectives and the used methodology and contributions; as well as three studies distributed over stimulation system level and sensing system level. The first study presents the development of close-loop compensatory tracking system to evaluate the usability and effectiveness of electrotactile sensory feedback in enabling real-time close-loop control in prosthetics. It examines and compares the subject\u2019s adaptive performance and tolerance to random latencies while performing the dynamic control task (i.e. position control) and simultaneously receiving either visual feedback or electrotactile feedback for communicating the momentary tracking error. Moreover, it reported the minimum time delay needed for an abrupt impairment of users\u2019 performance. The experimental results have shown that electrotactile feedback performance is less prone to changes with longer delays. However, visual feedback drops faster than electrotactile with increased time delays. This is a good indication for the effectiveness of electrotactile feedback in enabling close- loop control in prosthetics, since some delays are inevitable. The second study describes the development of a novel non-invasive compact multichannel interface for electrotactile feedback, containing 24 pads electrode matrix, with fully programmable stimulation unit, that investigates the ability of able-bodied human subjects to localize the electrotactile stimulus delivered through the electrode matrix. Furthermore, it designed a novel dual parameter -modulation (interleaved frequency and intensity) and compared it to conventional stimulation (same frequency for all pads). In addition and for the first time, it compared the electrotactile stimulation to mechanical stimulation. More, it exposes the integration of virtual prosthesis with the developed system in order to achieve better user experience and object manipulation through mapping the acquired real-time collected tactile data and feedback it simultaneously to the user. The experimental results demonstrated that the proposed interleaved coding substantially improved the spatial localization compared to same-frequency stimulation. Furthermore, it showed that same-frequency stimulation was equivalent to mechanical stimulation, whereas the performance with dual-parameter modulation was significantly better. The third study presents the realization of a novel, flexible, screen- printed e-skin based on P(VDF-TrFE) piezoelectric polymers, that would cover the fingertips and the palm of the prosthetic hand (particularly the Michelangelo hand by Ottobock) and an assistive sensorized glove for stroke patients. Moreover, it developed a new validation methodology to examine the sensors behavior while being solicited. The characterization results showed compatibility between the expected (modeled) behavior of the electrical response of each sensor to measured mechanical (normal) force at the skin surface, which in turn proved the combination of both fabrication and assembly processes was successful. This paves the way to define a practical, simplified and reproducible characterization protocol for e-skin patches In conclusion, by adopting innovative methodologies in sensing and stimulation systems, this thesis advances the overall development of close-loop sensory feedback human-machine interface used for restoration of sense of touch in prosthetics. Moreover, this research could lead to high-bandwidth high-fidelity transmission of tactile information for modern dexterous prostheses that could ameliorate the end user experience and facilitate it acceptance in the daily life

Sensor Developments for Electrophysiological Monitoring in Healthcare

Recent years have seen a renewal of interest in the development of sensor systems which can be used to monitor electrophysiological signals in a number of different settings. These include clinical, outside of the clinical setting with the subject ambulatory and going about their daily lives, and over long periods. The primary impetus for this is the challenge of providing healthcare for the ageing population based on home health monitoring, telehealth and telemedicine. Another stimulus is the demand for life sign monitoring of critical personnel such as fire fighters and military combatants. A related area of interest which, whilst not in the category of healthcare, utilises many of the same approaches, is that of sports physiology for both professional athletes and for recreation. Clinical diagnosis of conditions in, for example, cardiology and neurology remain based on conventional sensors, using established electrodes and well understood electrode placements. However, the demands of long term health monitoring, rehabilitation support and assistive technology for the disabled and elderly are leading research groups such as ours towards novel sensors, wearable and wireless enabled systems and flexible sensor arrays

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human–machine interfacing

Tactile feedback is relevant in a broad range of human–machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject’s hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy (i.e. median{IQR} of 88.6{11}% for static and 93.3{5}% for dynamic patterns). The proposed system is an important step towards the development of a high-density human–machine interfacing between the user and a robotic han

Multichannel biphasic muscle stimulation system for post stroke rehabilitation

We present biphasic stimulator electronics developed for a wearable functional electrical stimulation system. The reported stimulator electronics consist of a twenty four channel biphasic stimulator. The stimulator circuitry is physically smaller per channel and offers a greater degree of control over stimulation parameters than existing functional electrical stimulator systems. The design achieves this by using, off the shelf multichannel high voltage switch integrated circuits combined with discrete current limiting and dc blocking circuitry for the frontend, and field programmable gate array based logic to manage pulse timing. The system has been tested on both healthy adults and those with reduced upper limb function following a stroke. Initial testing on healthy users has shown the stimulator can reliably generate specific target gestures such as palm opening or pointing with an average accuracy of better than 4 degrees across all gestures. Tests on stroke survivors produced some movement but this was limited by the mechanical movement available in those users' hands

A system for electrotactile feedback using electronic skin and flexible matrix electrodes: Experimental evaluation

Myoelectric prostheses are successfully controlled using muscle electrical activity, thereby restoring lost motor functions. However, the somatosensory feedback from the prosthesis to the user is still missing. The sensory substitution methods described in the literature comprise mostly simple position and force sensors combined with discrete stimulation units. The present study describes a novel system for sophisticated electrotactile feedback integrating advanced distributed sensing (electronic skin) and stimulation (matrix electrodes). The system was tested in eight healthy subjects who were asked to recognize the shape, trajectory, and direction of a set of dynamic movement patterns (single lines, geometrical objects, letters) presented on the electronic skin. The experiments demonstrated that the system successfully translated the mechanical interaction into the moving electrotactile profiles, which the subjects could recognize with a good performance (shape recognition: 86±8% lines, 73±13% geometries, 72±12% letters). In particular, the subjects could identify the movement direction with a high confidence. These results are in accordance with previous studies investigating the recognition of moving stimuli in human subjects. This is an important development towards closed-loop prostheses providing comprehensive and sophisticated tactile feedback to the user, facilitating the control and the embodiment of the artificial device into the user body scheme

Electronic systems for the restoration of the sense of touch in upper limb prosthetics

In the last few years, research on active prosthetics for upper limbs focused on improving the human functionalities and the control. New methods have been proposed for measuring the user muscle activity and translating it into the prosthesis control commands. Developing the feed-forward interface so that the prosthesis better follows the intention of the user is an important step towards improving the quality of life of people with limb amputation. However, prosthesis users can neither feel if something or someone is touching them over the prosthesis and nor perceive the temperature or roughness of objects. Prosthesis users are helped by looking at an object, but they cannot detect anything otherwise. Their sight gives them most information. Therefore, to foster the prosthesis embodiment and utility, it is necessary to have a prosthetic system that not only responds to the control signals provided by the user, but also transmits back to the user the information about the current state of the prosthesis. This thesis presents an electronic skin system to close the loop in prostheses towards the restoration of the sense of touch in prosthesis users. The proposed electronic skin system inlcudes an advanced distributed sensing (electronic skin), a system for (i) signal conditioning, (ii) data acquisition, and (iii) data processing, and a stimulation system. The idea is to integrate all these components into a myoelectric prosthesis. Embedding the electronic system and the sensing materials is a critical issue on the way of development of new prostheses. In particular, processing the data, originated from the electronic skin, into low- or high-level information is the key issue to be addressed by the embedded electronic system. Recently, it has been proved that the Machine Learning is a promising approach in processing tactile sensors information. Many studies have been shown the Machine Learning eectiveness in the classication of input touch modalities.More specically, this thesis is focused on the stimulation system, allowing the communication of a mechanical interaction from the electronic skin to prosthesis users, and the dedicated implementation of algorithms for processing tactile data originating from the electronic skin. On system level, the thesis provides design of the experimental setup, experimental protocol, and of algorithms to process tactile data. On architectural level, the thesis proposes a design ow for the implementation of digital circuits for both FPGA and integrated circuits, and techniques for the power management of embedded systems for Machine Learning algorithms

Improved Functional Electrical Stimulation (FES) systems for optimized selectivity

En el presente trabajo de Tesis se aborda el tema de selectividad en la activación muscular en las aplicaciones de estimulación eléctrica funcional en el miembro superior. Este tipo de tecnología presenta una serie de problemas, de los cuales se destaca la dificultad de estimular solo un musculo debido al reducido tamaño de ellos y la alta densidad de músculos presentes. El problema este, llamado “oveflow" en inglés, aparece debido a la dificultad de controlar el camino de la corriente eléctrica debajo de la piel hacia el musculo. Los electrodos tipo matriz han sido la apuesta de varios estudios para afrontar estas dificultades y varias soluciones se han presentado los últimos 10 años. Estos estudios se enfocan en el desarrollo de sistemas y de la tecnología electrónica necesaria. Sin embargo, faltan estudios de los parámetros que influyen en la activación selectiva de los músculos y como optimizar todos estos parámetros. El objetivo de esta tesis es el desarrollo de un sistema de estimulación eléctrica funcional superficial, basado en los electrodos tipo matriz, que optimiza la selectividad muscular. Se conoce desde la bibliografía que el camino de la corriente eléctrica debajo de la piel depende de una serie de factores. Entre ellos se encuentran la posición, la forma y el tamaño del electrodo de cátodo, la posición del electrodo de ánodo y la impedancia de la membrana de gel del electrodo. Esta tesis se enfoca en optimizar estos factores que permiten modular el campo eléctrico que se genera debajo del electrodo. El sistema que se ha desarrollado se usó con ese objetivo. El sistema se ha evaluado en dos escenarios: en pacientes con temblor y pacientes con lesión medular

Pulse Signal System: Sensing, Data Acquisition and Body Area Network

Heart rate variability (HRV) is an important physiological signal of the human body, which can serve as a useful biomarker for the cardiovascular health status of an individual. There are many methods to measure the HRV using electrical devices, such as ECG and PPG etc. This work presents a novel HRV detection method which is based on pressure detection on the human wrist. This method has been compared with existing HRV detection methods. In this work, the proposed system for HRV detection is based on polyvinylidene difluoride (PVDF) sensor, which can measure tiny pressure on its surface. Three PVDF sensors are mounted on the wrist, and a three-channel conditioning circuit is used to amplify signals generated by the sensors. An analog-to-digital converter and Arduino microcontroller are used to sample and process the signal. Based on the obtained signals, the HRV can be processed and detected by the proposed PVDF-sensor-based system. Another contribution of this work is in designing a wireless body area network (WBAN) to transmit data acquired on the human body. This WBAN combines two different wireless network protocols, for both efficient power consumption and data rate. Bluetooth Low Energy protocol is used for transmitting data from the microcontroller to a personal device, and Wi-Fi is used to send data to other terminals. This provides the potential for remote HRV signal monitoring. A dataset consisting of two subjects was used to experimentally validate the proposed system design and signal processing method. ECG signals are acquired from subjects with wrist pulse signals for comparison as standard signal. The waveforms of ECG signals and wrist pulse signals are compared and HRV values are calculated from these two signals separately. The result shows that HRV calculated by wrist pulse has low error rate. A test of movement effect shows the sensor can resist mild motions of wrist. Some future improvements of system design and further signal processing methods are also discussed in the last chapter

Doctor of Philosophy

dissertationHands are so central to the human experience, yet we often take for granted the capacity to maneuver objects, to form a gesture, or to caress a loved-one’s hand. The effects of hand amputation can be severe, including functional disabilities, chronic phantom pain, and a profound sense of loss which can lead to depression and anxiety. In previous studies, peripheral-nerve interfaces, such as the Utah Slanted Electrode Array (USEA), have shown potential for restoring a sense of touch and prosthesis movement control. This dissertation represents a substantial step forward in the use of the USEAs for clinical careâ€"ultimately providing human amputees with widespread hand sensation that is functionally useful and psychologically meaningful. In completion of this ultimate objective, we report on three major advances. First, we performed the first dual-USEA implantations in human amputees; placing one USEA in the residual median nerve and another USEA in the residual ulnar nerve. Chapter 2 of this dissertation shows that USEAs provided full-hand sensory coverage, and that movement of the implant site to the upper arm in the second subject, proximal to nerve branch-points to extrinsic hand muscles, enabled activation of both proprioceptive sensory percepts and cutaneous percepts. Second, in Chapter 3, we report on successful use of USEA-evoked sensory percepts for functional discrimination tasks. We provide a comprehensive report of functional discrimination among USEA-evoked sensory percepts from three human subjects, including discrimination among multiple proprioceptive or cutaneous sensory percepts with different hand locations, sensory qualities, and/or intensities. Finally, in Chapter 4, we report on the psychological value of multiple degree of freedom prosthesis control, multisensor prosthesis sensation, and closed-loop control. This chapter represents the first report of prosthesis embodiment during closed-loop and open-loop prosthesis control by an amputee, as well as the most sophisticated closed-loop prosthesis control reported in literature to-date, including 5-degree-of-freedom motor control and sensory feedback from 4 hand locations. Ultimately, we expect that USEA-evoked hand sensations may be used as part of a take-home prosthesis system which will provide users with both advanced functional capabilities and a meaningful sense of embodiment and limb restoration