Real-Time Somatosensory Feedback for Neural Prosthesis Control: System Development and Experimental Validation

Recent advances in neural prosthetics have provided patients with the ability to use signals derived from motor areas of the cerebral cortex to directly control an external device under visually guided closed-loop control. To attain a more natural form of prosthesis control, it is desirable to develop systems capable of providing real-time somatosensory feedback as well as visual feedback, akin to how we naturally process sensory information to control our limbs. To this end, a sophisticated data acquisition, control and feedback system was developed for neural prosthetics and psychophysics research. The system deterministically collects and processes high volume neural ensemble activity, limb kinematics, and eye movements while generating visual stimuli in an immersive three-dimensional virtual reality (VR) environment. A vibrotactile feedback device was also developed and incorporated into the system. It delivers real-time limb kinematics feedback in the form of continuous, graded vibratory stimulation. A flexible and intuitive user interface allows the researcher to design experimental paradigms and adjust parameters on the fly during experiments. A psychophysical study was conducted using this system to evaluate the potential use of vibrotactile feedback as a sensory substitution method to provide somatosensory feedback for neural prosthesis control. The study also aimed to provide insight into the mechanisms of multimodal sensory processing and sensory-motor control. Able-bodied human subjects performed a trajectory-following reach task in the VR environment under different visual and vibrotactile feedback conditions. The study showed that vibrotactile feedback is capable of enhancing motor performance, implying that subjects were able to integrate and effectively use this new 'proprioceptive-like' sensory modality. Subjects were also able to partially maintain task performance using vibrotactile feedback in the absence of visual feedback. Improved motor learning and motor skill consolidation were also observed after training in the VR environment with vibrotactile feedback. These results suggest that vibrotactile feedback may be a viable method for delivering somatosensory feedback for applications such as neural prosthesis control, motor rehabilitation, and enhanced human-computer interaction

A forearm controller and tactile display

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2005.Includes bibliographical references (leaves 187-192).This thesis discusses the design and implementation of ARMadillo, a simple virtual environment interface in the form of a small wireless device that is worn on the forearm. Designed to be portable, intuitive, and low cost, the device tracks the orientation of the arm with accelerometers, magnetic field sensors, and gyroscopes, fusing the data with a quaternion based Unscented Kalman Filter. The orientation estimate is mapped to a virtual space that is perceived through a tactile display containing an array of vibrating motors. The controller is driven with an 8051 microcontroller, and includes a BlueTooth module and an extension slot for CompactFlash cards. The device was designed to be simple and modular, and can support a variety of interesting applications, some of which were implemented and will be discussed. These fall into two main classes. The first is a set of artistic applications, represented by a suite of virtual musical instruments that can be played with arm movements and felt through the tactile display, The second class involves utilitarian applications, including a custom Braille-like system called Arm Braille, and tactile guidance. A wearable Braille display intended to be used for reading navigational signs and text messages was tested on two sight-impaired subjects who were able to recognize Braille characters reliably after 25 minutes of training, and read words by the end of an hour.by David Matthew Sachs.S.M

Abstract— MULTI-CHANNEL VIBROTACTILE DISPLAY FOR TELEOPERATED ASSEMBLY

This paper presents the design and testing of a multi-channel vibrotactile display. It is composed of a cylindrical handle with four embedded vibrating elements driven by piezoelectric beams. Vibrations are transmitted to the hands through arrays of pins. The device was tested in sensory substitution for conveying force information during a teleoperated peg insertion. Results show that the device is effective in reducing peak forces during the insertion task