A point process framework for modeling electrical stimulation of the auditory nerve

Model-based studies of auditory nerve responses to electrical stimulation can provide insight into the functioning of cochlear implants. Ideally, these studies can identify limitations in sound processing strategies and lead to improved methods for providing sound information to cochlear implant users. To accomplish this, models must accurately describe auditory nerve spiking while avoiding excessive complexity that would preclude large-scale simulations of populations of auditory nerve fibers and obscure insight into the mechanisms that influence neural encoding of sound information. In this spirit, we develop a point process model of the auditory nerve that provides a compact and accurate description of neural responses to electric stimulation. Inspired by the framework of generalized linear models, the proposed model consists of a cascade of linear and nonlinear stages. We show how each of these stages can be associated with biophysical mechanisms and related to models of neuronal dynamics. Moreover, we derive a semi-analytical procedure that uniquely determines each parameter in the model on the basis of fundamental statistics from recordings of single fiber responses to electric stimulation, including threshold, relative spread, jitter, and chronaxie. The model also accounts for refractory and summation effects that influence the responses of auditory nerve fibers to high pulse rate stimulation. Throughout, we compare model predictions to published physiological data and explain differences in auditory nerve responses to high and low pulse rate stimulation. We close by performing an ideal observer analysis of simulated spike trains in response to sinusoidally amplitude modulated stimuli and find that carrier pulse rate does not affect modulation detection thresholds.Comment: 1 title page, 27 manuscript pages, 14 figures, 1 table, 1 appendi

Spectral Grouping of Electrically Encoded Sound Predicts Speech-in-Noise Performance in Cochlear Implantees

\ua9 2023, The Author(s). Objectives: Cochlear implant (CI) users exhibit large variability in understanding speech in noise. Past work in CI users found that spectral and temporal resolution correlates with speech-in-noise ability, but a large portion of variance remains unexplained. Recent work on normal-hearing listeners showed that the ability to group temporally and spectrally coherent tones in a complex auditory scene predicts speech-in-noise ability independently of the audiogram, highlighting a central mechanism for auditory scene analysis that contributes to speech-in-noise. The current study examined whether the auditory grouping ability also contributes to speech-in-noise understanding in CI users. Design: Forty-seven post-lingually deafened CI users were tested with psychophysical measures of spectral and temporal resolution, a stochastic figure-ground task that depends on the detection of a figure by grouping multiple fixed frequency elements against a random background, and a sentence-in-noise measure. Multiple linear regression was used to predict sentence-in-noise performance from the other tasks. Results: No co-linearity was found between any predictor variables. All three predictors (spectral and temporal resolution plus the figure-ground task) exhibited significant contribution in the multiple linear regression model, indicating that the auditory grouping ability in a complex auditory scene explains a further proportion of variance in CI users’ speech-in-noise performance that was not explained by spectral and temporal resolution. Conclusion: Measures of cross-frequency grouping reflect an auditory cognitive mechanism that determines speech-in-noise understanding independently of cochlear function. Such measures are easily implemented clinically as predictors of CI success and suggest potential strategies for rehabilitation based on training with non-speech stimuli

The temporal pattern of impulses in primary afferents analogously encodes touch and hearing information

An open question in neuroscience is the contribution of temporal relations between individual impulses in primary afferents in conveying sensory information. We investigated this question in touch and hearing, while looking for any shared coding scheme. In both systems, we artificially induced temporally diverse afferent impulse trains and probed the evoked perceptions in human subjects using psychophysical techniques. First, we investigated whether the temporal structure of a fixed number of impulses conveys information about the magnitude of tactile intensity. We found that clustering the impulses into periodic bursts elicited graded increases of intensity as a function of burst impulse count, even though fewer afferents were recruited throughout the longer bursts. The interval between successive bursts of peripheral neural activity (the burst-gap) has been demonstrated in our lab to be the most prominent temporal feature for coding skin vibration frequency, as opposed to either spike rate or periodicity. Given the similarities between tactile and auditory systems, second, we explored the auditory system for an equivalent neural coding strategy. By using brief acoustic pulses, we showed that the burst-gap is a shared temporal code for pitch perception between the modalities. Following this evidence of parallels in temporal frequency processing, we next assessed the perceptual frequency equivalence between the two modalities using auditory and tactile pulse stimuli of simple and complex temporal features in cross-sensory frequency discrimination experiments. Identical temporal stimulation patterns in tactile and auditory afferents produced equivalent perceived frequencies, suggesting an analogous temporal frequency computation mechanism. The new insights into encoding tactile intensity through clustering of fixed charge electric pulses into bursts suggest a novel approach to convey varying contact forces to neural interface users, requiring no modulation of either stimulation current or base pulse frequency. Increasing control of the temporal patterning of pulses in cochlear implant users might improve pitch perception and speech comprehension. The perceptual correspondence between touch and hearing not only suggests the possibility of establishing cross-modal comparison standards for robust psychophysical investigations, but also supports the plausibility of cross-sensory substitution devices

A Robust Encoding Scheme for Delivering Artificial Sensory Information via Direct Brain Stimulation

Innovations for creating somatosensation via direct electrical stimulation of the brain will be required for the next generation of bi-directional cortical neuroprostheses. The current lack of tactile perception and proprioceptive input likely imposes a fundamental limit on speed and accuracy of brain-controlled prostheses or re-animated limbs. This study addresses the unique challenge of identifying a robust, high bandwidth sensory encoding scheme in a high-dimensional parameter space. Previous studies demonstrated single dimensional encoding schemes delivering low bandwidth sensory information, but no comparison has been performed across parameters, nor with update rates suitable for real-time operation of a neuroprosthesis. Here, we report the first comprehensive measurement of the resolution of key stimulation parameters such as pulse amplitude, pulse width, frequency, train interval and number of pulses. Surprisingly, modulation of stimulation frequency was largely undetectable. While we initially expected high frequency content to be an ideal candidate for passing high throughput sensory signals to the brain, we found only modulation of very low frequencies were detectable. Instead, the charge-per-phase of each pulse yields the highest resolution sensory signal, and is the key parameter modulating perceived intensity. The stimulation encoding patterns were designed for high-bandwidth information transfer that will be required for bi-directional brain interfaces. Our discovery of the stimulation features which best encode perceived intensity have significant implications for design of any neural interface seeking to convey information directly to the brain via electrical stimulation

A Robust Encoding Scheme for Delivering Artificial Sensory Information via Direct Brain Stimulation

Innovations for creating somatosensation via direct electrical stimulation of the brain will be required for the next generation of bi-directional cortical neuroprostheses. The current lack of tactile perception and proprioceptive input likely imposes a fundamental limit on speed and accuracy of brain-controlled prostheses or re-animated limbs. This study addresses the unique challenge of identifying a robust, high bandwidth sensory encoding scheme in a high-dimensional parameter space. Previous studies demonstrated single dimensional encoding schemes delivering low bandwidth sensory information, but no comparison has been performed across parameters, nor with update rates suitable for real-time operation of a neuroprosthesis. Here, we report the first comprehensive measurement of the resolution of key stimulation parameters such as pulse amplitude, pulse width, frequency, train interval and number of pulses. Surprisingly, modulation of stimulation frequency was largely undetectable. While we initially expected high frequency content to be an ideal candidate for passing high throughput sensory signals to the brain, we found only modulation of very low frequencies were detectable. Instead, the charge-per-phase of each pulse yields the highest resolution sensory signal, and is the key parameter modulating perceived intensity. The stimulation encoding patterns were designed for high-bandwidth information transfer that will be required for bi-directional brain interfaces. Our discovery of the stimulation features which best encode perceived intensity have significant implications for design of any neural interface seeking to convey information directly to the brain via electrical stimulation

Minimum-error, energy-constrained source coding by sensory neurons

Neural coding, the process by which neurons represent, transmit, and manipulate physical signals, is critical to the function of the nervous system. Despite years of study, neural coding is still not fully understood. Efforts to model neural coding could improve both the understanding of the nervous system and the design of artificial devices which interact with neurons. Sensory receptors and neurons transduce physical signals into a sequence of action potentials, called a spike train. The principles which underly the translation from signal to spike train are still under investigation. From the perspective of an organism, neural codes which maximize the fidelity of the encoded signal (minimize encoding error), provide a competitive advantage. Selective pressure over evolutionary timescales has likely encouraged neural codes which minimize encoding error. At the same time, neural coding is metabolically expensive, which suggests that selective pressure would also encourage neural codes which minimize energy. Based on these assumptions, this work proposes a principle of neural coding which captures the trade-off between error and energy as a constrained optimization problem of minimizing encoding error while satisfying a constraint on energy. A solution to the proposed optimization problem is derived in the limit of high spike-rates. The solution is to track the instantaneous reconstruction error, and to time spikes when the error crosses a threshold value. In the limit of large signals, the threshold level is a constant, but in general it is signal dependent. This coding model, called the neural source coder, implies neurons should be able to track reconstruction error internally, using the error signal to precisely time spikes. Mathematically, this model is similar to existing adaptive threshold models, but it provides a new way to understand coding by sensory neurons. Comparing the predictions of the neural source coder to experimental data recorded from a peripheral neuron, the coder is able to predict spike times with considerable accuracy. Intriguingly, this is also true for a cortical neuron which has a low spike-rate. Reconstructions using the neural source coder show lower error than other spiking neuron models. The neural source coder also predicts the asymmetric spike-rate adaptation seen in sensory neurons (the primary-like response). An alternative expression for the neural source coder is as an instantaneous-rate coder of a rate function which depends on the signal, signal derivative, and encoding parameters. The instantaneous rate closely predicts experimental peri-stimulus time histograms. The addition of a stochastic threshold to the neural source coder accounts for the spike-time jitter observed in experimental datasets. Jittered spike-trains from the neural source coder show long-term interval statistics which closely match experimental recordings from a peripheral neuron. Moreover, the spike trains have strongly anti-correlated intervals, a feature observed in experimental data. Interestingly, jittered spike-trains do not improve reconstruction error for an individual neuron, but reconstruction error is reduced in simulations of small populations of independent neurons. This suggests that jittered spike-trains provide a method for small populations of sensory neurons to improve encoding error. Finally, a sound coding method for applying the neural source coder to timing spikes for cochlear implants is proposed. For each channel of the cochlear implant, a neural source coder can be used to time pulses to follow the patterns expected by peripheral neurons. Simulations show reduced reconstruction error compared to standard approaches using the signal envelope. Initial experiments with normal-hearing subjects show that a vocoder simulating this cochlear implant sound coding approach results in better speech perception thresholds when compared to a standard noise vocoder. Although further experiments with cochlear implant users are critical, initial results encourage further study of the proposed sound-coding method. Overall, the proposed principle of minimum-error, energy-constrained encoding for sensory neural coding can be implemented by a spike-timing model with a feedback loop which computes reconstruction error. This model of neural source coding predicts a wide range of experimental observations from both peripheral and cortical neurons. The close agreement between experimental data and the predictions of the neural source coder suggests a fundamental principle underlying neural coding

Sensory Communication

Contains table of contents for Section 2 and reports on five research projects.National Institutes of Health Contract 2 R01 DC00117National Institutes of Health Contract 1 R01 DC02032National Institutes of Health Contract 2 P01 DC00361National Institutes of Health Contract N01 DC22402National Institutes of Health Grant R01-DC001001National Institutes of Health Grant R01-DC00270National Institutes of Health Grant 5 R01 DC00126National Institutes of Health Grant R29-DC00625U.S. Navy - Office of Naval Research Grant N00014-88-K-0604U.S. Navy - Office of Naval Research Grant N00014-91-J-1454U.S. Navy - Office of Naval Research Grant N00014-92-J-1814U.S. Navy - Naval Air Warfare Center Training Systems Division Contract N61339-94-C-0087U.S. Navy - Naval Air Warfare Center Training System Division Contract N61339-93-C-0055U.S. Navy - Office of Naval Research Grant N00014-93-1-1198National Aeronautics and Space Administration/Ames Research Center Grant NCC 2-77

Toward More Versatile and Intuitive Cortical Brain–Machine Interfaces

Brain–machine interfaces have great potential for the development of neuroprosthetic applications to assist patients suffering from brain injury or neurodegenerative disease. One type of brain–machine interface is a cortical motor prosthetic, which is used to assist paralyzed subjects. Motor prosthetics to date have typically used the motor cortex as a source of neural signals for controlling external devices. The review will focus on several new topics in the arena of cortical prosthetics. These include using: recordings from cortical areas outside motor cortex; local field potentials as a source of recorded signals; somatosensory feedback for more dexterous control of robotics; and new decoding methods that work in concert to form an ecology of decode algorithms. These new advances promise to greatly accelerate the applicability and ease of operation of motor prosthetics

Doctor of Philosophy

dissertationThe primate auditory system is responsible for analyzing complex patterns of pressure differences and then synthesizing this information into a behaviorally relevant representation of the external world. How the auditory cortex accomplishes this complex task is unknown. This thesis examines the neural mechanisms underlying auditory perception in the primate auditory cortex, focusing on the neural representation of communication sounds. This thesis is composed of three studies of auditory cortical processing in the macaque and human. The first examines coding in primary and tertiary auditory cortex as it relates to the possibility for developing a stimulating auditory neural prosthesis. The second study applies an information theoretic approach to understanding information transfer between primary and tertiary auditory cortex. The final study examines visual influences on human tertiary auditory cortical processing during illusory audiovisual speech perception. Together, these studies provide insight into the cortical physiology underlying sound perception and insight into the creation of a stimulating cortical neural prosthesis for the deaf

Improvement of Speech Perception for Hearing-Impaired Listeners

Hearing impairment is becoming a prevalent health problem affecting 5% of world adult populations. Hearing aids and cochlear implant already play an essential role in helping patients over decades, but there are still several open problems that prevent them from providing the maximum benefits. Financial and discomfort reasons lead to only one of four patients choose to use hearing aids; Cochlear implant users always have trouble in understanding speech in a noisy environment. In this dissertation, we addressed the hearing aids limitations by proposing a new hearing aid signal processing system named Open-source Self-fitting Hearing Aids System (OS SF hearing aids). The proposed hearing aids system adopted the state-of-art digital signal processing technologies, combined with accurate hearing assessment and machine learning based self-fitting algorithm to further improve the speech perception and comfort for hearing aids users. Informal testing with hearing-impaired listeners showed that the testing results from the proposed system had less than 10 dB (by average) difference when compared with those results obtained from clinical audiometer. In addition, Sixteen-channel filter banks with adaptive differential microphone array provides up to six-dB SNR improvement in the noisy environment. Machine-learning based self-fitting algorithm provides more suitable hearing aids settings. To maximize cochlear implant users’ speech understanding in noise, the sequential (S) and parallel (P) coding strategies were proposed by integrating high-rate desynchronized pulse trains (DPT) in the continuous interleaved sampling (CIS) strategy. Ten participants with severe hearing loss participated in the two rounds cochlear implants testing. The testing results showed CIS-DPT-S strategy significantly improved (11%) the speech perception in background noise, while the CIS-DPT-P strategy had a significant improvement in both quiet (7%) and noisy (9%) environment