Active hearing mechanisms inspire adaptive amplification in an acoustic sensor system

Over many millions of years of evolution, nature has developed some of the most adaptable sensors and sensory systems possible, capable of sensing, conditioning and processing signals in a very power- and size-effective manner. By looking into biological sensors and systems as a source of inspiration, this paper presents the study of a bio-inspired concept of signal processing at the sensor level. By exploiting a feedback control mechanism between a front-end acoustic receiver and back-end neuronal based computation, a nonlinear amplification with hysteretic behavior is created. Moreover, the transient response of the front-end acoustic receiver can also be controlled and enhanced. A theoretical model is proposed and the concept is prototyped experimentally through an embedded system setup that can provide dynamic adaptations of a sensory system comprising a MEMS microphone placed in a closed-loop feedback system. It faithfully mimics the mosquito’s active hearing response as a function of the input sound intensity. This is an adaptive acoustic sensor system concept that can be exploit by sensor and system designers within acoustics and ultrasonic engineering fields

Bio-inspired active amplification in a MEMS microphone using feedback computation

Auditory signal processing relies on feedback mechanisms between mechanical and electrical systems that work together to enhance acoustic conditioning. In this paper a nonlinear amplification mechanism in the mosquito's auditory system is exploited as a model of inspiration. An acoustic system that provides active amplification of sound was developed using feedback computation integrated with a MEMS microphone to implement the concept. Experimental results generated by a purpose-built embedded system show signal amplification and hysteresis which replicate the response shown by the biological mosquito’s hearing system as a function of input sound intensity

Enhancing acoustic sensory responsiveness by exploiting bio-inspired feedback computation

Engineering acoustic sensors and systems that can be sensitive to small sound levels even when immersed by background noise may require out-of-the-box thinking. Biology can provide inspiration for that, allowing the engineering landscape to borrow interesting ideas and thus solve current human problems. Biological sensor and system designs are a result of many million years of evolutionary processes, which make them very-power efficient and well-adapted to perform their function in a living organism. This paper presents a theoretical study of a bio-inspired signal processing concept. The assumption is that by exploiting feedback computation between a front-end acoustic detector and a back-end neuronal based processing, the overall acoustic responsiveness of a sensory system can be controlled and enhanced to target signals of interest. Here, two methods of feedback signal entrainment are compared namely 1:1 and 2:1 resonance modes

Review of the applications of principles of insect hearing to microscale acoustic engineering challenges

When looking for novel, simple, and energy-efficient solutions to engineering problems, nature has proved to be an incredibly valuable source of inspiration. The development of acoustic sensors has been a prolific field for bioinspired solutions. With a diverse array of evolutionary approaches to the problem of hearing at small scales (some widely different to the traditional concept of "ear"), insects in particular have served as a starting point for several designs. From locusts to moths, through crickets and mosquitoes among many others, the mechanisms found in nature to deal with small-scale acoustic detection and the engineering solutions they have inspired are reviewed. The present article is comprised of three main sections corresponding to the principal problems faced by insects, namely frequency discrimination, which is addressed by tonotopy, whether performed by a specific organ or directly on the tympana; directionality, with solutions including diverse adaptations to tympanal structure; and detection of weak signals, through what is known as active hearing. The three aforementioned problems concern tiny animals as much as human-manufactured microphones and have therefore been widely investigated. Even though bioinspired systems may not always provide perfect performance, they are sure to give us solutions with clever use of resources and minimal post-processing, being serious contenders for the best alternative depending on the requisites of the problem

Decoding auditory attention and neural language processing in adverse conditions and different listener groups

This thesis investigated subjective, behavioural and neurophysiological (EEG) measures of speech processing in various adverse conditions and with different listener groups. In particular, this thesis focused on different neural processing stages and their relationship with auditory attention, effort, and measures of speech intelligibility. Study 1 set the groundwork by establishing a toolbox of various neural measures to investigate online speech processing, from the frequency following response (FFR) and cortical measures of speech processing, to the N400, a measure of lexico-semantic processing. Results showed that peripheral processing is heavily influenced by stimulus characteristics such as degradation, whereas central processing units are more closely linked to higher-order phenomena such as speech intelligibility. In Study 2, a similar experimental paradigm was used to investigate differences in neural processing between a hearing-impaired and a normal-hearing group. Subjects were presented with short stories in different levels of multi-talker babble noise, and with different settings on their hearing aids. Findings indicate that, particularly at lower noise levels, the hearing-impaired group showed much higher cortical entrainment than the normal- hearing group, despite similar levels of speech recognition. Intersubject correlation, another global neural measure of auditory attention, however, was similarly affected by noise levels in both the hearing-impaired and the normal-hearing group. This finding indicates extra processing in the hearing-impaired group only on the level of the auditory cortex. Study 3, in contrast to Studies 1 and 2 (which both investigated the effects of bottom-up factors on neural processing), examined the links between entrainment and top-down factors, specifically motivation; as well as reasons for the 5 higher entrainment found in hearing-impaired subjects in Study 2. Results indicated that, while behaviourally there was no difference between incentive and non-incentive conditions, neurophysiological measures of attention such as intersubject correlation were affected by the presence of an incentive to perform better. Moreover, using a specific degradation type resulted in subjects’ increased cortical entrainment under degraded conditions. These findings support the hypothesis that top-down factors such as motivation influence neurophysiological measures; and that higher entrainment to degraded speech might be triggered specifically by the reduced availability of spectral detail contained in speech

Biophysical modeling of a cochlear implant system: progress on closed-loop design using a novel patient-specific evaluation platform

The modern cochlear implant is one of the most successful neural stimulation devices, which partially mimics the workings of the auditory periphery. In the last few decades it has created a paradigm shift in hearing restoration of the deaf population, which has led to more than 324,000 cochlear implant users today. Despite its great success there is great disparity in patient outcomes without clear understanding of the aetiology of this variance in implant performance. Furthermore speech recognition in adverse conditions or music appreciation is still not attainable with today's commercial technology. This motivates the research for the next generation of cochlear implants that takes advantage of recent developments in electronics, neuroscience, nanotechnology, micro-mechanics, polymer chemistry and molecular biology to deliver high fidelity sound. The main difficulties in determining the root of the problem in the cases where the cochlear implant does not perform well are two fold: first there is not a clear paradigm on how the electrical stimulation is perceived as sound by the brain, and second there is limited understanding on the plasticity effects, or learning, of the brain in response to electrical stimulation. These significant knowledge limitations impede the design of novel cochlear implant technologies, as the technical specifications that can lead to better performing implants remain undefined. The motivation of the work presented in this thesis is to compare and contrast the cochlear implant neural stimulation with the operation of the physiological healthy auditory periphery up to the level of the auditory nerve. As such design of novel cochlear implant systems can become feasible by gaining insight on the question `how well does a specific cochlear implant system approximate the healthy auditory periphery?' circumventing the necessity of complete understanding of the brain's comprehension of patterned electrical stimulation delivered from a generic cochlear implant device. A computational model, termed Digital Cochlea Stimulation and Evaluation Tool (‘DiCoStET’) has been developed to provide an objective estimate of cochlear implant performance based on neuronal activation measures, such as vector strength and average activation. A patient-specific cochlea 3D geometry is generated using a model derived by a single anatomical measurement from a patient, using non-invasive high resolution computed tomography (HRCT), and anatomically invariant human metrics and relations. Human measurements of the neuron route within the inner ear enable an innervation pattern to be modelled which joins the space from the organ of Corti to the spiral ganglion subsequently descending into the auditory nerve bundle. An electrode is inserted in the cochlea at a depth that is determined by the user of the tool. The geometric relation between the stimulation sites on the electrode and the spiral ganglion are used to estimate an activating function that will be unique for the specific patient's cochlear shape and electrode placement. This `transfer function', so to speak, between electrode and spiral ganglion serves as a `digital patient' for validating novel cochlear implant systems. The novel computational tool is intended for use by bioengineers, surgeons, audiologists and neuroscientists alike. In addition to ‘DiCoStET’ a second computational model is presented in this thesis aiming at enhancing the understanding of the physiological mechanisms of hearing, specifically the workings of the auditory synapse. The purpose of this model is to provide insight on the sound encoding mechanisms of the synapse. A hypothetical mechanism is suggested in the release of neurotransmitter vesicles that permits the auditory synapse to encode temporal patterns of sound separately from sound intensity. DiCoStET was used to examine the performance of two different types of filters used for spectral analysis in the cochlear implant system, the Gammatone type filter and the Butterworth type filter. The model outputs suggest that the Gammatone type filter performs better than the Butterworth type filter. Furthermore two stimulation strategies, the Continuous Interleaved Stimulation (CIS) and Asynchronous Interleaved Stimulation (AIS) have been compared. The estimated neuronal stimulation spatiotemporal patterns for each strategy suggest that the overall stimulation pattern is not greatly affected by the temporal sequence change. However the finer detail of neuronal activation is different between the two strategies, and when compared to healthy neuronal activation patterns the conjecture is made that the sequential stimulation of CIS hinders the transmission of sound fine structure information to the brain. The effect of the two models developed is the feasibility of collaborative work emanating from various disciplines; especially electrical engineering, auditory physiology and neuroscience for the development of novel cochlear implant systems. This is achieved by using the concept of a `digital patient' whose artificial neuronal activation is compared to a healthy scenario in a computationally efficient manner to allow practical simulation times.Open Acces

Material, acoustic, and mechanical properties of mosquito and midge antennae

The aim of this PhD is to contribute to the understanding of antennal hearing in insects. To this end, Confocal Laser Scanning Microscopy (CLSM), Finite Element Modelling (FEM) and Laser Doppler Vibrometry (LDV) were employed. The combination of the first two allows, in absence of prior knowledge of the material properties of mosquito or midge antennae, to assess the material structure and to gauge which impact this distribution can have on the antenna's mechanical behaviour in response to sound. It was revealed that, rather than a simple beam of uniform cuticle, the antennae of the insects studied had patterns and distributions of hard and soft ring elements along the length of their antennae. These properties were simulated with FEM and showed that they can strongly influence the resonant frequency of the antennae. Further investigations were done on the vibrational behaviour of these insects with LDV. The males of T. brevipalpis, An. arabiensis and An. gambiae demonstrated strong self-oscillation with high Q-factors. Unlike the mosquitoes, vibrometry of C. riparius showed no relevant self-oscillation of the antenna. Female T. brevipalpis produced self-oscillation but weaker than the male. The three-dimensional pattern of self-oscillation in all investigated mosquito species follows a distorted elliptical path. When stimulated with sound, the self-oscillating antennae exhibited classic nonlinear behaviour such as entrainment and down-modulation. Taken together, this thesis highlights the complex mechanics of acoustic reception in mosquitoes and midges, both mechanically and structurally.The aim of this PhD is to contribute to the understanding of antennal hearing in insects. To this end, Confocal Laser Scanning Microscopy (CLSM), Finite Element Modelling (FEM) and Laser Doppler Vibrometry (LDV) were employed. The combination of the first two allows, in absence of prior knowledge of the material properties of mosquito or midge antennae, to assess the material structure and to gauge which impact this distribution can have on the antenna's mechanical behaviour in response to sound. It was revealed that, rather than a simple beam of uniform cuticle, the antennae of the insects studied had patterns and distributions of hard and soft ring elements along the length of their antennae. These properties were simulated with FEM and showed that they can strongly influence the resonant frequency of the antennae. Further investigations were done on the vibrational behaviour of these insects with LDV. The males of T. brevipalpis, An. arabiensis and An. gambiae demonstrated strong self-oscillation with high Q-factors. Unlike the mosquitoes, vibrometry of C. riparius showed no relevant self-oscillation of the antenna. Female T. brevipalpis produced self-oscillation but weaker than the male. The three-dimensional pattern of self-oscillation in all investigated mosquito species follows a distorted elliptical path. When stimulated with sound, the self-oscillating antennae exhibited classic nonlinear behaviour such as entrainment and down-modulation. Taken together, this thesis highlights the complex mechanics of acoustic reception in mosquitoes and midges, both mechanically and structurally

Understanding and Mimicking the Fly's Directional Hearing: Modeling, Sensor Development, and Experimental Studies

Microphone arrays have been widely used in sound source localization for many applications. In order to locate the sound in a discernible manner, the separation between microphones needs to be greater than a critical distance, which poses a fundamental constraint for the miniaturization of directional microphones. In nature, animal hearing organs are also governed by the size constraint; the smaller the organ size, the smaller the available directional cues for directional hearing. However, with an auditory organ separation of only 520 µm, the fly Ormia ochracea is found to exhibit remarkable ability to pinpoint its host cricket at 5 kHz. The key to this fly's phenomenal directional hearing ability is believed to be the mechanical coupling between the eardrums. This innovative solution can inspire one to find alternative approaches to tackle the challenge of developing miniature directional microphones. The overall goal of this dissertation work is to unravel the underlying physics of the fly ear hearing mechanisms, and to apply this understanding to develop and study novel bio-inspired miniature directional microphones. First, through mechanics and optimization analysis, a fundamental biological conclusion is reached: the fly ear can be viewed as a nature-designed optimal structure that is endowed with the dual optimality characteristic of maximum average directional sensitivity and minimum nonlinearity, at its working frequency of 5 kHz. It is shown that this dual optimality characteristic is only achievable when the right mechanical coupling between the eardrums is used (i.e., proper contributions from both rocking and bending modes are used). More importantly, it is further revealed that the dual optimality characteristic of the fly ear is replicable in a synthetic device, whose structural parameters can be tailored to work at any chosen frequency. Next, a novel bio-inspired directional microphone with mechanically coupled diaphragms is designed to capture the essential dynamics of the fly ear. To study the performance of this design, a novel continuum mechanics model is developed, which features two coupling modules, one for the mechanical coupling of the two diaphragms through a beam and the other for each diaphragm coupled through an air gap. Parametric studies are carried out to explore how the key normalized parameters affect the performance of this directional microphone. Finally, this mechanics model is used to guide the development of a large-scale microphone and a fly-ear sized microphone, both of which are experimentally studied by using a low-coherence fiber optic interferometric detection system. With the large-scale sensor, the importance of using proper contribution from both rocking and bending modes is validated. The fly-ear sized sensor is demonstrated to achieve the dual optimality characteristic at 8 kHz with a ten-fold amplification in the directional sensitivity, which is equivalent to that obtainable from a conventional microphone pair that is ten times larger in size. To best use this sensor for sound source localization, a robotic platform with a control scheme inspired by the fly's localization/lateralization scheme is developed, with which a localization accuracy of better than ±2 degrees (the same as the fly ear) is demonstrated in an indoor lab environment. This dissertation work provides a quantitative and mechanistic explanation for the fly's sound localization ability for the first time, and it provides a framework for the development of fly-ear inspired acoustic sensors that will impact many fronts