Neuromorphic audio processing through real-time embedded spiking neural networks.

In this work novel speech recognition and audio processing systems based on a spiking artificial cochlea and neural networks are proposed and implemented. First, the biological behavior of the animal’s auditory system is analyzed and studied, along with the classical mechanisms of audio signal processing for sound classification, including Deep Learning techniques. Based on these studies, novel audio processing and automatic audio signal recognition systems are proposed, using a bio-inspired auditory sensor as input. A desktop software tool called NAVIS (Neuromorphic Auditory VIsualizer) for post-processing the information obtained from spiking cochleae was implemented, allowing to analyze these data for further research. Next, using a 4-chip SpiNNaker hardware platform and Spiking Neural Networks, a system is proposed for classifying different time-independent audio signals, making use of a Neuromorphic Auditory Sensor and frequency studies obtained with NAVIS. To prove the robustness and analyze the limitations of the system, the input audios were disturbed, simulating extreme noisy environments. Deep Learning mechanisms, particularly Convolutional Neural Networks, are trained and used to differentiate between healthy persons and pathological patients by detecting murmurs from heart recordings after integrating the spike information from the signals using a neuromorphic auditory sensor. Finally, a similar approach is used to train Spiking Convolutional Neural Networks for speech recognition tasks. A novel SCNN architecture for timedependent signals classification is proposed, using a buffered layer that adapts the information from a real-time input domain to a static domain. The system was deployed on a 48-chip SpiNNaker platform. Finally, the performance and efficiency of these systems were evaluated, obtaining conclusions and proposing improvements for future works.Premio Extraordinario de Doctorado U

Artificial intelligence in studying and evaluation of otitis media by acoustic reflectometry

Abstract. Objective: Acute otitis media (AOM) is usually associated with upper respiratory tract infections and common colds, but many times it can last longer than the initial symptoms. An acoustic reflectometry device can be used to objectify the diagnostic process. The purpose of the study was to train the neural network to identify ears with symptoms of AOM using the acoustic response of the device. Methods: An acoustic reflectometry sample of 53 ears from 39 patients was collected during laryngoscopy operation from patients with recurrent ear infections. In addition to the acoustic samples, the doctor determined whether ear had visual signs of otitis media (OM) and whether there was effusion in it. These three parameters were used in the construction of feedforward neural network. Two neural network layouts were selected, one with samples of effusion-only sick ears and the other with sick ears based on other visual indications of OM, independent of effusion. Results: The sensitivity and specificity of the trained networks were about 90%. Two different groupings of samples clearly showed that diseased ears without effusion could be identified as sick with sensitivity of 80–90%, when similar ears were included in the category of sick ears. Network with sick ears with effusion as training material had a sensitivity of 20–30% identifying sick ears without effusion. The inclusion of both types of sick ears in single network caused slight drop in sensitivity and specificity compared to just one type. Conclusion: Acoustic reflectometry can detect more than just standard cases of acute otitis media, in which effusion typically occurs. An accurate neural network for identifying sick ears without effusion can be achieved with a relatively small sample size. This indicates a possibility of conducting an in-depth analysis of other diseases within the OM group or the transition between these diseases.Tekoäly otitis median tutkimisessa ja arvioimisessa akustisella reflektometrialla. Tiivistelmä. Työtarkoitus: Akuutti otitis media (AOM) yhdistetään yleensä ylempien hengitysteiden tulehduksiin ja nuhaan, mutta monesti otitis media (OM) oireet voivat kestää tulehdustilaa tai nuhaa pidempään. Akustista reflektometriaa käyttävän laitteen avulla diagnosointi prosessia voidaan tarkastella objektiivisesti. Työn tarkoitus oli opettaa neuroverkko, mikä tunnistaa AOM-oireisen korvan akustisen reflektometrin akustisesta mittauksesta. Menetelmät: Laryngoskopiaoperaation aikana kerättiin akustisella reflektometrialla otos 53 korvasta. Operaatio suoritettiin 39 potilaalle, joilla oli uusiutuvia korvatulehduksia. Akustisten näytteiden lisäksi operaation aikana lääkäri määritti visuaaliset OM-merkit ja eritteen määrän. Näitä kolmea tietoa käytettiin eteenpäin kytkeytyvän neuroverkon rakentamiseen. Kaksi neuroverkkoa rakennettiin, joissa ensimmäisessä oli pelkästään eritettä sisältävät korvat, ja toisessa kaikki visuaalisesti OM-merkit täyttävät korvat, mukaan lukien eritettä sisältävät korvat. Tulokset: Opetettujen neuroverkkojen sensitiivisyys ja spesifisyys olivat 90 % luokkaa. Kahteen ryhmään jaettu aineisto osoitti, että sairaat eritteettömät korvat voidaan tunnistaa sairaiksi 80–90 % sensitiivisyydellä, kun neuroverkolle opetetaan sekä eritteiset että eritteettömät sairaat korvat. Pelkästään eritteisiä korvia sisältävä neuroverkko tunnisti eritteettömät sairaat korvat 20–30 % sensitiivisyydellä. Eritteisten ja eritteettömien korvien käyttö samassa neuroverkossa laski sensitiivisyyttä ja spesifisyyttä verrattuna pelkkien eritteisten käyttöön. Johtopäätökset: Akustinen reflektometria voi tunnistaa muitakin tiloja kuin tyypillisen eritteisen akuutin otitis median. Pienellä näytemäärällä voidaan saavuttaa tarkka neuroverkko, mikä tunnistaa otitis median ilman eritteen läsnäoloa. Tämä viittaa mahdollisuuteen, että syvällisellä analyysillä voidaan saada lisää tietoa taudin etenemisestä tai taudin muista tiloista

Ultrafast and Ultralight Network-Based Intelligent System for Real-time Diagnosis of Ear diseases in Any Devices

Traditional ear disease diagnosis heavily depends on experienced specialists and specialized equipment, frequently resulting in misdiagnoses, treatment delays, and financial burdens for some patients. Utilizing deep learning models for efficient ear disease diagnosis has proven effective and affordable. However, existing research overlooked model inference speed and parameter size required for deployment. To tackle these challenges, we constructed a large-scale dataset comprising eight ear disease categories and normal ear canal samples from two hospitals. Inspired by ShuffleNetV2, we developed Best-EarNet, an ultrafast and ultralight network enabling real-time ear disease diagnosis. Best-EarNet incorporates the novel Local-Global Spatial Feature Fusion Module which can capture global and local spatial information simultaneously and guide the network to focus on crucial regions within feature maps at various levels, mitigating low accuracy issues. Moreover, our network uses multiple auxiliary classification heads for efficient parameter optimization. With 0.77M parameters, Best-EarNet achieves an average frames per second of 80 on CPU. Employing transfer learning and five-fold cross-validation with 22,581 images from Hospital-1, the model achieves an impressive 95.23% accuracy. External testing on 1,652 images from Hospital-2 validates its performance, yielding 92.14% accuracy. Compared to state-of-the-art networks, Best-EarNet establishes a new state-of-the-art (SOTA) in practical applications. Most importantly, we developed an intelligent diagnosis system called Ear Keeper, which can be deployed on common electronic devices. By manipulating a compact electronic otoscope, users can perform comprehensive scanning and diagnosis of the ear canal using real-time video. This study provides a novel paradigm for ear endoscopy and other medical endoscopic image recognition applications.Comment: This manuscript has been submitted to Neural Network

Fast frequency discrimination and phoneme recognition using a biomimetic membrane coupled to a neural network

In the human ear, the basilar membrane plays a central role in sound recognition. When excited by sound, this membrane responds with a frequency-dependent displacement pattern that is detected and identified by the auditory hair cells combined with the human neural system. Inspired by this structure, we designed and fabricated an artificial membrane that produces a spatial displacement pattern in response to an audible signal, which we used to train a convolutional neural network (CNN). When trained with single frequency tones, this system can unambiguously distinguish tones closely spaced in frequency. When instead trained to recognize spoken vowels, this system outperforms existing methods for phoneme recognition, including the discrete Fourier transform (DFT), zoom FFT and chirp z-transform, especially when tested in short time windows. This sound recognition scheme therefore promises significant benefits in fast and accurate sound identification compared to existing methods.Comment: 7 pages, 4 figure

Automated diagnosis of ear disease using ensemble deep learning with a big otoendoscopy image database

BACKGROUND: Ear and mastoid disease can easily be treated by early detection and appropriate medical care. However, short of specialists and relatively low diagnostic accuracy calls for a new way of diagnostic strategy, in which deep learning may play a significant role. The current study presents a machine learning model to automatically diagnose ear disease using a large database of otoendoscopic images acquired in the clinical environment. METHODS: Total 10,544 otoendoscopic images were used to train nine public convolution-based deep neural networks to classify eardrum and external auditory canal features into six categories of ear diseases, covering most ear diseases (Normal, Attic retraction, Tympanic perforation, Otitis externa±myringitis, Tumor). After evaluating several optimization schemes, two best-performing models were selected to compose an ensemble classifier, by combining classification scores of each classifier. FINDINGS: According to accuracy and training time, transfer learning models based on Inception-V3 and ResNet101 were chosen and the ensemble classifier using the two models yielded a significant improvement over each model, the accuracy of which is in average 93·67% for the 5-folds cross-validation. Considering substantial data-size dependency of classifier performance in the transfer learning, evaluated in this study, the high accuracy in the current model is attributable to the large database. INTERPRETATION: The current study is unprecedented in terms of both disease diversity and diagnostic accuracy, which is compatible or even better than an average otolaryngologist. The classifier was trained with data in a various acquisition condition, which is suitable for the practical environment. This study shows the usefulness of utilizing a deep learning model in the early detection and treatment of ear disease in the clinical situation. FUND: This research was supported by Brain Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT(NRF-2017M3C7A1049051).ope

New Approaches and Technologies to Improve Accuracy of Acute Otitis Media Diagnosis

: Several studies have shown that in recent years incidence of acute otitis media (AOM) has declined worldwide. However, related medical, social, and economic problems for patients, their families, and society remain very high. Better knowledge of potential risk factors for AOM development and more effective preventive interventions, particularly in AOM-prone children, can further reduce disease incidence. However, a more accurate AOM diagnosis seems essential to achieve this goal. Diagnostic uncertainty is common, and to avoid risks related to a disease caused mainly by bacteria, several children without AOM are treated with antibiotics and followed as true AOM cases. The main objective of this manuscript is to discuss the most common difficulties that presently limit accurate AOM diagnosis and the new approaches and technologies that have been proposed to improve disease detection. We showed that misdiagnosis can be dangerous or lead to relevant therapeutic mistakes. The need to improve AOM diagnosis has allowed the identification of a long list of technologies to visualize and evaluate the tympanic membrane and to assess middle-ear effusion. Most of the new instruments, including light field otoscopy, optical coherence tomography, low-coherence interferometry, and Raman spectroscopy, are far from being introduced in clinical practice. Video-otoscopy can be effective, especially when it is used in association with telemedicine, parents' cooperation, and artificial intelligence. Introduction of otologic telemedicine and use of artificial intelligence among pediatricians and ENT specialists must be strongly promoted in order to reduce mistakes in AOM diagnosis

A Fluidic Soft Robot for Needle Guidance and Motion Compensation in Intratympanic Steroid Injections

Intratympanic steroid injections are commonly employed in treating ear diseases, such as sudden sensorineural hearing loss or Meniere's disease through drug delivery via the middle ear. Whilst being an effective treatment, the procedure has to be performed by a trained surgeon to avoid delicate regions in the patient's anatomy and is considered painful despite the use of topical anaesthesia. In this letter we introduce a fluid-driven soft robotic system which aims at increasing patient-comfort during the injection by counteracting unwanted needle motion, reducing the cognitive load of the clinician by autonomously identifying sensitive regions in the ear and de-risking the procedure by steering the needle towards the desired injection site. A design comprising of six embedded fluidic actuators is presented, which allow for translation and rotation of the needle as well as adaptive stiffening in the coupling between needle and ear canal. The system's steering-capabilities are investigated and the differential kinematics derived to demonstrate trajectory tracking in Cartesian space. A vision system is developed which enables tracking of anatomical landmarks on the tympanic membrane and thus locating the desired needle insertion site. The integrated system shows the ability to provide a safe guide for the inserted needle towards a desired target direction while significantly reducing needle motion. The proposed tracking algorithm is able to identify the desired needle insertion site and could be employed to avoid delicate anatomical regions