Music pre-processing for cochlear implants

Abstract

A Cochlear Implant (CI) is a medical device that enables profoundly hearing impaired people to perceive sounds by electrically stimulating the auditory nerve using an electrode array implanted in the cochlea. The focus of most research on signal processing for CIs has been on strategies to improve speech understanding in quiet and in background noise, since the main aim for implanting a CInbsp;(and still is) to restore the ability to communicate. Most CI users perform quite well in terms of speech understanding. On the other hand, music perception and appreciation are generally very poor. The main goal of this PhD project was to investigate and to improve the poor music enjoyment in CI users. An initial experiment with multi-track recordings was carried out to examine the music mixing preferences for different instruments in polyphonic or complex music. In general, a preference for clear vocals and attenuated instruments was observed with preservation of bass and drums. Based on this knowledge, a music pre-processing scheme for mono and stereo recordings wasnbsp;which is capable of balancing vocals/bass/drums against the other instruments. The scheme is based on thenbsp;of harmonic and percussive components in the spectrogram and on the spatial information of the instruments in typical stereo recordings. Subsequently, the music pre-processing scheme was evaluated in a take-home experiment with post-lingually deafened CI users andnbsp;genres of music, providing encouraging results for building a tool for music training or rehabilitation programs.Acknowledgement 3 Abstract 6 Abstract (Dutch translation) 8 List of acronyms and abbreviations 10 Table of Contents 15 List of figures 19 List of tables 23 Chapter 1 : Introduction 25 1.1 The human auditory system 25 1.1.1 Anatomy of the human ear 25 1.1.2 Hearing loss 27 1.2 Cochlear implants 28 1.2.1 The CI system 28 1.2.2 Sound processing in CI 30 1.2.3 Performance of CI users 35 1.3 Cochlear implants and music 36 1.3.1 Music perception with CI 37 1.3.2 Music appreciation with CI 41 1.4 The music signal 42 1.4.1 Music complexity 43 1.4.2 Sound source separation techniques 43 1.5 Motivation 45 1.6 Objectives and outline 46 Chapter 2 : Music mixing preferences of cochlear implant recipients: a pilot study 51 2.1 Abstract 51 2.2 Introduction 52 2.3 Methods 56 2.3.1 Sound material 56 2.3.2 Subjects 58 2.3.3 Experiment 1 60 2.3.4 Experiment 2 61 2.3.5 Statistics 64 2.4 Results 65 2.4.1 Experiment 1 65 2.4.2 Experiment 2 68 2.5 Discussion 74 2.6 Conclusion 78 Chapter 3 : A Harmonic/Percussive Sound Separation based Music Pre-Processing Scheme for Cochlear Implant Users 79 3.1 Abstract 79 3.2 Introduction 80 3.3 Music pre-processing scheme 82 3.4 Objective testing 86 3.5 Subjective testing 90 3.6 Conclusion 92 Chapter 4 : A stereo music pre-processing scheme for cochlear implant users 93 4.1 Abstract 93 4.2 Introduction 94 4.3 Stereo music pre-processing scheme 98 4.3.1 Vocals & Drums Extraction 99 4.3.2 Bass frequency extraction 107 4.3.3 Stereo binary mask 108 4.3.4 Stereo music pre-processing output 113 4.4 Methods 113 4.4.1 Sound material 114 4.4.2 Subjects 115 4.4.3 Perceptual evaluation 116 4.5 Results 117 4.6 Discussion 119 4.7 Conclusion 123 Chapter 5 : Evaluation of stereo music pre-processing for cochlear implant users 125 5.1 Abstract 125 5.2 Introduction 126 5.3 Methods 129 5.3.1 Sound material 129 5.3.2 Subjects 130 5.3.3 Take-home device 131 5.3.4 Test procedure 133 5.4 Results 135 5.5 Discussion 140 5.6 Conclusion 143 Chapter 6 : Conclusions and future work 145 6.1 Music mixing preference experiment 146 6.2 Music pre-processing scheme 147 6.3 Take-home evaluation 148 6.4 Future work 149 6.4.1 Music feature preference 150 6.4.2 Advanced music pre-processing 151 6.4.3 Fitting a music map 152 Appendix 153 Bibliography 171 Curriculum Vitae 184 List of publications 185nrpages: 187status: publishe