The ability to assess detection and discrimination of speech by infants has proved elusive. Dr Iain Jackson and colleagues discuss how new technologies and fresh approaches might offer valuable insight into young infants’ behavioural responses to sound

Hunter, L.

Jackson, I.

Jones, P. R.

Moore, D.

Munro, K.

English

City Research Online

              City, University of London Institutional RepositoryCitation: Jackson, I., Hunter, L., Moore, D., Jones, P. R. ORCID: 0000-0001-7672-8397 and Munro, K. (2018). Facing up to the challenge of behavioural observation in infant hearing assessment. ENT and Audiology News, This is the published version of the paper. This version of the publication may differ from the final published version. Permanent repository link:  http://openaccess.city.ac.uk/21556/Link to published version: Copyright and reuse: City Research Online aims to make research outputs of City, University of London available to a wider audience. Copyright and Moral Rights remain with the author(s) and/or copyright holders. URLs from City Research Online may be freely distributed and linked to.City Research Online:            http://openaccess.city.ac.uk/            publications@city.ac.ukCity Research OnlineAUDIOLOGY FEATUREThe limits of current behavioural assessment in  young infantsFinding out whether an adult can hear something can often be as straightforward as simply asking them. Getting the same information from an infant is considerably less straightforward. However, obtaining information about an infant’s ability to detect and discriminate between sounds is crucial when making management decisions. This is especially important if we are to realise the benefits of newborn screening and early intervention. Objective physiologic tests, such as otoacoustic emissions (OAE) and auditory brainstem response (ABR), have become the most popular assessments used in young infants to assess the peripheral auditory system. Cortical evoked potentials show promise, but require sophisticated technology, and lengthy recording times, making them unsuitable for widespread screening programmes. However, physiologic techniques in current use do not provide information to assess benefit of intervention. Behavioural responses are considered the ‘gold standard’ to assess perception of sound, and provide a measure of functioning of the entire auditory system. That said, obtaining reliable and resource-efficient behavioural responses in young infants is challenging for a number of reasons, including: • Young infants’ behavioural responses are limited to higher sound intensities.• Behavioural responses can be subtle and highly variable (both within and between individuals).• Tests require expert examiners, which makes them expensive, and limits our ability to provide widespread screening.• Manual procedures are slow, and infants habituate rapidly to sounds presented to them, meaning that many become too fatigued to complete the test.The general consensus in clinical practice is that behavioural observations should not (or cannot) be relied upon until the infant reaches a developmental age of seven-to-nine months and head turns to sounds can be elicited and rewarded. Even then, test procedures can be time- and labour-intensive, and scoring is subject to bias. Technology opens the door to novel approachesFortunately, new technologies offer opportunities for cheaper, faster, and more reliable testing. An ever-increasing ability to capture different behaviours allows us to reconsider what responses are available for the tools we use. Outside of audiology, techniques such as eye tracking and pupillometry have been widely adopted in developmental psychology, and have transformed the variety and depth of questions infancy researchers are able to explore. At least two novel approaches have been found to work in efforts to measure hearing and vision efficiently and accurately in babies; (i) maximising engagement with a video display minimises attention lapses, and (ii) adopting a flexible, integrative approach to automation that simulates previous observer-based methods [2]. Elsewhere, video-based analyses now offer other surprising insights, which are often inaccessible to human observers. Recent work has shown, for example, that it is possible to detect subtle movements associated with babies’ breathing, or to detect heart-rate from imperceptible, pixel-level changes in the colour of faces [3].In addition to impressive developments in the measurement Facing up to the challenge of behavioural observation in infant hearing assessmentBY IAIN JACKSON, LISA HUNTER, DAVID MOORE, PETE JONES AND KEVIN MUNROThe ability to assess detection and discrimination of speech by infants has proved elusive. Dr Iain Jackson and colleagues discuss how new technologies and fresh approaches might offer valuable insight into young infants’ behavioural responses to sound. “The ability to capture an increasing range of behaviours allows us to reconsider what responses are available to us for the tools we use.”Figure 1. An illustration of the importance of early intervention; the effect of age at fitting of hearing aids on the language outcomes of children with different levels of hearing loss (Better-ear 4-frequency average). (Adapted with permission from Ching et al [1]; personal correspondence with Harvey Dillon.) ent and audiology news | NOVEMBER/DECEMBER 2018 | VOL 27 NO 5 | www.entandaudiologynews.comAUDIOLOGY FEATUREof responses, modern computation techniques, such as machine learning, increasingly promise to offer insights and predictive value using healthcare data, including in the assessment of infants. For example, in a recent study, Goodfellow and colleagues placed wearable sensors on infants, and recorded their spontaneous movements over the course of a day [4]. Their machine learning algorithms were able to differentiate between the movements of typically developing infants and those at risk for developmental delays, potentially paving the way for a clinically useful tool for predictive assessment. Applying new technologies to infant audiometryAs part of the NIHR Manchester Biomedical Research Centre, we are currently looking to combine automatic facial recognition, head tracking, and eye tracking, along with state-of-the-art machine learning algorithms, in order to be able to detect when an infant hears a sound.Infants excel at sending signals with their faces – all caregivers are acutely aware of how easy it is to recognise when an infant is unhappy! In addition to clear and obvious signals, like crying, there might also be other valuable markers we can capitalise on. Facial recognition software can detect and track a wide range of facial behaviours, from broad, global features such as head-turns, down to more subtle changes in individual features or areas of the face. For example, infants may raise their eyebrows ever so slightly when they hear a sound, or smile, or frown, or let their mouths fall open slightly. Whatever the response might be, if one or more responses occur frequently enough, it will provide a pattern for the algorithm to recognise, and thus provide a signal for researchers to measure. In this approach, we take video recordings of many infants’ faces and capture a range of examples of the spontaneous expressions and movements infants commonly make in the absence of sound. We also take video recordings of whatever their faces do when we play sounds. We can then use facial recognition software to map a number of points to the faces in each scenario. Finally, we can ask a computer to look for common patterns in the points, and, if patterns are found, whether they are different enough to tell the two scenarios apart. If differences can be reliably detected between responses which occur when infants do or don’t hear sounds, or when they hear differences between sounds, then we can begin to use this approach to predict a wide range of clinically-relevant responses in individual infants, such as the detection of conversational speech, discrimination of speech sounds, identification of minimum response level, and so on. Such a system would provide hugely valuable insights into infants’ hearing very efficiently, cheaply, and quickly. It would require minimal specialist equipment, no equipment to be in contact with the infant, and could be conducted by a single researcher or clinician. Where might improvements in technology lead us in the future?In addition to the success of lab-based eye tracking paradigms, recent work suggests that infants’ attention can be measured by webcam video of their eye movements, and that even detailed eye tracking can be performed via an ordinary webcam [5,6]. Similar success with our facial recognition approach would open up the possibility of remote, teleaudiology assessment of infants in their own homes, helping to reduce obstacles for hard-to-reach populations and offering the flexibility that caregivers frequently require. Another possible development is that tests become increasingly tailored to individuals. Automation of the delivery of the test allows for real-time, response-contingent, Figure 2. Example of automatic face detection using OpenFace, capturing the overall head position (blue box) and facial features (red dots) of a four-month-old infant. “Success with our facial recognition approach would open up the possibility of remote, teleaudiology assessment of infants in their own homes, helping to reduce obstacles for hard-to-reach populations.”“For many caregivers, it can be difficult to recognise hearing impairment in their baby, and to appreciate the effects hearing loss can have. Behavioural testing can be hugely important in helping to show the need for and benefit of treatment.”Rachel Booth PhD MSc, Principal Clinical Scientist/Head of Paediatric Audiology.“Behavioural assessment is crucial for clinical decisions about treatment, and intervention is delayed until measurement becomes feasible. Reliable behavioural assessment of younger infants would allow us to lower the age of intervention, helping to significantly improve outcomes in children’s language and cognitive development.”Iain A Bruce MD FRCS (ORL-HNS), Consultant Paediatric Otolaryngologist/Honorary Clinical Professor of Paediatric Otolaryngology MAHSC, University of Manchester.The Royal Manchester Children’s Hospital offers specialist and tertiary services for children with hearing loss. It is the largest and busiest children’s hospital in the UK, as well as the UK’s largest audiology and otology centre. ent and audiology news | NOVEMBER/DECEMBER 2018 | VOL 27 NO 5 | www.entandaudiologynews.comselection and presentation of the stimuli we use to elicit responses. Content of stimuli could easily incorporate material directly relevant to each individual infant, such as images of caregivers’ faces or samples of caregivers’ speech, in order to maximise infant attention and the likelihood of a response. References1. Ching TYC, Dillon H, Button L, et al. Age at Intervention for Permanent Hearing Loss and 5-Year Language Outcomes. Pediatrics 2017;140(3):e20164274. 2. Jones PR, Kalwarowsky S, Braddick, et al. Optimizing the rapid measurement of detection thresholds in infants. Journal of Vision 2015;15(11):2. 3. Durand F, Freeman WT, Rubinstein M. A World of Movement. Scientific American 2015;312(1):48-51. 4. Goodfellow D, Zhi R, Funke, R. Predicting Infant Motor Development Status using Day Long Movement Data from Wearable Sensors. https://arxiv.org/pdf/1807.02617.pdf. Last accessed October 2018.5. Semmelmann K, Weigelt S. Online webcam-based eye tracking in cognitive science: A first look. Behavior Research Methods 2018;50(2):451-65. 6. Tran M, Cabral L, Patel R, Cusack R. Online recruitment and testing of infants with Mechanical Turk. Journal of Experimental Child Psychology 2017;156:168-78. AUDIOLOGY FEATUREAUTHORSIain R Jackson, PhD, Research Associate; Developmental psychologist, Manchester Centre for Audiology and Deafness, School of Health Sciences, University of Manchester, UK.E: Iain.jackson@manchester.ac.ukProf Lisa L Hunter, PhD, Scientific Director of Audiology, Cincinnati Children’s Hospital, Professor of Otolaryngology and Communication Sciences and Disorders, University of Cincinnati.E: Lisa.hunter@cchmc.orgPete R Jones, PhD, Research Fellow, The Child Vision Lab, UCL Institute of Ophthalmology, London, UK; The NIHR Biomedical Research Centre at Moorfields Eye Hospital, London, UK; University of London, London, UK.E: p.r.jones@ucl.ac.ukProf David R Moore,Director, Communication Sciences Research Center, Cincinnati Children’s Hospital, USA; Professor of Otolaryngology and Neuroscience, University of Cincinnati, USA; Professor of Auditory Neuroscience, Manchester Centre for Audiology and Deafness, School of Health Sciences, University of Manchester, UK.E: david.moore2@cchmc.orgProf Kevin J Munro, Ewing Professor of Audiology; Head of Manchester Centre for Audiology and Deafness, University of Manchester; Honorary Consultant Clinical Scientist, Manchester University Hospital Foundation Trust, UK.E: Kevin.munro@manchester.ac.ukAcknowledgement: This research is funded by the NIHR Manchester Biomedical Research Centre. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health. ent and audiology news | NOVEMBER/DECEMBER 2018 | VOL 27 NO 5 | www.entandaudiologynews.com

Facing up to the challenge of behavioural observation in infant hearing assessment

Jackson, Iain

Hunter, Lisa L.

Moore, David

Jones, Pete

Munro, Kevin

The University of Manchester - Institutional Repository

The University of Manchester ResearchFacing up the challenge of behavioural observation ininfant hearing assessmentLink to publication record in Manchester Research ExplorerCitation for published version (APA):Jackson, I., Hunter, L. L., Moore, D., Jones, P., & Munro, K. (2018). Facing up the challenge of behaviouralobservation in infant hearing assessment. ENT & Audiology news, 27(5).https://www.entandaudiologynews.com/features/audiology-features/post/facing-up-to-the-challenge-of-behavioural-observation-in-infant-hearing-assessmentPublished in:ENT & Audiology newsCiting this paperPlease note that where the full-text provided on Manchester Research Explorer is the Author Accepted Manuscriptor Proof version this may differ from the final Published version. If citing, it is advised that you check and use thepublisher's definitive version.General rightsCopyright and moral rights for the publications made accessible in the Research Explorer are retained by theauthors and/or other copyright owners and it is a condition of accessing publications that users recognise andabide by the legal requirements associated with these rights.Takedown policyIf you believe that this document breaches copyright please refer to the University of Manchester’s TakedownProcedures [http://man.ac.uk/04Y6Bo] or contact uml.scholarlycommunications@manchester.ac.uk providingrelevant details, so we can investigate your claim.Download date:09. Jun. 2022AUDIOLOGY FEATUREThe limits of current behavioural assessment in  young infantsFinding out whether an adult can hear something can often be as straightforward as simply asking them. Getting the same information from an infant is considerably less straightforward. However, obtaining information about an infant’s ability to detect and discriminate between sounds is crucial when making management decisions. This is especially important if we are to realise the benefits of newborn screening and early intervention. Objective physiologic tests, such as otoacoustic emissions (OAE) and auditory brainstem response (ABR), have become the most popular assessments used in young infants to assess the peripheral auditory system. Cortical evoked potentials show promise, but require sophisticated technology, and lengthy recording times, making them unsuitable for widespread screening programmes. However, physiologic techniques in current use do not provide information to assess benefit of intervention. Behavioural responses are considered the ‘gold standard’ to assess perception of sound, and provide a measure of functioning of the entire auditory system. That said, obtaining reliable and resource-efficient behavioural responses in young infants is challenging for a number of reasons, including: • Young infants’ behavioural responses are limited to higher sound intensities.• Behavioural responses can be subtle and highly variable (both within and between individuals).• Tests require expert examiners, which makes them expensive, and limits our ability to provide widespread screening.• Manual procedures are slow, and infants habituate rapidly to sounds presented to them, meaning that many become too fatigued to complete the test.The general consensus in clinical practice is that behavioural observations should not (or cannot) be relied upon until the infant reaches a developmental age of seven-to-nine months and head turns to sounds can be elicited and rewarded. Even then, test procedures can be time- and labour-intensive, and scoring is subject to bias. Technology opens the door to novel approachesFortunately, new technologies offer opportunities for cheaper, faster, and more reliable testing. An ever-increasing ability to capture different behaviours allows us to reconsider what responses are available for the tools we use. Outside of audiology, techniques such as eye tracking and pupillometry have been widely adopted in developmental psychology, and have transformed the variety and depth of questions infancy researchers are able to explore. At least two novel approaches have been found to work in efforts to measure hearing and vision efficiently and accurately in babies; (i) maximising engagement with a video display minimises attention lapses, and (ii) adopting a flexible, integrative approach to automation that simulates previous observer-based methods [2]. Elsewhere, video-based analyses now offer other surprising insights, which are often inaccessible to human observers. Recent work has shown, for example, that it is possible to detect subtle movements associated with babies’ breathing, or to detect heart-rate from imperceptible, pixel-level changes in the colour of faces [3].In addition to impressive developments in the measurement Facing up to the challenge of behavioural observation in infant hearing assessmentBY IAIN JACKSON, LISA HUNTER, DAVID MOORE, PETE JONES AND KEVIN MUNROThe ability to assess detection and discrimination of speech by infants has proved elusive. Dr Iain Jackson and colleagues discuss how new technologies and fresh approaches might offer valuable insight into young infants’ behavioural responses to sound. “The ability to capture an increasing range of behaviours allows us to reconsider what responses are available to us for the tools we use.”Figure 1. An illustration of the importance of early intervention; the effect of age at fitting of hearing aids on the language outcomes of children with different levels of hearing loss (Better-ear 4-frequency average). (Adapted with permission from Ching et al [1]; personal correspondence with Harvey Dillon.) ent and audiology news | NOVEMBER/DECEMBER 2018 | VOL 27 NO 5 | www.entandaudiologynews.comAUDIOLOGY FEATUREof responses, modern computation techniques, such as machine learning, increasingly promise to offer insights and predictive value using healthcare data, including in the assessment of infants. For example, in a recent study, Goodfellow and colleagues placed wearable sensors on infants, and recorded their spontaneous movements over the course of a day [4]. Their machine learning algorithms were able to differentiate between the movements of typically developing infants and those at risk for developmental delays, potentially paving the way for a clinically useful tool for predictive assessment. Applying new technologies to infant audiometryAs part of the NIHR Manchester Biomedical Research Centre, we are currently looking to combine automatic facial recognition, head tracking, and eye tracking, along with state-of-the-art machine learning algorithms, in order to be able to detect when an infant hears a sound.Infants excel at sending signals with their faces – all caregivers are acutely aware of how easy it is to recognise when an infant is unhappy! In addition to clear and obvious signals, like crying, there might also be other valuable markers we can capitalise on. Facial recognition software can detect and track a wide range of facial behaviours, from broad, global features such as head-turns, down to more subtle changes in individual features or areas of the face. For example, infants may raise their eyebrows ever so slightly when they hear a sound, or smile, or frown, or let their mouths fall open slightly. Whatever the response might be, if one or more responses occur frequently enough, it will provide a pattern for the algorithm to recognise, and thus provide a signal for researchers to measure. In this approach, we take video recordings of many infants’ faces and capture a range of examples of the spontaneous expressions and movements infants commonly make in the absence of sound. We also take video recordings of whatever their faces do when we play sounds. We can then use facial recognition software to map a number of points to the faces in each scenario. Finally, we can ask a computer to look for common patterns in the points, and, if patterns are found, whether they are different enough to tell the two scenarios apart. If differences can be reliably detected between responses which occur when infants do or don’t hear sounds, or when they hear differences between sounds, then we can begin to use this approach to predict a wide range of clinically-relevant responses in individual infants, such as the detection of conversational speech, discrimination of speech sounds, identification of minimum response level, and so on. Such a system would provide hugely valuable insights into infants’ hearing very efficiently, cheaply, and quickly. It would require minimal specialist equipment, no equipment to be in contact with the infant, and could be conducted by a single researcher or clinician. Where might improvements in technology lead us in the future?In addition to the success of lab-based eye tracking paradigms, recent work suggests that infants’ attention can be measured by webcam video of their eye movements, and that even detailed eye tracking can be performed via an ordinary webcam [5,6]. Similar success with our facial recognition approach would open up the possibility of remote, teleaudiology assessment of infants in their own homes, helping to reduce obstacles for hard-to-reach populations and offering the flexibility that caregivers frequently require. Another possible development is that tests become increasingly tailored to individuals. Automation of the delivery of the test allows for real-time, response-contingent, Figure 2. Example of automatic face detection using OpenFace, capturing the overall head position (blue box) and facial features (red dots) of a four-month-old infant. “Success with our facial recognition approach would open up the possibility of remote, teleaudiology assessment of infants in their own homes, helping to reduce obstacles for hard-to-reach populations.”“For many caregivers, it can be difficult to recognise hearing impairment in their baby, and to appreciate the effects hearing loss can have. Behavioural testing can be hugely important in helping to show the need for and benefit of treatment.”Rachel Booth PhD MSc, Principal Clinical Scientist/Head of Paediatric Audiology.“Behavioural assessment is crucial for clinical decisions about treatment, and intervention is delayed until measurement becomes feasible. Reliable behavioural assessment of younger infants would allow us to lower the age of intervention, helping to significantly improve outcomes in children’s language and cognitive development.”Iain A Bruce MD FRCS (ORL-HNS), Consultant Paediatric Otolaryngologist/Honorary Clinical Professor of Paediatric Otolaryngology MAHSC, University of Manchester.The Royal Manchester Children’s Hospital offers specialist and tertiary services for children with hearing loss. It is the largest and busiest children’s hospital in the UK, as well as the UK’s largest audiology and otology centre. ent and audiology news | NOVEMBER/DECEMBER 2018 | VOL 27 NO 5 | www.entandaudiologynews.comselection and presentation of the stimuli we use to elicit responses. Content of stimuli could easily incorporate material directly relevant to each individual infant, such as images of caregivers’ faces or samples of caregivers’ speech, in order to maximise infant attention and the likelihood of a response. References1. Ching TYC, Dillon H, Button L, et al. Age at Intervention for Permanent Hearing Loss and 5-Year Language Outcomes. Pediatrics 2017;140(3):e20164274. 2. Jones PR, Kalwarowsky S, Braddick, et al. Optimizing the rapid measurement of detection thresholds in infants. Journal of Vision 2015;15(11):2. 3. Durand F, Freeman WT, Rubinstein M. A World of Movement. Scientific American 2015;312(1):48-51. 4. Goodfellow D, Zhi R, Funke, R. Predicting Infant Motor Development Status using Day Long Movement Data from Wearable Sensors. https://arxiv.org/pdf/1807.02617.pdf. Last accessed October 2018.5. Semmelmann K, Weigelt S. Online webcam-based eye tracking in cognitive science: A first look. Behavior Research Methods 2018;50(2):451-65. 6. Tran M, Cabral L, Patel R, Cusack R. Online recruitment and testing of infants with Mechanical Turk. Journal of Experimental Child Psychology 2017;156:168-78. AUDIOLOGY FEATUREAUTHORSIain R Jackson, PhD, Research Associate; Developmental psychologist, Manchester Centre for Audiology and Deafness, School of Health Sciences, University of Manchester, UK.E: Iain.jackson@manchester.ac.ukProf Lisa L Hunter, PhD, Scientific Director of Audiology, Cincinnati Children’s Hospital, Professor of Otolaryngology and Communication Sciences and Disorders, University of Cincinnati.E: Lisa.hunter@cchmc.orgPete R Jones, PhD, Research Fellow, The Child Vision Lab, UCL Institute of Ophthalmology, London, UK; The NIHR Biomedical Research Centre at Moorfields Eye Hospital, London, UK; University of London, London, UK.E: p.r.jones@ucl.ac.ukProf David R Moore,Director, Communication Sciences Research Center, Cincinnati Children’s Hospital, USA; Professor of Otolaryngology and Neuroscience, University of Cincinnati, USA; Professor of Auditory Neuroscience, Manchester Centre for Audiology and Deafness, School of Health Sciences, University of Manchester, UK.E: david.moore2@cchmc.orgProf Kevin J Munro, Ewing Professor of Audiology; Head of Manchester Centre for Audiology and Deafness, University of Manchester; Honorary Consultant Clinical Scientist, Manchester University Hospital Foundation Trust, UK.E: Kevin.munro@manchester.ac.ukAcknowledgement: This research is funded by the NIHR Manchester Biomedical Research Centre. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health. ent and audiology news | NOVEMBER/DECEMBER 2018 | VOL 27 NO 5 | www.entandaudiologynews.com

Facing up the challenge of behavioural observation in infant hearing assessment

https://openaccess.city.ac.uk/id/eprint/21556/1/PE_entnd18-jackson-etalnew.pdf

Facing up to the challenge of behavioural observation in infant hearing assessment

Abstract

Similar works

Full text

Available Versions

City Research Online

The University of Manchester - Institutional Repository