Large-scale analysis of frequency modulation in birdsong data bases

DS & MP are supported by an EPSRC Leadership Fellowship EP/G007144/1. Our thanks to Alan McElligott for helpful advice while preparing the manuscript; Sašo Muševič for discussion and for making his DDM software available; and Rémi Gribonval and team at INRIA Rennes for discussion and software development during a research visit

Neurally driven synthesis of learned, complex vocalizations

Brain machine interfaces (BMIs) hold promise to restore impaired motor function and serve as powerful tools to study learned motor skill. While limb-based motor prosthetic systems have leveraged nonhuman primates as an important animal model,1–4 speech prostheses lack a similar animal model and are more limited in terms of neural interface technology, brain coverage, and behavioral study design.5–7 Songbirds are an attractive model for learned complex vocal behavior. Birdsong shares a number of unique similarities with human speech,8–10 and its study has yielded general insight into multiple mechanisms and circuits behind learning, execution, and maintenance of vocal motor skill.11–18 In addition, the biomechanics of song production bear similarity to those of humans and some nonhuman primates.19–23 Here, we demonstrate a vocal synthesizer for birdsong, realized by mapping neural population activity recorded from electrode arrays implanted in the premotor nucleus HVC onto low-dimensional compressed representations of song, using simple computational methods that are implementable in real time. Using a generative biomechanical model of the vocal organ (syrinx) as the low-dimensional target for these mappings allows for the synthesis of vocalizations that match the bird's own song. These results provide proof of concept that high-dimensional, complex natural behaviors can be directly synthesized from ongoing neural activity. This may inspire similar approaches to prosthetics in other species by exploiting knowledge of the peripheral systems and the temporal structure of their output.Fil: Arneodo, Ezequiel Matías. University of California; Estados Unidos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Física La Plata. Universidad Nacional de La Plata. Facultad de Ciencias Exactas. Instituto de Física La Plata; ArgentinaFil: Chen, Shukai. University of California; Estados UnidosFil: Brown, Daril E.. University of California; Estados UnidosFil: Gilja, Vikash. University of California; Estados UnidosFil: Gentner, Timothy Q.. The Kavli Institute For Brain And Mind; Estados Unidos. University of California; Estados Unido

Seeing sound: a new way to illustrate auditory objects and their neural correlates

This thesis develops a new method for time-frequency signal processing and examines the relevance of the new representation in studies of neural coding in songbirds. The method groups together associated regions of the time-frequency plane into objects defined by time-frequency contours. By combining information about structurally stable contour shapes over multiple time-scales and angles, a signal decomposition is produced that distributes resolution adaptively. As a result, distinct signal components are represented in their own most parsimonious forms.  Next, through neural recordings in singing birds, it was found that activity in song premotor cortex is significantly correlated with the objects defined by this new representation of sound. In this process, an automated way of finding sub-syllable acoustic transitions in birdsongs was first developed, and then increased spiking probability was found at the boundaries of these acoustic transitions. Finally, a new approach to study auditory cortical sequence processing more generally is proposed. In this approach, songbirds were trained to discriminate Morse-code-like sequences of clicks, and the neural correlates of this behavior were examined in primary and secondary auditory cortex. It was found that a distinct transformation of auditory responses to the sequences of clicks exists as information transferred from primary to secondary auditory areas. Neurons in secondary auditory areas respond asynchronously and selectively -- in a manner that depends on the temporal context of the click. This transformation from a temporal to a spatial representation of sound provides a possible basis for the songbird's natural ability to discriminate complex temporal sequences

New horizons for female birdsong : evolution, culture and analysis tools : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Ecology at Massey University, Auckland, New Zealand

Published papers appear in Appendix 7.1. and 7.2 respectively under a CC BY 4.0 and CC BY licence: Webb, W. H., Brunton, D. H., Aguirre, J. D., Thomas, D. B., Valcu, M., & Dale, J. (2016). Female song occurs in songbirds with more elaborate female coloration and reduced sexual dichromatism. Frontiers in Ecology and Evolution, 4(22). https://doi.org/10.3389/fevo.2016.00022 Yukio Fukuzawa, Wesley Webb, Matthew Pawley, Michelle Roper, Stephen Marsland, Dianne Brunton, & Andrew Gilman. (2020). Koe: Web-based software to classify acoustic units and analyse sequence structure in animal vocalisations. Methods in Ecology and Evolution, 11(3). https://doi.org/10.1111/2041-210X.13336As a result of male-centric, northern-hemisphere-biased sexual selection theory, elaborate female traits in songbirds have been largely overlooked as unusual or non-functional by-products of male evolution. However, recent research has revealed that female song is present in most surveyed songbirds and was in fact the ancestral condition to the clade. Additionally, a high proportion of songbird species have colourful females, and both song and showy colours have demonstrated female-specific functions in a growing number of species. We have much to learn about the evolution and functions of elaborate female traits in general, and female song in particular. This thesis extends the horizons of female birdsong research in three ways: (1) by revealing the broad-scale evolutionary relationship of female song and plumage elaboration across the songbirds, (2) by developing new accessible tools for the measurement and analysis of song complexity, and (3) by showing—through a detailed field study on a large natural metapopulation—how vocal culture operates differentially in males and females. First, to understand the drivers of elaborate female traits, I tested the evolutionary relationship between female song presence and plumage colouration across the songbirds. I found strong support for a positive evolutionary correlation between traits, with female song more prevalent amongst species with elaborated female plumage. These results suggest that contrary to the idea of trade-off between showy traits, female plumage colouration and female song likely evolved together under similar selection pressures and that their respective functions are reinforcing. Second, I introduce new bioacoustics software, Koe, designed to meet the need for detailed classification and analysis of song complexity. The program enables visualisation, segmentation, rapid classification and analysis of song structure. I demonstrate Koe with a case study of New Zealand bellbird Anthornis melanura song, showcasing the capabilities for large-scale bioacoustics research and its application to female song. Third, I conducted one of the first detailed field-based analyses of female song culture, studying an archipelago metapopulation of New Zealand bellbirds. Comparing between male and female sectors of each population, I found equal syllable diversity, largely separate repertoires, and contrasting patterns of sharing between sites—revealing female dialects and pronounced sex differences in cultural evolution. By combining broad-scale evolutionary approaches, novel song analysis tools, and a detailed field study, this thesis demonstrates that female song can be as much an elaborate signal as male song. I describe how future work can build on these findings to expand understanding of elaborate female traits

Automatic bird species identification employing an unsupervised discovery of vocalisation units

An automatic analysis of bird vocalisations for the identification of bird species, the study of their behaviour and their means of communication is important for a better understanding of the environment in which we are living and in the context of environmental protection. The high variability of vocalisations within different individuals makes species’ identification challenging for bird surveyors. Hence, the availability of a reliable automatic bird identification system through their vocalisations, would be of great interest to professionals and amateurs alike. A part of this thesis provides a biological survey on the scientific theories of the study of bird vocalisation and corresponding singing behaviours. Another section of this thesis aims to discover a set of element patterns produced by each bird species in a large corpus of the natural field recordings. Also this thesis aims to develop an automatic system for the identification of bird species from recordings. Two HMM based recognition systems are presented in this research. Evaluations have been demonstrated where the proposed element based HMM system obtained a recognition accuracy of over 93% by using 3 seconds of detected signal and over 39% recognition error rate reduction, compared to the baseline HMM system of the same complexity

Emergence of linguistic laws in human voice

Submitted for publicationSubmitted for publicatio

Dissimilarity-based multiple instance classification and dictionary learning for bioacoustic signal recognition

In this thesis, two promising and actively researched fields from pattern recognition (PR) and digital signal processing (DSP) are studied, adapted and applied for the automated recognition of bioacoustic signals: (i) learning from weakly-labeled data, and (ii) dictionary-based decomposition. The document begins with an overview of the current methods and techniques applied for the automated recognition of bioacoustic signals, and an analysis of the impact of this technology at global and local scales. This is followed by a detailed description of my research on studying two approaches from the above-mentioned fields, multiple instance learning (MIL) and dictionary learning (DL), as solutions to particular challenges in bioacoustic data analysis. The most relevant contributions and findings of this thesis are the following ones: 1) the proposal of an unsupervised recording segmentation method of audio birdsong recordings that improves species classification with the benefit of an easier implementation since no manual handling of recordings is required; 2) the confirmation that, in the analyzed audio datasets, appropriate dissimilarity measures are those which capture most of the overall differences between bags, such as the modified Hausdorff distance and the mean minimum distance; 3) the adoption of dissimilarity adaptation techniques for the enhancement of dissimilarity-based multiple instance classification, along with the potential further enhancement of the classification performance by building dissimilarity spaces and increasing training set sizes; 4) the proposal of a framework for solving MIL problems by using the one nearest neighbor (1-NN) classifier; 5) a novel convolutive DL method for learning a representative dictionary from a collection of multiple-bird audio recordings; 6) such a DL method is successfully applied to spectrogram denoising and species classification; and, 7) an efficient online version of the DL method that outperforms other state-of-the-art batch and online methods, in both, computational cost and quality of the discovered patternsResumen : En esta tesis se estudian, adaptan y aplican dos prometedoras y activas áreas del reconocimiento de patrones (PR) y procesamiento digital de señales (DSP): (i) aprendizaje débilmente supervisado y (ii) descomposiciones basadas en diccionarios. Inicialmente se hace una revisión de los métodos y técnicas que actualmente se aplican en tareas de reconocimiento automatizado de señales bioacústicas y se describe el impacto de esta tecnología a escalas nacional y global. Posteriormente, la investigación se enfoca en el estudio de dos técnicas de las áreas antes mencionadas, aprendizaje multi-instancia (MIL) y aprendizaje de diccionarios (DL), como soluciones a retos particulares del análisis de datos bioacústicos. Las contribuciones y hallazgos ms relevantes de esta tesis son los siguientes: 1) se propone un método de segmentacin de grabaciones de audio que mejora la clasificación automatizada de especies, el cual es fácil de implementar ya que no necesita información supervisada de entrenamiento; 2) se confirma que, en los conjuntos de datos analizados, las medidas de disimilitudes que capturan las diferencias globales entre bolsas funcionan apropiadamente, tales como la distancia modificada de Hausdorff y la distancia media de los mínimos; 3) la adopción de técnicas de adaptación de disimilitudes para mejorar la clasificación multi-instancia, junto con el incremento potencial del desempeño por medio de la construcción de espacios de disimilitudes y el aumento del tamaño de los conjuntos de entrenamiento; 4) se presenta un esquema para la solución de problemas MIL por medio del clasificador del vecino ms cercano (1-NN); 5) se propone un método novedoso de DL, basado en convoluciones, para el aprendizaje automatizado de un diccionario representativo a partir de un conjunto de grabaciones de audio de múltiples vocalizaciones de aves; 6) dicho mtodo DL se utiliza exitosamente como técnica de reducción de ruido en espectrogramas y clasificación de grabaciones bioacústicas; y 7) un método DL, de procesamiento en línea, que supera otros métodos del estado del arte en costo computacional y calidad de los patrones descubiertosDoctorad

Variability in Singing and in Song in the Zebra Finch

Variability is a defining feature of the oscine song learning process, reflected in song and in the neural pathways involved in song learning. For the zebra finch, juveniles learning to sing typically exhibit a high degree of vocal variability, and this variability appears to be driven by a key brain nucleus. It has been suggested that this variability is a necessary part of a trial-­â€and-­â€error learning process in which the bird must search for possible improvements to its song. Our work examines the role this variability plays in learning in two ways: through behavioral experiments with juvenile zebra finches, and through a computational model of parts of the oscine brain. Previous studies have shown that some finches exhibit less variability during the learning process than others by producing repetitive vocalizations. A constantly changing song model was played to juvenile zebra finches to determine whether auditory stimuli can affect this behavior. This stimulus was shown to cause an overall increase in repetitiveness; furthermore, there was a correlation between repetitiveness at an early stage in the learning process and the length of time a bird is repetitive overall, and birds that were repetitive tended to repeat the same thing over an extended period of time. The role of a key brain nucleus involved in song learning was examined through computational modeling. Previous studies have shown that this nucleus produces variability in song, but can also bias the song of a bird in such a way as to reduce errors while singing. Activity within this nucleus during singing is predominantly uncorrelated with the timing of the song, however a portion of this activity is correlated in such a manner. The modeling experiments consider the possibility that this persistent signal is part of a trial-­â€and-­â€error search and contrast this with the possibility that the persistent signal is the product of some mechanism to directly improve song. Simulation results show that a mixture of timing-­â€dependent and timing-­â€independent activity in this nucleus produces optimal learning results for the case where the persistent signal is a key component of a trial-­â€and-­â€error search, but not in the case where this signal will directly improve song. Although a mixture of timing-­â€locked and timing-­â€independent activity produces optimal results, the ratio found to be optimal within the model differs from what has been observed in vivo. Finally, novel methods for the analysis of birdsong, motivated by the high variability of juvenile song, are presented. These methods are designed to work with sets of song samples rather than through pairwise comparison. The utility of these methods is demonstrated, as well as results illustrating how such methods can be used as the basis for aggregate measures of song such as repertoire complexity