A Technique for Characterizing the Development of Rhythms in Bird Song

The developmental trajectory of nervous system dynamics shows hierarchical structure on time scales spanning ten orders of magnitude from milliseconds to years. Analyzing and characterizing this structure poses significant signal processing challenges. In the context of birdsong development, we have previously proposed that an effective way to do this is to use the dynamic spectrum or spectrogram, a classical signal processing tool, computed at multiple time scales in a nested fashion. Temporal structure on the millisecond timescale is normally captured using a short time Fourier analysis, and structure on the second timescale using song spectrograms. Here we use the dynamic spectrum on time series of song features to study the development of rhythm in juvenile zebra finch. The method is able to detect rhythmic structure in juvenile song in contrast to previous characterizations of such song as unstructured. We show that the method can be used to examine song development, the accuracy with which rhythm is imitated, and the variability of rhythms across different renditions of a song. We hope that this technique will provide a standard, automated method for measuring and characterizing song rhythm

A flowchart of the nested spectral analysis as described in the text.

<p>A flowchart of the nested spectral analysis as described in the text.</p

The relations of motif durations and the fundamental frequency of the rhythm spectrogram.

<p>Changes in the motif duration show up as changes in the fundamental frequency of the rhythm spectrogram as described in the text.</p

Rhythm of juvenile songs.

<p>The rhythm of juvenile song can be identified early during development, as described in the text.</p

A spectrogram of an adult zebra finch song.

<p>This song has three repetitions of the motif. An occurrence of song is called a bout.</p

Regular song spectrograms versus Rhythm spectrograms.

<p>A. A regular song spectrogram using a 10msec sliding window, showing power up to several kHz. B. Rhythm spectrograms display longer time scales. These are computed by estimating the dynamic spectrum of an appropriate song feature (amplitude in the above example). Each column of the rhythm spectrogram represents the averaged spectrum of song features sung during an hour long interval. C. Rhythm spectrograms that were generated using a point process that marks the onsets of syllables.</p

Multimedia Signal Processing for Behavioral Quantification in Neuroscience

While there have been great advances in quantification of the genotype of organisms, including full genomes for many species, the quantification of phenotype is at a comparatively primitive stage. Part of the reason is technical difficulty: the phenotype covers a wide range of characteristics, ranging from static morphological features, to dynamic behavior. The latter poses challenges that are in the area of multimedia signal processing. Automated analysis of video and audio recordings of animal and human behavior is a growing area of research, ranging from the behavioral phenotyping of genetically modified mice or drosophila to the study of song learning in birds and speech acquisition in human infants. This paper reviews recent advances and identifies key problems for a range of behavior experiments that use audio and video recording. This research area offers both research challenges and an application domain for advanced multimedia signal processing. There are a number of MMSP tools that now exist which are directly relevant for behavioral quantification, such as speech recognition, video analysis and more recently, wired and wireless sensor networks for surveillance. The research challenge is to adapt these tools and to develop new ones required for studying human and animal behavior in a high throughput manner while minimizing human intervention. In contrast with consumer applications, in the research arena there is less of a penalty for computational complexity, so that algorithmic quality can be maximized through the utilization of larger computational resources that are available to the biomedical researcher