Smart sampling of environmental audio recordings for biodiversity monitoring

This thesis contributes to the field of acoustic environmental monitoring by developing novel semiautomated methods of processing long audio recordings to conduct species richness surveys efficiently. These methods allow a machine to select rich subset of the recordings though estimations of acoustic variety, which can then be presented to the human listener for species identifications. This work represents a step towards more effective biodiversity monitoring of vocal species that can be performed at a larger scale than is possible with traditional methods

Heat maps for aggregating bioacoustic annotations

In our large library of annotated environmental recordings of animal vocalizations, searching annotations by label can return thousands of results. We propose a heat map of aggregated annotation time and frequency bounds, maintaining the shape of the annotations as they appear on the spectrogram. This compactly displays the distribution of annotation bounds for the user's query, and allows them to easily identify unusual annotations. Key to this is allowing zero values on the map to be differentiated from areas where there are single annotations

Datatrack: An R package for managing data in a multi-stage experimental workflow: data versioning and provenance considerations in interactive scripting

In experimental research using computation, a workflow is a sequence of steps involving some data processing or analysis where the output of one step may be used as the input of another. The processing steps may involve user-supplied parameters, that when modified, result in a new version of input to the downstream steps, in turn generating new versions of their own output. As more experimentation is done, the results of these various steps can become numerous. It is important to keep track of which data output is dependent on which other generated data, and which parameters were used. In many situations, scientific workflow management systems solve this problem, but these systems are best suited to collaborative, distributed experiments using a variety of services, possibly batch processing parameter sweeps. This paper presents an R package for managing and navigating a network of interdependent data. It is intended as a lightweight tool that provides some visual data provenance information to the experimenter to allow them to manage their generated data as they run experiments within their familiar scripting environment, where it may not be desirable to commit to a fully-blown comprehensive workflow manager. The package consists of wrapper functions for writing and reading output data that can be called from within the R analysis scripts, as well as a visualization of the data-output dependency graph rendered within the R-studio console. Thus, it offers benefit to the experimenter while requiring minimal commitment for integration in their existing working environment

Birdcall retrieval from environmental acoustic recordings using image processing

Acoustic recordings of the environment provide an effective means to monitor bird species diversity. To facilitate exploration of acoustic recordings, we describe a content-based birdcall retrieval algorithm. A query birdcall is a region of spectrogram bounded by frequency and time. Retrieval depends on a similarity measure derived from the orientation and distribution of spectral ridges. The spectral ridge detection method caters for a broad range of birdcall structures. In this paper, we extend previous work by incorporating a spectrogram scaling step in order to improve the detection of spectral ridges. Compared to an existing approach based on MFCC features, our feature representation achieves better retrieval performance for multiple bird species in noisy recordings

Acoustic feature extraction using perceptual wavelet packet decomposition for frog call classification

Frog protection has become increasingly essential due to the rapid decline of its biodiversity. Therefore, it is valuable to develop new methods for studying this biodiversity. In this paper, a novel feature extraction method is proposed based on perceptual wavelet packet decomposition for classifying frog calls in noisy environments. Pre-processing and syllable segmentation are first applied to the frog call. Then, a spectral peak track is extracted from each syllable if possible. Track duration, dominant frequency and oscillation rate are directly extracted from the track. With k-means clustering algorithm, the calculated dominant frequency of all frog species is clustered into k parts, which produce a frequency scale for wavelet packet decomposition. Based on the adaptive frequency scale, wavelet packet decomposition is applied to the frog calls. Using the wavelet packet decomposition coefficients, a new feature set named perceptual wavelet packet decomposition sub-band cepstral coefficients is extracted. Finally, a k-nearest neighbour (k-NN) classifier is used for the classification. The experiment results show that the proposed features can achieve an average classification accuracy of 97.45% which outperforms syllable features (86.87%) and Mel-frequency cepstral coefficients (MFCCs) feature (90.80%)

Assistive classification for improving the efficiency of avian species richness surveys

Avian species richness surveys, which measure the total number of unique avian species, can be conducted via remote acoustic sensors. An immense quantity of data can be collected, which, although rich in useful information, places a great workload on the scientists who manually inspect the audio. To deal with this big data problem, we calculated acoustic indices from audio data at a one-minute resolution and used them to classify one-minute recordings into five classes. By filtering out the non-avian minutes, we can reduce the amount of data by about 50% and improve the efficiency of determining avian species richness. The experimental results show that, given 60 one-minute samples, our approach enables to direct ecologists to find about 10% more avian species

Practical analysis of big acoustic sensor data for environmental monitoring

Monitoring the environment with acoustic sensors is an effective method for understanding changes in ecosystems. Through extensive monitoring, large-scale, ecologically relevant, datasets can be produced that can inform environmental policy. The collection of acoustic sensor data is a solved problem; the current challenge is the management and analysis of raw audio data to produce useful datasets for ecologists. This paper presents the applied research we use to analyze big acoustic datasets. Its core contribution is the presentation of practical large-scale acoustic data analysis methodologies. We describe details of the data workflows we use to provide both citizen scientists and researchers practical access to large volumes of ecoacoustic data. Finally, we propose a work in progress large-scale architecture for analysis driven by a hybrid cloud-and-local production-grade website

Clustering and visualization of long-duration audio recordings for rapid exploration avian surveys

Acoustic recordings have been shown to be an effective way to conduct avian species surveys, whereby a trained expert listens to the audio and records observations, a task that can be very time consuming. In practice, most identification of species are first made by visual inspection of the spectrogram, with listening then performed for verification. This paper presents an approach for a surveyor to rapidly scan long duration recordings of environmental audio by automatically filtering parts with low activity and repetitions of the same call types. Recordings are segmented into fixed-length one-second non-overlapping clips. A classifier filters segments of low activity using features that are robust to different levels of background noise. The non-silent segments are then clustered using a feature representation derived from Time-domain Cepstral Coefficients, calculated from the discrete Fourier transform of downsampled spectrogram rows. This time-invariant feature representation allows for arbitrary segmentation, which is advantageous because segmentation of complex audio soundscapes into individual events is difficult and prone to errors. A visualization tool displays a representative segment from each cluster, grouped hierarchically, allowing an ecological researcher to very rapidly visually scan through the entire variety of audio events that occurred throughout the long recording, without wasting time on silent portions of the recording or repetitions of the same call-type. This tool provides functionality missing from both time-consuming audio players and black-box pattern recognizers, allowing conservation scientists to visually explore the entirety of their recordings

Acoustic classification of Australian anurans using syllable features

Acoustic classification of anurans (frogs) has received increasing attention for its promising application in biological and environment studies. In this study, a novel feature extraction method for frog call classification is presented based on the analysis of spectrograms. The frog calls are first automatically segmented into syllables. Then, spectral peak tracks are extracted to separate desired signal (frog calls) from background noise. The spectral peak tracks are used to extract various syllable features, including: syllable duration, dominant frequency, oscillation rate, frequency modulation, and energy modulation. Finally, a k-nearest neighbor classifier is used for classifying frog calls based on the results of principal component analysis. The experiment results show that syllable features can achieve an average classification accuracy of 90.5% which outperforms Mel-frequency cepstral coefficients features (79.0%)

Clustering acoustic events in environmental recordings for species richness surveys

Environmental acoustic recordings can be used to perform avian species richness surveys, whereby a trained ornithologist can observe the species present by listening to the recording. This could be made more efficient by using computational methods for iteratively selecting the richest parts of a long recording for the human observer to listen to, a process known as “smart sampling”. This allows scaling up to much larger ecological datasets. In this paper we explore computational approaches based on information and diversity of selected samples. We propose to use an event detection algorithm to estimate the amount of information present in each sample. We further propose to cluster the detected events for a better estimate of this amount of information. Additionally, we present a time dispersal approach to estimating diversity between iteratively selected samples. Combinations of approaches were evaluated on seven 24-hour recordings that have been manually labeled by bird watchers. The results show that on average all the methods we have explored would allow annotators to observe more new species in fewer minutes compared to a baseline of random sampling at dawn