Co-Localization of Audio Sources in Images Using Binaural Features and Locally-Linear Regression

This paper addresses the problem of localizing audio sources using binaural measurements. We propose a supervised formulation that simultaneously localizes multiple sources at different locations. The approach is intrinsically efficient because, contrary to prior work, it relies neither on source separation, nor on monaural segregation. The method starts with a training stage that establishes a locally-linear Gaussian regression model between the directional coordinates of all the sources and the auditory features extracted from binaural measurements. While fixed-length wide-spectrum sounds (white noise) are used for training to reliably estimate the model parameters, we show that the testing (localization) can be extended to variable-length sparse-spectrum sounds (such as speech), thus enabling a wide range of realistic applications. Indeed, we demonstrate that the method can be used for audio-visual fusion, namely to map speech signals onto images and hence to spatially align the audio and visual modalities, thus enabling to discriminate between speaking and non-speaking faces. We release a novel corpus of real-room recordings that allow quantitative evaluation of the co-localization method in the presence of one or two sound sources. Experiments demonstrate increased accuracy and speed relative to several state-of-the-art methods.Comment: 15 pages, 8 figure

Wireless Localization in the Absence of GPS

In this thesis, wireless localization is investigated based on multiple noisy estimates of the time-difference of arrival (TDOA) at each pair of the n \u3e= 4 sensors with different but known locations using WiFi opportunistic signals. Our work is comprehensive and includes channel estimation, symbol detection, TDOA estimation, and location estimation. To mitigate the multipath issue induced by wideband signals, such as WiFi, frequency-division is employed to decompose the wideband RF signal into multiple non-overlapping narrowband signals. To minimize the adverse effects of the clock drift, time-division is proposed to divide the signal into multiple non-overlapping signals in the time domain. In addition, Kalman filtering is proposed, assuming the wide-sense stationary and uncorrelated scattering (WSSUS) channel and the first order auto-regressive (AR) model are used. Because of the multiple TDOA estimates at each pair of the WiFi receiver sensors, an efficient algorithm is developed to estimate the target location. The localization technique developed in this thesis can also be extended to other radio frequency (RF) signals, as shown in our simulation study for out-door localization. The simulation results in our thesis show the effectiveness of the wireless localization, although further work is needed to resolve the nonlinear estimation problem involved in localization based on TDOA estimates

Locating and extracting acoustic and neural signals

This dissertation presents innovate methodologies for locating, extracting, and separating multiple incoherent sound sources in three-dimensional (3D) space; and applications of the time reversal (TR) algorithm to pinpoint the hyper active neural activities inside the brain auditory structure that are correlated to the tinnitus pathology. Specifically, an acoustic modeling based method is developed for locating arbitrary and incoherent sound sources in 3D space in real time by using a minimal number of microphones, and the Point Source Separation (PSS) method is developed for extracting target signals from directly measured mixed signals. Combining these two approaches leads to a novel technology known as Blind Sources Localization and Separation (BSLS) that enables one to locate multiple incoherent sound signals in 3D space and separate original individual sources simultaneously, based on the directly measured mixed signals. These technologies have been validated through numerical simulations and experiments conducted in various non-ideal environments where there are non-negligible, unspecified sound reflections and reverberation as well as interferences from random background noise. Another innovation presented in this dissertation is concerned with applications of the TR algorithm to pinpoint the exact locations of hyper-active neurons in the brain auditory structure that are directly correlated to the tinnitus perception. Benchmark tests conducted on normal rats have confirmed the localization results provided by the TR algorithm. Results demonstrate that the spatial resolution of this source localization can be as high as the micrometer level. This high precision localization may lead to a paradigm shift in tinnitus diagnosis, which may in turn produce a more cost-effective treatment for tinnitus than any of the existing ones

ORCA-SPY enables killer whale sound source simulation, detection, classification and localization using an integrated deep learning-based segmentation

Acoustic identification of vocalizing individuals opens up new and deeper insights into animal communications, such as individual-/group-specific dialects, turn-taking events, and dialogs. However, establishing an association between an individual animal and its emitted signal is usually non-trivial, especially for animals underwater. Consequently, a collection of marine species-, array-, and position-specific ground truth localization data is extremely challenging, which strongly limits possibilities to evaluate localization methods beforehand or at all. This study presents ORCA-SPY, a fully-automated sound source simulation, classification and localization framework for passive killer whale (Orcinus orca) acoustic monitoring that is embedded into PAMGuard, a widely used bioacoustic software toolkit. ORCA-SPY enables array- and position-specific multichannel audio stream generation to simulate real-world ground truth killer whale localization data and provides a hybrid sound source identification approach integrating ANIMAL-SPOT, a state-of-the-art deep learning-based orca detection network, followed by downstream Time-Difference-Of-Arrival localization. ORCA-SPY was evaluated on simulated multichannel underwater audio streams including various killer whale vocalization events within a large-scale experimental setup benefiting from previous real-world fieldwork experience. Across all 58,320 embedded vocalizing killer whale events, subject to various hydrophone array geometries, call types, distances, and noise conditions responsible for a signal-to-noise ratio varying from −14.2 dB to 3 dB, a detection rate of 94.0 % was achieved with an average localization error of 7.01∘. ORCA-SPY was field-tested on Lake Stechlin in Brandenburg Germany under laboratory conditions with a focus on localization. During the field test, 3889 localization events were observed with an average error of 29.19∘ and a median error of 17.54∘. ORCA-SPY was deployed successfully during the DeepAL fieldwork 2022 expedition (DLFW22) in Northern British Columbia, with a mean average error of 20.01∘ and a median error of 11.01∘ across 503 localization events. ORCA-SPY is an open-source and publicly available software framework, which can be adapted to various recording conditions as well as animal species

Geolocation of a Known Altitude Target Using TDOA and GROA in the Presence of Receiver Location Uncertainty

This paper considers the problem of geolocating a target on the Earth surface using the target signal time difference of arrival (TDOA) and gain ratio of arrival (GROA) measurements when the receiver positions are subject to random errors. The geolocation Cramer-Rao lower bound (CRLB) is derived and the performance improvement due to the use of target altitude information is quantified. An algebraic geolocation solution is developed and its approximate efficiency under small Gaussian noise is established analytically. Its sensitivity to the target altitude error is also studied. Simulations justify the validity of the theoretical developments and illustrate the good performance of the proposed geolocation method

Dual-Satellite Source Geolocation with Time and Frequency Offsets and Satellite Location Errors

This paper considers locating a static source on Earth using the time difference of arrival (TDOA) and frequency difference of arrival (FDOA) measurements obtained by a dual-satellite geolocation system. The TDOA and FDOA from the source are subject to unknown time and frequency offsets because the two satellites are imperfectly time-synchronized or frequency-locked. The satellite locations are not known accurately as well. To make the source position identifiable and mitigate the effect of satellite location errors, calibration stations at known positions are used. Achieving the maximum likelihood (ML) geolocation performance usually requires jointly estimating the source position and extra variables (i.e., time and frequency offsets as well as satellite locations), which is computationally intensive. In this paper, a novel closed-form geolocation algorithm is proposed. It first fuses the TDOA and FDOA measurements from the source and calibration stations to produce a single pair of TDOA and FDOA for source geolocation. This measurement fusion step eliminates the time and frequency offsets while taking into account the presence of satellite location errors. The source position is then found via standard TDOA-FDOA geolocation. The developed algorithm has low complexity and performance analysis shows that it attains the Cramér-Rao lower bound (CRLB) under Gaussian noises and mild conditions. Simulations using a challenging scenario with a short-baseline dual-satellite system verify the theoretical developments and demonstrate the good performance of the proposed algorithm

Majorization-Minimization based Hybrid Localization Method for High Precision Localization in Wireless Sensor Networks

This paper investigates the hybrid source localization problem using the four radio measurements - time of arrival (TOA), time difference of arrival (TDOA), received signal strength (RSS) and angle of arrival (AOA). First, after invoking tractable approximations in the RSS and AOA models, the maximum likelihood estimation (MLE) problem for the hybrid TOA-TDOA-RSS-AOA data model is derived. Then, in the MLE, which has the least-squares objective, weights determined using the range-based characteristics of the four heterogeneous measurements, are introduced. The resultant weighted least-squares problem obtained, which is non-smooth and non-convex, is solved using the principle of the majorization-minimization (MM), leading to an iterative algorithm that has a guaranteed convergence. The key feature of the proposed method is that it provides a unified framework where localization using any possible merger out of these four measurements can be implemented as per the requirement/application. Extensive numerical simulations are conducted to study the estimation efficiency of the proposed method. The proposed method employing all four measurements is compared against a conventionally used method and also against the proposed method employing only limited combinations of the four measurements. The results obtained indicate that the hybrid localization model improves the localization accuracy compared to the heterogeneous measurements. The integration of different measurements also yields good accuracy in the presence of non-line of sight (NLOS) errors