Structure from sound with incomplete data

In this paper, we consider the problem of jointly localizing a microphone array and identifying the direction of arrival of acoustic events. Under the assumption that the sources are in the far field, this problem can be formulated as a constrained low-rank matrix factorization with an unknown column offset. Our focus is on handling missing entries, particularly when the measurement matrix does not contain a single complete column. This case has not received attention in the literature and is not handled by existing algorithms, however it is prevalent in practice. We propose an iterative algorithm that works with pairwise differences between the measurements eliminating the dependence on the unknown offset. We demonstrate state-of-the-art performance both in terms of accuracy and versatility

Self-localization in Ad Hoc Indoor Acoustic Networks

The increasing use of mobile technology in everyday life has aroused interest into developing new ways of utilizing the data collected by devices such as mobile phones and wearable devices. Acoustic sensors can be used to localize sound sources if the positions of spatially separate sensors are known or can be determined. However, the process of determining the 3D coordinates by manual measurements is tedious especially with increasing number of sensors. Therefore, the localization process has to be automated. Satellite based positioning is imprecise for many applications and requires line-of-sight to the sky. This thesis studies localization methods for wireless acoustic sensor networks and the process is called self-localization.This thesis focuses on self-localization from sound, and therefore the term acoustic is used. Furthermore, the development of the methods aims at utilizing ad hoc sensor networks, which means that the sensors are not necessarily installed in the premises like meeting rooms and other purpose-built spaces, which often have dedicated audio hardware for spatial audio applications. Instead of relying on such spaces and equipment, mobile devices are used, which are combined to form sensor networks.For instance, a few mobile phones laid on a table can be used to create a sensor network built for an event and it is inherently dismantled once the event is over, which explains the use of the term ad hoc. Once positions of the devices are estimated, the network can be used for spatial applications such as sound source localization and audio enhancement via spatial filtering. The main purpose of this thesis is to present the methods for self-localization of such an ad hoc acoustic sensor network. Using off-the-shelf ad hoc devices to establish sensor networks enables implementation of many spatial algorithms basically in any environment.Several acoustic self-localization methods have been introduced over the years. However, they often rely on specialized hardware and calibration signals. This thesis presents methods that are passive and utilize environmental sounds such as speech from which, by using time delay estimation, the spatial information of the sensor network can be determined. Many previous self-localization methods assume that audio captured by the sensors is synchronized. This assumption cannot be made in an ad hoc sensor network, since the different sensors are unaware of each other without specific signaling that is not available without special arrangement.The methods developed in this thesis are evaluated with simulations and real data recordings. Scenarios in which the targets of positioning are stationary and in motion are studied. The real world recordings are made in closed spaces such as meeting rooms. The targets are approximately 1 – 5 meters apart. The positioning accuracy is approximately five centimeters in a stationary scenario, and ten centimeters in a moving-target scenario on average. The most important result of this thesis is presenting the first self-localization method that uses environmental sounds and off-the-shelf unsynchronized devices, and allows the targets of self-localization to move

ベイズ法によるマイクロフォンアレイ処理

京都大学0048新制・課程博士博士(情報学)甲第18412号情博第527号新制||情||93(附属図書館)31270京都大学大学院情報学研究科知能情報学専攻(主査)教授 奥乃 博, 教授 河原 達也, 准教授 CUTURI CAMETO Marco, 講師 吉井 和佳学位規則第4条第1項該当Doctor of InformaticsKyoto UniversityDFA

Inter-Node Distance Estimation from Multipath Delay Differences of Channels to Observer Nodes

We study the estimation of distance d between two wireless nodes by means of their wideband channels to a third node, called observer. The motivating principle is that the channel impulse responses are similar for small d and drift apart when d increases. Following this idea we propose specific distance estimators based on the differences of path delays of the extractable multipath components. In particular, we derive such estimators for rich multipath environments and various important cases: with and without clock synchronization as well as errors on the extracted path delays (e.g. due to limited bandwidth). The estimators readily support (and benefit from) the presence of multiple observers. We present an error analysis and, using ray tracing in an exemplary indoor environment, show that the estimators perform well in realistic conditions. We describe possible localization applications of the proposed scheme and highlight its major advantages: it requires neither precise synchronization nor line-of-sight connection. This could make wireless user tracking feasible in dynamic indoor settings.Comment: To appear at IEEE ICC 2019. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Scalable Tactile Sensing E-Skins Through Spatial Frequency Encoding

Most state-of-the-art tactile sensing arrays are not scalable to large numbers of sensing units due to their raster-scanned readout. This readout scheme results in a high degree of wiring complexity and a tradeoff between spatial and temporal resolution. In this thesis I present the use of spatial frequency encoding to develop asynchronous tactile sensor arrays with single-wire sensor transduction, no per-taxel electronics, and no scanning latency. I demonstrate this through two prototype devices, Neuroskin 1, which is developed using fabric-based e-textile materials, and Neuroskin 2, which is developed using fPCB. Like human skin, Neuroskin has a temporal resolution of 1 kHz and innate data compression where tactile data from an MxN Neuroskin is compressed into M+N values. Neuroskin 2 requires only four interface wires (regardless of its number of sensors) and can be easily scaled up through its development as an fPCB. To demonstrate the utility of the prototypes, Neuroskin was mounted onto a biomimetic robotic finger to palpate different textures and perform a texture discrimination task. Neuroskin 1 and 2 achieved 87% and 76% classification accuracy respectively in the texture discrimination task. Overall, the method of spatial-frequency encoding is theoretically scalable to support sensor arrays with thousands of sensing elements without latency, and the resolution of a Neuroskin array is only limited by the ADC sampling rate. Future tactile sensing systems can utilize the spatial frequency encoding architecture presented here to be dense, numerous, and flexible while retaining excellent temporal resolution

Scalable positioning of commodity mobile devices using audio signals

This thesis explores the problem of computing a position map for co-located mobile devices. The positioning should happen in a scalable manner without requiring specialized hardware and without requiring specialized infrastructure (except basic Wi-Fi or cellular access). At events like meetings, talks, or conferences, a position map can aid spontaneous communication among users based on their relative position in two ways. First, it enables users to choose message recipients based on their relative position, which also enables the position-based distribution of documents. Second, it enables senders to attach their position to messages, which can facilitate interaction between speaker and audience in a lecture hall and enables the collection of feedback based on users’ location. In this thesis, we present Sonoloc, a mobile app and system that, by relying on acoustic signals, allows a set of commodity smart devices to determine their relative positions. Sonoloc can position any number of devices within acoustic range with a constant number of acoustic signals emitted by a subset of devices. Our experimental evaluation with up to 115 devices in real rooms shows that – despite substantial background noise – the system can locate devices with an accuracy of tens of centimeters using no more than 15 acoustic signals.Diese Dissertation befasst sich mit dem Problem, eine Positionskarte von sich am gleichen Ort befindenden mobilen Geräten zu berechnen. Dies soll skalierbar, ohne Verwendung von spezialisierter Hardware oder Infrastruktur (ausgenommen einfache WLAN- oder Mobilfunkzugang) erfolgen. Bei Veranstaltungen wie Meetings, Diskussionen oder Konferenzen kann eine Positionskarte die Benutzer bei spontaner Kommunikation mithilfe der relativen Positionen in zweierlei Hinsicht unterstützen. Erstens ermöglicht sie den Benutzern, die Empfänger von Nachrichten aufgrund deren Position zu wählen, was auch eine positionsabhängige Verteilung von Unterlagen erlaubt. Zweitens ermöglicht sie den Sendern, ihre Position in die Nachrichten zu integrieren, was eine Interaktion zwischen Referent und Zuhörer in einem Hörsaal und die Sammlung von positionsabhängigen Rückmeldungen erlaubt. In dieser Dissertation stellen wir die Mobile-App und das System Sonoloc vor, das mithilfe von Tonsignalen erlaubt, die relative Position handelsüblicher, intelligenter Geräte zu bestimmen. Sonoloc kann eine beliebige Zahl von Geräten innerhalb des Hörbereichs durch eine gleichbleibende Zahl von Tonsignalen, die von einer Teilmenge der Geräte gesendet werden, lokalisieren. Unsere experimentelle Analyse mit bis zu 115 Geräten in echten Räumen zeigt, dass das System trotz signifikanter Hintergrundgeräusche unter Verwendung von bis zu 15 Tonsignalen mit einer Genauigkeit von wenigen Dezimetern Geräte lokalisieren kann.This work was supported in part by the European Research Council (ERC Synergy imPACT 610150), the German Science Foundation (DFG CRC 1223), the Japan Society for the Promotion of Science (Grant-in-Aid for Scientific Research (A), KAKENHI Grant Number 16H01735), and the National Science Foundation (NSF Awards CNS 1526635 and CNS 1314857)

Models for learning reverberant environments

Reverberation is present in all real life enclosures. From our workplaces to our homes and even in places designed as auditoria, such as concert halls and theatres. We have learned to understand speech in the presence of reverberation and also to use it for aesthetics in music. This thesis investigates novel ways enabling machines to learn the properties of reverberant acoustic environments. Training machines to classify rooms based on the effect of reverberation requires the use of data recorded in the room. The typical data for such measurements is the Acoustic Impulse Response (AIR) between the speaker and the receiver as a Finite Impulse Response (FIR) filter. Its representation however is high-dimensional and the measurements are small in number, which limits the design and performance of deep learning algorithms. Understanding properties of the rooms relies on the analysis of reflections that compose the AIRs and the decay and absorption of the sound energy in the room. This thesis proposes novel methods for representing the early reflections, which are strong and sparse in nature and depend on the position of the source and the receiver. The resulting representation significantly reduces the coefficients needed to represent the AIR and can be combined with a stochastic model from the literature to also represent the late reflections. The use of Finite Impulse Response (FIR) for the task of classifying rooms is investigated, which provides novel results in this field. The aforementioned issues related to AIRs are highlighted through the analysis. This leads to the proposal of a data augmentation method for the training of the classifiers based on Generative Adversarial Networks (GANs), which uses existing data to create artificial AIRs, as if they were measured in real rooms. The networks learn properties of the room in the space defined by the parameters of the low-dimensional representation that is proposed in this thesis.Open Acces