Hearing the shape of a room

PMCID: PMC3725052The final published version of this article can be found here: www.pnas.org/cgi/doi/10.1073/pnas.130993211

Acoustic Echoes Reveal Room Shape

Imagine that you are blindfolded inside an unknown room. You snap your fingers and listen to the room’s response. Can you hear the shape of the room? Some people can do it naturally, but can we design computer algorithms that hear rooms? We show how to compute the shape of a convex polyhedral room from its response to a known sound, recorded by a few microphones. Geometric relationships between the arrival times of echoes enable us to “blindfoldedly” estimate the room geometry. This is achieved by exploiting the properties of Euclidean distance matrices. Furthermore, we show that under mild conditions, first-order echoes provide a unique description of convex polyhedral rooms. Our algorithm starts from the recorded impulse responses and proceeds by learning the correct assignment of echoes to walls. In contrast to earlier methods, the proposed algorithm reconstructs the full three-dimensional geometry of the room from a single sound emission, and with an arbitrary geometry of the microphone array. As long as the microphones can hear the echoes, we can position them as we want. Besides answering a basic question about the inverse problem of room acoustics, our results find applications in areas such as architectural acoustics, indoor localization, virtual reality and audio forensics

Indoor Localization Solutions for a Marine Industry Augmented Reality Tool

In this report are described means for indoor localization in special, challenging circum-stances in marine industry. The work has been carried out in MARIN project, where a tool based on mobile augmented reality technologies for marine industry is developed. The tool can be used for various inspection and documentation tasks and it is aimed for improving the efficiency in design and construction work by offering the possibility to visualize the newest 3D-CAD model in real environment. Indoor localization is needed to support the system in initialization of the accurate camera pose calculation and auto-matically finding the right location in the 3D-CAD model. The suitability of each indoor localization method to the specific environment and circumstances is evaluated.Siirretty Doriast

On Distributed and Acoustic Sensing for Situational Awareness

Recent advances in electronics enable the development of small-sized, low-cost, low-power, multi-functional sensor nodes that possess local processing capability as well as to work collaboratively through communications. They are able to sense, collect, and process data from the surrounding environment locally. Collaboration among the nodes are enabled due to their integrated communication capability. Such a system, generally referred to as sensor networks are widely used in various of areas, such as environmental monitoring, asset tracking, indoor navigation, etc. This thesis consists of two separate applications of such mobile sensors. In this first part, we study decentralized inference problems with dependent observations in wireless sensor networks. Two separate problems are addressed in this part: one pertaining to collaborative spectrum sensing while the other on distributed parameter estimation with correlated additive Gaussian noise. In the second part, we employ a single acoustic sensor with co-located microphone and loudspeaker to reconstruct a 2-D convex polygonal room shape. For spectrum sensing, we study the optimality of energy detection that has been widely used in the literature. This thesis studies the potential optimality (or sub-optimality) of the energy detector in spectrum sensing. With a single sensing node, we show that the energy detector is provably optimal for most cases and for the case when it is not theoretically optimal, its performance is nearly indistinguishable from the true optimal detector. For cooperative spectrum sensing where multiple nodes are employed, we use a recently proposed framework for distributed detection with dependent observations to establish the optimality of energy detector for several cooperative spectrum sensing systems and point out difficulties for the remaining cases. The second problem in decentralized inference studied in this thesis is to investigate the impact of noise correlation on decentralized estimation performance. For a tandem network with correlated additive Gaussian noises, we establish that threshold quantizer on local observations is optimal in the sense of maximizing Fisher information at the fusion center; this is true despite the fact that subsequent estimators may differ at the fusion center, depending on the statistical distribution of the parameter to be estimated. In addition, it is always beneficial to have the better sensor (i.e. the one with higher signal-to-noise ratio) serve as the fusion center in a tandem network for all correlation regimes. Finally, we identify different correlation regimes in terms of their impact on the estimation performance. These include the well known case where negatively correlated noises benefit estimation performance as it facilitates noise cancellation, as well as two distinct regimes with positively correlated noises compared with that of the independent case. In the second part of this thesis, a practical problem of room shape reconstruction using first-order acoustic echoes is explored. Specifically, a single mobile node, with co-located loudspeaker, microphone and internal motion sensors, is deployed and times of arrival of the first-order echoes are measured and used to recover room shape. Two separate cases are studied: the first assumes no knowledge about the sensor trajectory, and the second one assumes partial knowledge on the sensor movement. For either case, the uniqueness of the mapping between the first-order echoes and the room geometry is discussed. Without any trajectory information, we show that first-order echoes are sufficient to recover 2-D room shapes for all convex polygons with the exception of parallelograms. Algorithmic procedure is developed to eliminate the higher-order echoes among the collected echoes in order to retrieve the room geometry. In the second case, the mapping is proved for any convex polygonal shapes when partial trajectory information from internal motion sensors is available.. A practical algorithm for room reconstruction in the presence of noise and higher order echoes is proposed

Listening to Distances and Hearing Shapes:Inverse Problems in Room Acoustics and Beyond

A central theme of this thesis is using echoes to achieve useful, interesting, and sometimes surprising results. One should have no doubts about the echoes' constructive potential; it is, after all, demonstrated masterfully by Nature. Just think about the bat's intriguing ability to navigate in unknown spaces and hunt for insects by listening to echoes of its calls, or about similar (albeit less well-known) abilities of toothed whales, some birds, shrews, and ultimately people. We show that, perhaps contrary to conventional wisdom, multipath propagation resulting from echoes is our friend. When we think about it the right way, it reveals essential geometric information about the sources--channel--receivers system. The key idea is to think of echoes as being more than just delayed and attenuated peaks in 1D impulse responses; they are actually additional sources with their corresponding 3D locations. This transformation allows us to forget about the abstract \emph{room}, and to replace it by more familiar \emph{point sets}. We can then engage the powerful machinery of Euclidean distance geometry. A problem that always arises is that we do not know \emph{a priori} the matching between the peaks and the points in space, and solving the inverse problem is achieved by \emph{echo sorting}---a tool we developed for learning correct labelings of echoes. This has applications beyond acoustics, whenever one deals with waves and reflections, or more generally, time-of-flight measurements. Equipped with this perspective, we first address the ``Can one hear the shape of a room?'' question, and we answer it with a qualified ``yes''. Even a single impulse response uniquely describes a convex polyhedral room, whereas a more practical algorithm to reconstruct the room's geometry uses only first-order echoes and a few microphones. Next, we show how different problems of localization benefit from echoes. The first one is multiple indoor sound source localization. Assuming the room is known, we show that discretizing the Helmholtz equation yields a system of sparse reconstruction problems linked by the common sparsity pattern. By exploiting the full bandwidth of the sources, we show that it is possible to localize multiple unknown sound sources using only a single microphone. We then look at indoor localization with known pulses from the geometric echo perspective introduced previously. Echo sorting enables localization in non-convex rooms without a line-of-sight path, and localization with a single omni-directional sensor, which is impossible without echoes. A closely related problem is microphone position calibration; we show that echoes can help even without assuming that the room is known. Using echoes, we can localize arbitrary numbers of microphones at unknown locations in an unknown room using only one source at an unknown location---for example a finger snap---and get the room's geometry as a byproduct. Our study of source localization outgrew the initial form factor when we looked at source localization with spherical microphone arrays. Spherical signals appear well beyond spherical microphone arrays; for example, any signal defined on Earth's surface lives on a sphere. This resulted in the first slight departure from the main theme: We develop the theory and algorithms for sampling sparse signals on the sphere using finite rate-of-innovation principles and apply it to various signal processing problems on the sphere