Pyroomacoustics: A Python package for audio room simulations and array processing algorithms

We present pyroomacoustics, a software package aimed at the rapid development and testing of audio array processing algorithms. The content of the package can be divided into three main components: an intuitive Python object-oriented interface to quickly construct different simulation scenarios involving multiple sound sources and microphones in 2D and 3D rooms; a fast C implementation of the image source model for general polyhedral rooms to efficiently generate room impulse responses and simulate the propagation between sources and receivers; and finally, reference implementations of popular algorithms for beamforming, direction finding, and adaptive filtering. Together, they form a package with the potential to speed up the time to market of new algorithms by significantly reducing the implementation overhead in the performance evaluation step.Comment: 5 pages, 5 figures, describes a software packag

FRIDA: FRI-Based DOA Estimation for Arbitrary Array Layouts

In this paper we present FRIDA---an algorithm for estimating directions of arrival of multiple wideband sound sources. FRIDA combines multi-band information coherently and achieves state-of-the-art resolution at extremely low signal-to-noise ratios. It works for arbitrary array layouts, but unlike the various steered response power and subspace methods, it does not require a grid search. FRIDA leverages recent advances in sampling signals with a finite rate of innovation. It is based on the insight that for any array layout, the entries of the spatial covariance matrix can be linearly transformed into a uniformly sampled sum of sinusoids.Comment: Submitted to ICASSP201

A Modular and Robust Physics-Based Approach for Lensless Image Reconstruction

In this paper, we present a modular approach for reconstructing lensless measurements. It consists of three components: a newly-proposed pre-processor, a physics-based camera inverter to undo the multiplexing of lensless imaging, and a post-processor. The pre- and post-processors address noise and artifacts unique to lensless imaging before and after camera inversion respectively. By training the three components end-to-end, we obtain a 1.9 dB increase in PSNR and a 14% relative improvement in a perceptual image metric (LPIPS) with respect to previously proposed physics-based methods. We also demonstrate how the proposed pre-processor provides more robustness to input noise, and how an auxiliary loss can improve interpretability.Comment: Demo notebook: https://go.epfl.ch/lensless-modula

Teaching Practical DSP with Off-the-Shelf Hardware and Free Software

In this paper we describe our approach to teaching applied digital signal processing (DSP) using freely-available software and inexpensive off-the-shelf hardware components. The pedagogical framework is built around simple real-time audio processing algorithms that provide immediate and engaging feedback to the students. At the same time, our end goal is to build a learning module that can be easily reproduced by fellow instructors and used in our Massive Open Online Course (MOOC). We use Python for the initial algorithmic prototyping and then transition to a hardware implementation using an STMicroelectronics' Nucleo-64 core with Adafruit I/O breakout boards. We employ modern documentation tools based on Git and Markdown in order to share the material and the exercises, allowing students and educators to access, reproduce, and contribute changes to the content

Sparse Recovery of Strong Reflectors With an Application to Non-Destructive Evaluation

In this paper we show that it is sufficient to recover the locations of K strong reflectors within an insonified medium from three receive elements and 2K+1 samples per element. The proposed approach leverages advances in sampling signals with a finite rate of innovation along each element and rank properties from the Euclidean distance matrix construction across elements. With the proposed approach, it is not necessary to construct an image in order to identify strong reflective sources, which is why much fewer receive elements are needed. However, the assumed transmit scheme still uses a standard linear array in order to excite the entire medium with sufficient energy. The approach is validated with simulated data and a measurement that emulates a scenario in non-destructive evaluation

Hardware And Software For Reproducible Research In Audio Array Signal Processing

In our demo, we present two hardware platforms for prototyping audio array signal processing. Pyramic is a 48-channel microphone array fitted on an FPGA and Compact Six is a portable microphone array with six microphones, closer to the technical constraints of consumer electronics. A browser based interface was developed that allows the user to interact with the audio stream from the arrays in real time. The software component of this demo is a Python module with implementations of basic audio signal processing blocks and popular techniques like STFT, beamforming, and DoA. Both the hardware design files and the software are open source and freely shared. As part of a collaboration with IBM Research, their beamforming and imaging technologies will also be portrayed. The hardware will be demonstrated through an installation processing the microphone signals into light patterns on a circular LED array. The demo will be interactive and let visitors play with different algorithms for DoA (SRP, FRIDA [1], Bluebild) and beamforming (MVDR, Flexibeam [2]). The availability of an open platform with reference implementations encourages reproducible research and minimizes setup-time when testing and benchmarking new audio array signal processing algorithms. It can also serve as a useful educational tool, providing a means to work with real-life signals

Morphological component analysis for sparse regularization in plane wave imaging

Classical ultrasound image reconstruction mainly relies on the well-known delay-and-sum (DAS) beamforming for its simplicity and real-time capability. Sparse regularization methods propose an alternative to DAS which lead to a better inversion of the ill-posed problem resulting from the acoustic wave propagation. In the following work, a new sparse regularization method is proposed which includes a component-based modelling of the radio-frequency images as well as a pointspread- function-adaptive sparsity prior. The proposed method, evaluated on the PICMUS dataset,outperforms the classical DAS in terms of contrast and resolution

Privacy-Enhancing Optical Embeddings for Lensless Classification

Lensless imaging can provide visual privacy due to the highly multiplexed characteristic of its measurements. However, this alone is a weak form of security, as various adversarial attacks can be designed to invert the one-to-many scene mapping of such cameras. In this work, we enhance the privacy provided by lensless imaging by (1) downsampling at the sensor and (2) using a programmable mask with variable patterns as our optical encoder. We build a prototype from a low-cost LCD and Raspberry Pi components, for a total cost of around 100 USD. This very low price point allows our system to be deployed and leveraged in a broad range of applications. In our experiments, we first demonstrate the viability and reconfigurability of our system by applying it to various classification tasks: MNIST, CelebA (face attributes), and CIFAR10. By jointly optimizing the mask pattern and a digital classifier in an end-to-end fashion, low-dimensional, privacy-enhancing embeddings are learned directly at the sensor. Secondly, we show how the proposed system, through variable mask patterns, can thwart adversaries that attempt to invert the system (1) via plaintext attacks or (2) in the event of camera parameters leaks. We demonstrate the defense of our system to both risks, with 55% and 26% drops in image quality metrics for attacks based on model-based convex optimization and generative neural networks respectively. We open-source a wave propagation and camera simulator needed for end-to-end optimization, the training software, and a library for interfacing with the camera