Bodenmüller, Tim

Sewtz, Marco

Triebel, Rudolph

Institute of Transport Research:Publications

Design of a Microphone Array for Rollin’ JustinMarco Sewtz Tim Bodenmüller Rudolph TriebelI. INTRODUCTIONFor a humanoid robot operating in populated environments it isa key competence to interact with humans naturally and intuitively.Therefore, research in human-robot interaction explores the interpre-tation of visual, auditive and even tactile sensing modalities. Oneimportant aspect here is to recognize robustly the intention of thehuman interacting with the robot. To achieve this, robot audition isa suitable modality, as it allows for detecting and tracking speakersfrom arbitrary positions around the robot and also from distant places.In this paper we present the design of a microphone array for the headof our humanoid robot Rollin’ Justin that allows us to localize andtrack sound sources within a certain distance to the robot. Until now,high-level interfacing to the robot was only possible using a tabletor by verbal communication via a headset using speech recognition.However, neither method allows to localize the operator. With themicrophone array in the head we can do sound source localizationand then re-position the head sensors towards the speaker, enablingadvanced interaction possibilities.In recent years, the research in the field of service robotics focuseson human-centered interfaces. This includes easy-to-use as well aseasy-to-understand systems like robots with audio input [?]. Severalsystems have been developed using microphone arrays to extractspeech [?], [?]. More complex systems use multiple techniques forprocessing including sound source localization, feature extraction andspeech-to-text engines [?].In the following we present the design of a microphone array forRollin’ Justin for sound source localization and speech recognition.First we describe the considered household scenario and presentan overview of the proposed processing system. We than discussour design considerations regarding the microphone array as wellas possible sound source localization and speech processing. Weconclude with a description of system integration.II. DESIGNFor the design of our microphone array we consider the followingscenario: our robot is located in a typical indoor environment, forinstance an apartment (Figure 1). This implies that we have multiplesources of noise, e.g. a fridge, and reverberations. In addition, wesuppose that there is only one person at a time speaking to the robot,called the operator. The expected distance r between the robot andthe operator is between 1 m and 4 m. We assume that, from therobot’s point of view the operator and any other sound source have aminimum tangential distance of at least dmin = 1 m. This results ina minimum angular distance θmin = arctan(dmin/rmax) ≈ ±14◦separating the speech from any other sound source. Figure 2billustrates this scenario.A. System OverviewWe plan our system as illustrated in Figure 2a. The sound fromthe sound sources is received and sampled by a microphone array. InAll authors are with: Institute of Robotics and Mechatronics,German Aerospace Center (DLR), Wessling, Germany. Email:marco.sewtz@dlr.de, tim.bodenmueller@dlr.de,rudolph.triebel@dlr.deFigure 1: Lab environment imitating a typical small apartment(a)(b)Figure 2: (left) System overview and (right) illustration of assumedaudio scenario.a preprocessing step we remove ambient noise and the robot’s ego-noise. Then, we localize the sound source by calculating the directionand distance of the source. This is used by subsequent beam formingto improve the signal for speech recognition.B. Microphone ArrayFor the design of the microphone array, we consider the expectedsound signals, the processing requirements, as well as geometricallimitations. In general, we follow the approach of a broadbandmicrophone sub-array [?]. Hence, we divide the frequency spectruminto three sub-bands (< 1 kHz, 1-2 kHz, > 2 kHz), each handledby a specific sub-array. For lower frequencies, we need large dis-tances between microphones to capture longer delays. For higherfrequencies, we have to use a smaller spacing. However, the range offeasible distances between the microphones is small due to limitedspace within the robot’s face mask. Also, they have to be placed ona single plane because of limited space and appearance constraints.In total we use eight microphones that are arranged as shown inFigure 4b and grouped into sub-arrays as shown in Figure 4a. Theouter microphones have a distance of 146 mm. To improve the speechquality [?], we sample with at least 16 kHz instead of the usual 8 kHz.The signals are bandpass filtered, amplified and summed up to thecomplete signal. We optimized the microphone positions by a free-air simulation. The resulting combined directivity pattern is shown inFigure 3. The main lobe of the array is focused around our defined14◦ corridor at -3 dB, and sources at larger angles are suppressed. Inthe lower frequency range, the lobe is somewhat less focused due toa limited maximum inter-signal delay.Figure 3: Directivity pattern for the combined array approach. Thefigure shows the pattern truncated at -3 dB. The dashed red lineillustrates a ±14◦ corridor.(a) Sub-Array concept(b) CAD drawing of the proposedmicrophone positionsFigure 4: (left) Illustration of the sub-array approach, (right) CADdrawing of the microphones on the robot’s face.C. Sound Source LocalizationFor localization, we only use signal snippets that contain speech,which we identify by means of the Long-Term Speech Diver-gence [?]. For an accurate and robust localization from these snippetswe use a modified version of the MUSIC algorithm [?].The main principle of the estimation process relies on the delaybetween the received signals. In case of a linear array, the delay isproportional to the spacing between the microphones. For our 2Darray design, the delay ∆ti between microphone i and a referencepoint assuming a signal from direction θ is given by the projectiononto the direction vector of the arriving sound wave, i.e.∆ti = (p>i eθ)/c0 ,where pi is the position of the i-th microphone with respect to thereference point, eθ the direction unit vector of the sound wave andc0 ≈ 343 m/s the speed of sound. Figure 5 illustrates this relation.D. Further ProcessingWith the direction of the sound source, we steer our system towardsthe source using a delay-and-sum beamformer in combination withour sub-array approach. We expect a high increase in signal-to-noiseratio as well as noise suppression from this technique. The retrievedspeech signal will be used to extract commands from the operator.We will use already available offline speech processing engines suchas CMU Sphinx [?], which has already been used with our headset.We also plan to adapt voice assistant paradigms as in the NAOMIProject [?].Figure 5: Illustration of the delay calculation for an arriving soundwave. The intersections of the dashed lines show the projection ontothe direction vector ~eθ . The red dot represents the reference point.III. INTEGRATIONFor the array we chose SPH0645LM4H-8 MEMS microphoneswith I2S support1. They will be sampled simultaneously by the nativeI2S ports of an Nvidia Jetson TX2 board mounted onto an AuvideaJ140 carrier. The whole system will be integrated physically into thehead of the robot, and the software will be part of our “Links andNodes” framework.IV. CONCLUSIONWe presented the design of a microphone array for sound sourcelocalization and speech processing on the humanoid robot Rollin’Justin. We introduced our overall system architecture, gave anoverview of the subsequent sound processing, and described designconsiderations, implementation details and preliminary evaluations.In a next step, we will finalize the implementation on the robot andexecute several experiments in realistic scenarios.V. ACKNOWLEDGMENTWe want to thank Werner Friedl for his contributions to themechanical design and implementation.1https://www.knowles.com/

Design of a Microphone Array for Rollin Justin

https://elib.dlr.de/137626/1/sound_source_loc_for_hum_robots.pdf

Design of a Microphone Array for Rollin Justin

Abstract

Similar works

Full text

Available Versions

Institute of Transport Research:Publications