Data-driven Auditory Contrast Enhancement for Everyday Sounds and Sonifications

Hermann T, Weger M. Data-driven Auditory Contrast Enhancement for Everyday Sounds and Sonifications. In: Proceedings of the 25th International Conference on Auditory Display (ICAD 2019). Newcastle: ICAD; 2019.We introduce Auditory Contrast Enhancement (ACE) as a technique to enhance sounds at hand of a given collection of sound or sonification examples that belong to different classes, such as sounds of machines with and without a certain malfunction, or medical data sonifications for different pathologies/conditions. A frequent use case in inductive data mining is the discovery of patterns in which such groups can be discerned, to guide subsequent paths for modelling and feature extraction. ACE provides researchers with a set of methods to render focused auditory perspectives that accentuate inter-group differences and in turn also enhance the intra-group similarity, i.e, it warps sounds so that our human built-in metrics for assessing differences between sounds is better aligned to systematic differences between sounds belonging to different classes. We unfold and detail the concept along three different lines: temporal, spectral and spectrotemporal auditory contrast enhancement and we demonstrate their performance at hand of given sound and sonification collections

AltAR/table: A Platform for Plausible Auditory Augmentation

Presented at the 27th International Conference on Auditory Display (ICAD 2022) 24-27 June 2022, Virtual conference.Auditory feedback from everyday interactions can be augmented to project digital information in the physical world. For that purpose, auditory augmentation modulates irrelevant aspects of already existing sounds while at the same time preserving relevant ones. A strategy for maintaining a certain level of plausibility is to metaphorically modulate the physical object itself. By mapping information to physical parameters instead of arbitrary sound parameters, it is assumed that even untrained users can draw on prior knowledge. Here we present AltAR/table, a hard- and software platform for plausible auditory augmentation of flat surfaces. It renders accurate augmentations of rectangular plates by capturing the structure-borne sound, feeding it through a physical sound model, and playing it back through the same object in real time. The implementation solves basic problems of equalization, active feedback control, spatialization, hand tracking, and low-latency signal processing. AltAR/table provides the technical foundations of object-centered auditory augmentations, for embedding sonifications into everyday objects such as tables, walls, or floors

Plausible Auditory Augmentation of Physical Interaction

Weger M, Hermann T, Höldrich R. Plausible Auditory Augmentation of Physical Interaction. In: Proceedings of the 24th International Conference on Auditory Display. Sonification as ADSR. (ICAD 2018). Michigan: ICAD; 2018: 97-104.Interactions with physical objects usually evoke sounds, i.e., audi-tory feedback that depends on the interacting objects (e.g., table,hand, or pencil) and interaction type (e.g., tapping or scratching).The continuous real-time adaptation of sound during interactionenables the manipulation/refinement of perceived characteristics(size, material) of physical objects. Furthermore, when controlledby unrelated external data, the resulting ambient sonifications cankeep users aware of changing data. This article introduces the con-cept ofplausibilityto the topic of auditory augmentations of phys-ical interactions, aiming at providing an experimentation platformfor investigating surface-based physical interactions, understand-ing relevant acoustic cues, redefining these via auditory augmenta-tion / blended sonification and particularly to empirically measurethe plausibility limits of such auditory augmentations. Besidesconceptual contributions along the trade-off between plausibilityand usability, a practical experimentation system is introduced, to-gether with a very first qualitative pilot study

Real-time Auditory Contrast Enhancement

Weger M, Hermann T, Höldrich R. Real-time Auditory Contrast Enhancement. In: Proceedings of the 25th International Conference on Auditory Display (ICAD 2019). Newcastle: International Community for Auditory Display (ICAD); In Press.Every day, we rely on the information that is encoded in the auditory feedback of our physical interactions. With the goal to perceptually enhance those sound characteristics that are relevant to us -- especially within professional practices such as percussion and auscultation, we introduce the method of real-time Auditory Contrast Enhancement (ACE). The method is derived from algorithms for speech enhancement as well from the remarkable sound processing mechanisms of our ears. ACE is achieved by individual sharpening of spectral as well as temporal structures contained in a sound. It is designed for real-time application with low latency. The discussed examples illustrate that the proposed algorithms are able to significantly enhance spectral and temporal contrast. ### Sound examples and interaction examples #### **ACE of Impact Sounds** ##### **S1.0-xxx.wav** see article for further details

Data-driven auditory contrast enhancement for everyday sounds and sonifications

Presented at the 25th International Conference on Auditory Display (ICAD 2019) 23-27 June 2019, Northumbria University, Newcastle upon Tyne, UK.We introduce Auditory Contrast Enhancement (ACE) as a technique to enhance sounds at hand of a given collection of sound or sonification examples that belong to different classes, such as sounds of machines with and without a certain malfunction, or medical data sonifications for different pathologies/conditions. A frequent use case in inductive data mining is the discovery of patterns in which such groups can be discerned, to guide subsequent paths for modelling and feature extraction. ACE provides researchers with a set of methods to render focussed auditory perspectives that accentuate inter-group differences and in turn also enhance the intra-group similarity, i.e. it warps sounds so that our human built-in metrics for assessing differences between sounds is better aligned to systematic differences between sounds belonging to different classes. We unfold and detail the concept along three different lines: temporal, spectral and spectrotemporal auditory contrast enhancement and we demonstrate their performance at hand of given sound and sonification collections

Recognizability and perceived urgency of bicycle bells

Raising awareness about how alarm sounds are perceived and evaluated by an individual in traffic scenery is important for developing new alarm designs, as well as for improving existing ones. Bearing a positive contribution to road safety, cyclists and pedestrians especially can benefit from appropriate alarming bell and horn sounds. Primarily, the alarm signal should evoke a precise idea of what is the source of the warning and the desired reaction to it. Furthermore, it should not be masked by other noises thus going undetected by the ear. Finally, an appropriate warning signal should transmit the urgency of a given situation, while at the same time, it should not cause other road users and pedestrians to startle. In two listening experiments, we examined the perception of commonly available bicycle bells and horns. Average typicality or recognizability as a bicycle bell among other everyday sounds has been investigated through a free identification task. In a second experiment, we tested perceived urgency of the warning sounds in relation to traffic noise. This article further provides a survey on non-verbal alarm design, as well as an analysis of acoustic properties of common bicycle bells and horns. Consequently, a linear regression model presents the relationship between named properties and perceived urgency. It is our intention to give an insight into the often unattended but important issue of the perception of auditory warning sounds in our everyday acoustic environment

Plausible auditory augmentation of physical interaction

Interactions with physical objects usually evoke sounds, i.e., auditory feedback that depends on the interacting objects (e.g., table, hand, or pencil) and interaction type (e.g., tapping or scratching). The continuous real-time adaptation of sound during interaction enables the manipulation/refinement of perceived characteristics (size, material) of physical objects. Furthermore, when controlled by unrelated external data, the resulting ambient sonifications can keep users aware of changing data. This article introduces the concept of plausibility to the topic of auditory augmentations of physical interactions, aiming at providing an experimentation platform for investigating surface-based physical interactions, understanding relevant acoustic cues, redefining these via auditory augmentation / blended sonification and particularly to empirically measure the plausibility limits of such auditory augmentations. Besides conceptual contributions along the trade-off between plausibility and usability, a practical experimentation system is introduced, together with a very first qualitative pilot study

Real-time auditory contrast enhancement

Presented at the 25th International Conference on Auditory Display (ICAD 2019) 23-27 June 2019, Northumbria University, Newcastle upon Tyne, UK.Every day, we rely on the information that is encoded in the auditory feedback of our physical interactions. With the goal to perceptually enhance those sound characteristics that are relevant to us - especially within professional practices such as percussion and auscultation - we introduce the method of real-time Auditory Contrast Enhancement (ACE). It is derived from algorithms for speech enhancement as well as from the remarkable sound processing mechanisms of our ears. ACE is achieved by individual sharpening of spectral and temporal structures contained in a sound while maintaining its natural gestalt. With regard to the targeted real-time applications, the proposed method is designed for low latency. As the discussed examples illustrate, it is able to significantly enhance spectral and temporal contrast

Supplementary Material for "Data-driven Auditory Contrast Enhancement for Everyday Sounds and Sonifications"

Hermann T, Weger M. Supplementary Material for "Data-driven Auditory Contrast Enhancement for Everyday Sounds and Sonifications". Bielefeld University; 2019.We introduce Auditory Contrast Enhancement (ACE) as a technique to enhance sounds at hand of a given collection of sound or sonification examples that belong to different classes, such as sounds of machines with and without a certain malfunction, or medical data sonifications for different pathologies/conditions. A frequent use case in inductive data mining is the discovery of patterns in which such groups can be discerned, to guide subsequent paths for modelling and feature extraction. ACE provides researchers with a set of methods to render focused auditory perspectives that accentuate inter-group differences and in turn also enhance the intra-group similarity, i.e, it warps sounds so that our human built-in metrics for assessing differences between sounds is better aligned to systematic differences between sounds belonging to different classes. We unfold and detail the concept along three different lines: temporal, spectral and spectrotemporal auditory contrast enhancement and we demonstrate their performance at hand of given sound and sonification collections. #### **ACE of Impact Sounds: table interaction sounds** ##### **S1.0-table-stimuli.wav** see article for further details. ##### **S1.1-table__spectral__mode=median_filtered_ts__order=2.5__medfilt=9__nlgain=0.3.wav** see article for further details. ##### **S1.2-table__spectral__mode=tanh_gained_ts__order=6.25__medfilt=25__nlgain=0.16.wav** see article for further details. ##### **S1.3-table__spectral__mode=tanh_gained_ts__order=26.75__medfilt=17__nlgain=0.17.wav** see article for further details. ##### **S1.4-table__spectral__mode=db_median_filtered_ps__order=47.5__medfilt=25__nlgain=0.3.wav** see article for further details. ##### **S1.5-table__temporal__mode=median_filtered_ts__order=3.5__medfilt=5__nlgain=0.3.wav** see article for further details. ##### **S1.6-table__spectrotemporal__mode=only_highest_ts__ts_threshold=3.311__sigma=0.316__mix=1.0.wav** see article for further details. ##### **S1.7-table__spectrotemporal__mode=only_highest_ts__ts_threshold=13.18__sigma=0.316__mix=1.0.wav** see article for further details. ##### **S1.8-table__spectrotemporal__mode=blurred_ts_mask__ts_threshold=6.0256__sigma=0.331__mix=1.0.wav** see article for further details. ##### **S1.9-table__spectrotemporal__mode=blurred_ts_mask__ts_threshold=6.0256__sigma=0.871__mix=0.998.wav** see article for further details. #### **ACE of Impact Sounds: finger snap interaction sounds** ##### **S2.0-snap-stimuli.wav** see article for further details. ##### **S2.1-snap__spectral__mode=median_filtered_ts__order=1.0__medfilt=1__nlgain=0.3.wav** see article for further details. ##### **S2.2-snap__spectral__mode=tanh_gained_ts__order=4.5__medfilt=1__nlgain=0.3.wav** see article for further details. ##### **S2.3-snap__spectral__mode=tanh_gained_ts__order=4.5__medfilt=1__nlgain=0.12.wav** see article for further details. ##### **S2.4-snap__spectral__mode=median_filtered_ts__order=4.5__medfilt=1__nlgain=0.3.wav** see article for further details. ##### **S2.5-snap__spectrotemporal__mode=only_highest_ts__ts_threshold=3.02__sigma=0.316__mix=1.0.wav** see article for further details. #### **ACE of synthesized sounds: SuperCollider Klank plus Noise** ##### **S3.0-klanks-stimuli.wav** see article for further details. ##### **S3.1-klanks__spectral__mode=median_filtered_ts__medfilt=15__order=1.75__nlgain=0.0.wav** see article for further details. ##### **S3.2-klanks__spectral__mode=tanh_gained_ts__order=3.0__medfilt=9__nlgain=0.21.wav** see article for further details. ##### **S3.3-klanks__spectral__mode=tanh_gained_ts__medfilt=27__order=7.0__nlgain=0.23.wav** see article for further details. ##### **S3.4-klanks__spectral__mode=median_filtered_ts__order=9.0__medfilt=1__nlgain=0.04.wav** see article for further details. ##### **S3.5-klanks__spectrotemporal__mode=blurred_ts_mask__ts_threshold=0.832__sigma=0.661__mix=0.998.wav** see article for further details. #### **ACE of continuous sounds: Vowel sounds class a vs. ä** ##### **S4.0-vowel-stimuli.wav** see article for further details. ##### **S4.1-vowel__spectral__mode=tanh_gained_ts__order=4.25__medfilt=39__nlgain=0.09.wav** see article for further details. ##### **S4.2-vowel__spectral__mode=median_filtered_ts__order=10.75__medfilt=39__nlgain=0.01.wav** see article for further details. ##### **S4.3-vowel__spectrotemporal__mode=only_highest_ts__ts_threshold=2.291__sigma=0.912__mix=1.0.wav** see article for further details. ##### **S4.4-vowel-a-ae-transition-orig.wav** see article for further details. ##### **S4.5-vowel-a-ae-transition-aced.wav** see article for further details. #### **ACE of time series sonifications: daily energy consumption profiles of a building** ##### **S5.0-building-stimuli.wav** see article for further details. ##### **S5.1-building__spectral__mode=median_filtered_ts__medfilt=5__order=1.0__nlgain=0.01.wav** see article for further details. ##### **S5.2-building__spectral__mode=tanh_gained_ts__medfilt=3__order=0.5__nlgain=0.01.wav** see article for further details. ##### **S5.3-building__temporal__mode=median_filtered_ts__medfilt=7__order=2.75__nlgain=0.3.wav** see article for further details. ##### **S5.4-building__spectrotemporal__mode=only_highest_ts__ts_threshold=3.981__sigma=0.603__mix=0.995.wav** see article for further details. #### **ACE of manipulated music recordings: Temporal ACE gives condensed difference thumbnails** ##### **S6.0-miles-stimuli.wav** see article for further details. ##### **S6.1-miles__temporal__mode=median_filtered_ts__order=0.5__medfilt=7__nlgain=0.3.wav** see article for further details. ##### **S6.2-miles__temporal__mode=tanh_gained_ts__order=0.5__medfilt=7__nlgain=0.12.wav** see article for further details

Supplementary Material for "Real-time Auditory Contrast Enhancement"

Weger M, Hermann T, Höldrich R. Supplementary Material for "Real-time Auditory Contrast Enhancement". Bielefeld University; 2019.Every day, we rely on the information that is encoded in the auditory feedback of our physical interactions. With the goal to perceptually enhance those sound characteristics that are relevant to us – especially within professional practices such as percussion and auscultation – we introduce the method of real-time Audi- tory Contrast Enhancement (ACE). It is derived from algorithms for speech enhancement as well as from the remarkable sound processing mechanisms of our ears. ACE is achieved by individual sharpening of spectral and temporal structures contained in a sound while maintaining its natural gestalt. With regard to the targeted real-time applications, the proposed method is designed for low latency. As the discussed examples illustrate, it is able to significantly enhance spectral and temporal contrast. ### Sound examples #### **ACE of impact Sounds** ##### **S1.1_knock-knock.wav** see article for further details. ##### **S1.2_knock-knock_LI.wav** see article for further details. ##### **S1.3_knock-knock_LI+EX.wav** see article for further details. ##### **S1.4_knock-knock_LI+EX+DP.wav** see article for further details. ##### **S1.5_knock-knock_TCE.wav** see article for further details. ##### **S1.6_knock-knock_LI+TCE.wav** see article for further details. ##### **S1.7_knock-knock_LI+EX+TCE.wav** see article for further details. ##### **S1.8_knock-knock_LI+EX+DP+TCE.wav** see article for further details. #### **ACE of machine sounds** ##### **S2.1_machine.wav** see article for further details. ##### **S2.2_machine_EX.wav** see article for further details. ##### **S2.3_machine_TCE.wav** see article for further details. ##### **S2.4_machine_EX+TCE.wav** see article for further details