Turning Gigabytes into Gigs: “Songification” and Live Music Data

Complex data is challenging to understand when it is represented as written communication even when it is structured in a table. How- ever, choosing to represent data in creative ways can aid our under- standing of complex ideas and patterns. In this regard, the creative industries have a great deal to offer data-intensive scholarly disci- plines. Music, for example, is not often used to interpret data, yet the rhythmic nature of music lends itself to the representation and anal- ysis of temporal data.Taking the music industry as a case study, this paper explores how data about historical live music gigs can be analysed, extend- ed and re-presented to create new insights. Using a unique process called ‘songification’ we demonstrate how enhanced auditory data design can provide a medium for aural intuition. The case study also illustrates the benefits of an expanded and inclusive view of research; in which computation and communication, method and media, in combination enable us to explore the larger question of how we can employ technologies to produce, represent, analyse, deliver and exchange knowledge

Sonification design patterns

Proceedings of the 9th International Conference on Auditory Display (ICAD), Boston, MA, July 7-9, 2003.Most product designers have little or no experience with sonifications. Designers from a range of different domains use a common method called Design Patterns to describe ``solutions to problems in context'' in a way that can be readily understood and reused. Design Patterns may provide a way to communicate sonification research results with product designers and other design communities. I have written a handful of prototype Sonification Design Patterns from papers in the ICAD 2002 proceedings. The papers I selected had clear statements of hypotheses, results to support them, and repeated examples elsewhere in the proceedings. These Patterns are now on the SonificationDesignPatterns site on the WikiWeb and can be edited and added to using any internet browser. The lively development of SonificationDesignPatterns by the ICAD community may help build sonification-specific vocabulary, identify sonification hypotheses, and allow product designers to pick up and apply our research

Sonification using digital waveguides and 2- and 3-dimensional digital waveguide mesh

Presented at the 11th International Conference on Auditory Display (ICAD2005)We describe a method of auditory display of complex data, object identification, and classification using digital waveguides and waveguide mesh. Our overall goal is to distinguish highly dimensional data sets from one another in such a way that reveals meaningful differences in a particular context. In this paper we provide a summary of the application of waveguide and waveguide mesh architectures to sonification, and demonstrate the digital waveguide, 2- and 3-dimensional mesh in a variety of sonification tasks

Perceptual distance in timbre space

Presented at the 11th International Conference on Auditory Display (ICAD2005)This paper describes a perceptual space for timbre, defines an ob- jective metric that takes into account perceptual orthogonality, and measures the quality of timbre interpolation applicable to percep- tually valid timbral sonification. We discuss two timbre represen- tations and measure perceptual judgment. We determined that a timbre space based on Mel-frequency cepstral coefficients (MFCC) is a good model for perceptual timbre space

SonData : um toolkit para sonorização de dados interactiva

Tese de mestrado. Multimédia. Faculdade de Engenharia. Universidade do Porto. 201

The Bird's Ear View: Audification for the Spectral Analysis of Heliospheric Time Series Data.

The sciences are inundated with a tremendous volume of data, and the analysis of rapidly expanding data archives presents a persistent challenge. Previous research in the field of data sonification suggests that auditory display may serve a valuable function in the analysis of complex data sets. This dissertation uses the heliospheric sciences as a case study to empirically evaluate the use of audification (a specific form of sonification) for the spectral analysis of large time series. Three primary research questions guide this investigation, the first of which addresses the comparative capabilities of auditory and visual analysis methods in applied analysis tasks. A number of controlled within-subject studies revealed a strong correlation between auditory and visual observations, and demonstrated that auditory analysis provided a heightened sensitivity and accuracy in the detection of spectral features. The second research question addresses the capability of audification methods to reveal features that may be overlooked through visual analysis of spectrograms. A number of open-ended analysis tasks quantitatively demonstrated that participants using audification regularly discovered a greater percentage of embedded phenomena such as low-frequency wave storms. In addition, four case studies document collaborative research initiatives in which audification contributed to the acquisition of new domain-specific knowledge. The final question explores the potential benefits of audification when introduced into the workflow of a research scientist. A case study is presented in which a heliophysicist incorporated audification into their working practice, and the “Think-Aloud” protocol is applied to gain a sense for how audification augmented the researcher’s analytical abilities. Auditory observations are demonstrated to make significant contributions to ongoing research, including the detection of previously unidentified equipment-induced artifacts. This dissertation provides three primary contributions to the field: 1) an increased understanding of the comparative capabilities of auditory and visual analysis methods, 2) a methodological framework for conducting audification that may be transferred across scientific domains, and 3) a set of well-documented cases in which audification was applied to extract new knowledge from existing data archives. Collectively, this work presents a “bird’s ear view” afforded by audification methods—a macro understanding of time series data that preserves micro-level detail.PhDDesign ScienceUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111561/1/rlalexan_1.pd

Crystallization sonification of high-dimensional datasets

Presented at the 8th International Conference on Auditory Display (ICAD), Kyoto, Japan, July 2-5, 2002.This paper introduces Crystallization Sonification, a sonification model for exploratory analysis of high-dimensional datasets. The model is designed to provide information about the intrinsic data dimensionality (which is a local feature) and the global data dimensionality, as well as the transitions between a local and global view on a dataset. Furthermore the sound allows to display the clustering in high-dimensional datasets. The model defines a crystal growth process in the high-dimensional data-space which starts at a user selected ``condensation nucleus'' and incrementally includes neighboring data according to some growth criterion. The sound summarizes the temporal evolution of this crystal growth process. For introducing the model, a simple growth law is used. Other growth laws which are used in the context of hierarchical clustering are also suited and their application in crystallization sonification offers new ways to inspect the results of data clustering as an alternative to dendrogram plots. In this paper, the sonification model is described and example sonifications are presented for some synthetic high-dimensional datasets

Crystallization sonification of high-dimensional datasets

Hermann T, Ritter H. Crystallization sonification of high-dimensional datasets. ACM Trans. Applied Perception. 2005;2(4):550-558.This paper introduces Crystallization Sonification, a sonification model for exploratory analysis of high-dimensional datasets. The model is designed to provide information about the intrinsic data dimensionality (which is a local feature) and the global data dimensionality, as well as the transitions between a local and global view on a dataset. Furthermore the sound allows to display the clustering in high-dimensional datasets. The model defines a crystal growth process in the high-dimensional data-space which starts at a user selected ``condensation nucleus'' and incrementally includes neighboring data according to some growth criterion. The sound summarizes the temporal evolution of this crystal growth process. For introducing the model, a simple growth law is used. Other growth laws which are used in the context of hierarchical clustering are also suited and their application in crystallization sonification offers new ways to inspect the results of data clustering as an alternative to dendrogram plots. In this paper, the sonification model is described and example sonifications are presented for some synthetic high-dimensional datasets

Crystallization Sonification of High-dimensional Datasets

Hermann T, Ritter H. Crystallization Sonification of High-dimensional Datasets. In: Nakatsu R, Kawahara H, eds. Proceedings of the International Conference on Auditory Display. ICAD; 2002: 76-81.This paper introduces Crystallization Sonification, a sonification model for exploratory analysis of high-dimensional datasets. The model is designed to provide information about the intrinsic data dimensionality (which is a local feature) and the global data dimensionality, as well as the transitions between a local and global view on a dataset. Furthermore the sound allows to display the clustering in high-dimensional datasets. The model defines a crystal growth process in the high-dimensional data-space which starts at a user selected “condensation nucleus” and incrementally includes neighboring data according to some growth criterion. The sound summarizes the temporal evolution of this crystal growth process. For introducing the model, a simple growth law is used. Other growth laws which are used in the context of hierarchical clustering are also suited and their application in crystallization sonification offers new ways to inspect the results of data clustering as an alternative to dendrogram plots. In this paper, the sonification model is described and example sonifications are presented for some synthetic high-dimensional datasets

Supplementary Material for "Crystallization Sonification of High-dimensional Datasets"

Hermann T, Ritter H. Supplementary Material for "Crystallization Sonification of High-dimensional Datasets". Bielefeld University; 2002.#### Sound Demonstrations for the Data Crystallization Sonification (DCS) Model * Table 1: Sound Examples for Crystallization Sonification for 5d Gaussian dataset File/Track: DCS started (E1a) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704156"> at center, (E1b) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704157"> in tail, (E1c) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704158"> from far outside Description: DCS for dataset that is N{0, I_5} excited at different locations Duration: 1.4 s * Table 2/Figure 4: Mixture of 2 Gaussians * DCS started at point A: [E2a](https://pub.uni-bielefeld.de/download/2704155/2704159) * DCS started at point B: [E2b](https://pub.uni-bielefeld.de/download/2704155/2704160) * Table 3: Sound examples for DCS on variation of the energy decay time File/Track: tau_(1/2) =&nbsp; (E3a) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704161"> 0.001, (E3b) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704162"> 0.005, (E3c) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704164"> 0.01, (E3d) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704163"> 0.05, (E3e) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704165"> 0.1, (E3f) <a href= "https://pub.uni-bielefeld.de/download/2704155/2704166"> 0.2 Description: DCS for a mixture of two Gaussians varying the energy decay time tau_(1/2) Duration: 1.4 s * Table 4: Sound examples for DCS on variation of the sonification time File/Track: T = <a href= "https://pub.uni-bielefeld.de/download/2704155/2704167"> 0.2s (E4a) , <a href= "https://pub.uni-bielefeld.de/download/2704155/2704168"> 0.5s (E4b), <a href= "https://pub.uni-bielefeld.de/download/2704155/2704169"> 1s (E4c) , <a href= "https://pub.uni-bielefeld.de/download/2704155/2704171"> 2s (E4d)&nbsp; , <a href= "https://pub.uni-bielefeld.de/download/2704155/2704170"> 4s (E4e) ,&nbsp; <a href= "https://pub.uni-bielefeld.de/download/2704155/2704172"> 8s (E4f) Description: DCS for a mixture of two Gaussians on varying the duration T Duration: 0.2s -- 8s * Table 5: Sound examples for DCS for different excitation locations File/Track: starting point:&nbsp; <a href= "https://pub.uni-bielefeld.de/download/2704155/2704174"> C0 (E5a), <a href= "https://pub.uni-bielefeld.de/download/2704155/2704173"> C1(E5b), <a href= "https://pub.uni-bielefeld.de/download/2704155/2704175"> C2(E5c) Description: DCS for a mixture of three Gaussians in 10d space with different rang(Sigma) = {2,4,8} Duration: 1.9 s * Table 6: Sound examples for DCS for the mixture of a 2d uniform distribution and a 5d Gaussian File/Track: condensation nucleus in (x0,x1)-plane at: E6a <a href= "https://pub.uni-bielefeld.de/download/2704155/2704176"> (-6,0)=C1, E6b<a href= "https://pub.uni-bielefeld.de/download/2704155/2704177">(-3,0)=C2, E6c<a href= "https://pub.uni-bielefeld.de/download/2704155/2704178">( 0,0)=C0 Description: DCS for a mixture of a uniform 2d and a 5d Gaussian Duration: 2.16 s </table