Design Strategies for Adaptive Social Composition: Collaborative Sound Environments

In order to develop successful collaborative music systems a variety of subtle interactions need to be identified and integrated. Gesture capture, motion tracking, real-time synthesis, environmental parameters and ubiquitous technologies can each be effectively used for developing innovative approaches to instrument design, sound installations, interactive music and generative systems. Current solutions tend to prioritise one or more of these approaches, refining a particular interface technology, software design or compositional approach developed for a specific composition, performer or installation environment. Within this diverse field a group of novel controllers, described as ‘Tangible Interfaces’ have been developed. These are intended for use by novices and in many cases follow a simple model of interaction controlling synthesis parameters through simple user actions. Other approaches offer sophisticated compositional frameworks, but many of these are idiosyncratic and highly personalised. As such they are difficult to engage with and ineffective for groups of novices. The objective of this research is to develop effective design strategies for implementing collaborative sound environments using key terms and vocabulary drawn from the available literature. This is articulated by combining an empathic design process with controlled sound perception and interaction experiments. The identified design strategies have been applied to the development of a new collaborative digital instrument. A range of technical and compositional approaches was considered to define this process, which can be described as Adaptive Social Composition. Dan Livingston

Interactive, Evolutionary Textured Sound Composition

We describe a system that maps the interaction between two people to control a genetic process for generating music. We start with a population of melodies encoded genetically. This population is allowed to breed every biological cycle creating new members of the population based upon the semantics of the spatial relationship between two people moving in a large, physical space. A pre-specified hidden melody is used to select a melody from the population to play every musical cycle. The overlapping of selected melodies provides an intriguing textured musical space

Interactive Spatialization And Sound Design Using An Evolutionary System

We present an interactive sound spatialization and synthesis system based on Interaural Time Difference (ITD) model and Evolutionary Computation. We define a Sonic Localization Field using sound attenuation and ITD azimuth angle parameters and, in order to control an adaptive algorithm, we used pairs of these parameters as Spatial Sound Genotypes (SSG). They are extracted from waveforms which are considered individuals of a Population Set. A user-interface receives input from a generic gesture interface (such as a NIME device) and interprets them as ITD cues. Trajectories provided by these signals are used as Target Sets of an evolutionary algorithm. A Fitness procedure optimizes locally the distance between the Target Set and the SSG pairs. Through a parametric score the user controls dynamic changes in the sound output.293298Blauert, J., (1997) Spatial Hearing: The Psychophysics of Human Sound Localization, , Cambridge: MIT PressPulkki, V., Virtual sound source positioning using vector base amplitude panning (1997) Journal of the Audio Engineering Society, 45, pp. 456-66Chowning, J.M., The simulation of moving sound sources (1970) Audio Engineering Society 39th Convention, , presented at, New York, NY, USAMichael, C., Jens, H., William, L.M., Cyberspatial audio technology (1999) J. Acous. Soc. Jap., 20 (6), pp. 389-395. , NovBarbieri, T., Bianchi, A., Sbattella, L., Minus-two: Multimedia, sound spatialization and 3D representation for cognitively impaired children (2004) Computer Helping People with Special Needs, , Lecture Notes in Computer Science, Springer Berlin/HeidelbergKelly, J.B., Dennis, P.P., (1991) Coding of Interaural Time Differences of Transients in Auditory Cortex of Rattus Norvegicus: Implications for the Evolution of Mammalian Sound Localization, 55 (1), pp. 39-44. , Hearing ResearchBirchfield, S.T., Gangishetty, R., Acoustic localization by interaural level difference (2005) IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), , Philadelphia, Pennsylvania, MarchBrungart, D.S., William, M.R., "Auditory localization of nearby sources. Head-related transfer functions (1999) The Journal of the Acoustical Society of America, 106 (3), pp. 1465-1479. , SeptemberMurray, J.C., Erwin, H.R., Wermter, S., Robotic sound source localization using interaural time difference and cross-correlation (2004) Proceedings of the KI-2004, , SeptemberFels, S.S., Manzolli, J., Interactive, evolutionary textured sound composition (2001) 6th Eurographics Workshop on Multimedia, pp. 153-164. , SeptManzolli, J., Fornari, J., Maia Jr., A., (2001) Damiani F. the Evolutionary Sound Syn-thesis Method, , Short-paper. ACM multimedia, ISBN:1-58113-394-4. USAFornari, J., Manzolli, J., Maia Jr., A., Damiani, F., The evolutionary sound synthesis method (2001) SCI Conference, , Orlando, USAFornari, J., Maia Jr., A., Manzolli, J., A síntese evolutiva guiada pela espacialização sonora (2006) XVI Congresso da Associação Nacional de Pesquisa e Pós-graduação (ANPPOM), , BrasíliaFornari, J., Maia Jr., A., Manzolli, J., Creating soundscapes using evolutionary spatial control (2007) Proceedings of the EvoMusarts, , Spring-Verlag,ValenciaHolland, J.H., (1992) Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, , Control and Artificial Intelligence, MIT Press, Cambridge MAHolland, J.H., (1996) Hidden Order: How Adaptation Builds Complexity, , Addison-WesleyVon Foerster, H., On self-organizing systems and their environments (1960) Self-Organizing Systems, pp. 31-50. , M. C. Yovits und S. Cameron (Hg.), Pergamon Press, LondonCaetano, M., Jônatas Manzolli, J., Fernando Von Zuben, F., Self-organizing bio-inspired sound transformation (2007) Proceedings of the EvoMusarts, , Spring-Verlag,ValenciaMurray, R., Schafer, M., (1977) The Soundscape, , ISBN 0- 89281-455-461Truax, B., (1978) Handbook for Acoustic Ecology, , ISBN 0-88985-011-1

Soundscape Design Through Evolutionary Engines

Two implementations of an Evolutionary Sound Synthesis method using the Interaural Time Difference (ITD) and psychoacoustic descriptors are presented here as a way to develop criteria for fitness evaluation. We also explore a relationship between adaptive sound evolution and three soundscape characteristics: key-sounds, key-signals and sound-marks. Sonic Localization Field is defined using a sound attenuation factor and ITD azimuth angle, respectively (Ii, Li. These pairs are used to build Spatial Sound Genotypes (SSG) and they are extracted from a waveform population set. An explanation on how our model was initially written in MATLAB is followed by a recent Pure Data (Pd) implementation. It also elucidates the development and use of parametric scores, a triplet of psychoacoustic descriptors and the correspondent graphical user interface.1435164Back, T., (1996) Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms, , Oxford Univ. PressTodd, S., Latham, W., (1992) Evolutionary Art and Computers, , Academic PressGoldberg, D.E., The Race, the Hurdle, and the Sweet Spot: Lessons from Genetic Algorithms for the Automation of Design Innovation and Creativity (1999) Evolutionary Design by Computers, pp. 105-118. , Bentley, P, ed, Morgan Kaufmann Publisher, San FranciscoBiles, J.A., Gen Jam: A Genetic Algorithm for Generating Jazz Solos (1994) Proceedings of the 1994 International Computer Music Conference, pp. 131-137. , http://www.it.rit.edu/~jab/GenJam.html, ICMC94, JanMiranda, E.R., On the evolution of music in a society of self-taught digital creatures (2003) Digital Creativity, 14 (1), pp. 29-42Mitchell, T.J., Pipe, A.G., Convergence Synthesis of Dynamic Frequency 'Modulation. Tones Using an Evolution Strategy (2005) Applications on Evolutionary Computing, Springer Verlag, Berlin, EvoMusart Conference, Computer Lectures, 3449, pp. 533-538J. McDermott, N. J.L. Griffith and Michael. O'Neill. Evolutionary Computation Applied to Sound Synthesis. In The Art of Artificial Evolution, J. Romero and P. Machado (eds.), Springer Verlag, Berlin, Natural Computing Series, ISSN 1619-7127, pages 81.-101, 2008Moroni, A., Manzolli, J., von Zuben, F., Gudwin, R., Vox Populi: An Interactive Evolutionary System for Algorithmic Music Composition (2000) Leonardo Music Journal, 10, pp. 49-54. , San Francisco,USA, MIT PressFels, S., Manzolli, J., Interactive, Evolutionary Textured Sound Composition (2001) Proceedings of the 6th Eurographics Workshop on Multimedia, pp. 153-164J. Manzolli, J. E. Fornari, A. Maio Jr. and F. Damiani. The Evolutionary Sound Synthesis Method. In Proceedings of the ACM Multimedia Conference, page 585-587, ISBN:1-58113-394-4. USA, 2001Fomari, J., Maia Jr., A., Manzolli, J., Creating Soundscapes using Evolutionary Spatial Control (2007) Applications of Evolutionary Computing, Lectures in Computer Science, 4448, pp. 517-526. , Springer-Verlag Berlin/HeidelbergFornari, J., Maia Jr., A., Manzolli, J., Interactive Soundscape Design with Evolutionary Sound (2007) Proceedings of the 2007 International Computer Music Conference (ICMC 2007), pp. 427-430. , Copenhagen, DenmarkCaetano, M., Manzolli, J., von Zuben, F.J., Application of an Artificial Immune System in a Compositional (2005) Lecture Notes in Computer Science. Timbre Design Technique, 3627, pp. 389-403. , C. Jacob et al, eds, ICARISHolland, J.H., (1975) Adaptation in Natural and Artificial Systems, , University of Michigan PressBlauert, J., (1997) Spatial hearing: The psychophysics of human sound localization, , Cambridge: MIT PressPulkki, V., Virtual sound source positioning using vector base amplitude panning (1997) Journal of the Audio Engineering Society, 45, pp. 456-466Chowning, J.M., The simulation of moving sound sources (1971) Proceedings of the Audio Engineering Society 39th Convention, 1 (2-6). , New York, NY, USA, 19::, JanCohen, M., Herder, J., Martens, W.L., Cyberspatial Audio Technology (1999) J. Acous. Soc. Jap, 20 (6), pp. 389-395. , NovBarbieri, T., Bianchi, A., Sbattella, L., MinusTwo: Multimedia, Sound Spatialization and 3D Representation for Cognitively Impaired Children (2004) Lecture Notes in Computer Science, 3118, pp. 1054-1061. , Computer helping people with Special Needs, Springer-Verlag Berlin /HeidelbergKelly, J.B., Phillips, D.P., Coding of Interaural time differences of transients in auditory cortex of Rattus norvegicus: Implications for the evolution of mammalian sound localization (1991) Hearing Research, 55 (1), pp. 39-44S.T. Birchfield., S.T. and R. Gangishetty, Acoustic Localization by Interaural Level Difference. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Philadelphia, Pennsylvania, 4:1109-1112, D.O.I. 10.1109/1CASSP.2005.1416207, 2005Brungart, D.S., Rabinowitz, W.M., Auditory localization of nearby sources. Head-related transfer functions (1999) The Journal of the Acoustical Society of America, 106 (3), pp. 1465-1479. , SeptJ. Murray, S. Wermter, H. Erwin. Auditory robotic tracking of sound sources using hybrid cross-correlation and recurrent networks. In Proceedings of the IEEE/RSJ Intelligent Robots and Systems (IROS 2005), pages 3554-3559 D. O. I. 10.1109/IROS.2005.1545093, 2005Murray Schafer, R., (1977) The Soundscape, , ISBN 0-89281-455-1Truax, B., (1978) Handbook for Acoustic Ecology, , ISBN 0-88985-011-9M. Leman, V. Vermeulen, L. de Voogdt, D. Moelants, D., & Lesaffre, M. Correlation of Gestural Musical Audio Cues. In Proceedings of Gesture-Based Communication in Human-Computer Interaction: 5th International Gesture 'Workshop, GW 2003, pages 40-54, 2004Dahlsteds, P., Evolution in Creative Sound Design (2007) Evolutionary Computer Music, pp. 79-99. , ed, E.R. Miranda & J.L. Biles, Springer-Verlag LondonL. H. H. Law, S. Phon-Amnuaisuk. Towards Music Fitness Evaluation with the Hierarchical SOM. In Applications of Evolutionary Computing, LNCS, Springer Berlin, 4974/2008, pages 443-452, 2008J. Manzolli. continuaMENTE: integrating Percussion, Audiovisual and Improvisation. To be published in the Proceedings of the International Computer Music Conference (ICMC 2008), Belfast, Irland, 200