RoboJam: A Musical Mixture Density Network for Collaborative Touchscreen Interaction

RoboJam is a machine-learning system for generating music that assists users of a touchscreen music app by performing responses to their short improvisations. This system uses a recurrent artificial neural network to generate sequences of touchscreen interactions and absolute timings, rather than high-level musical notes. To accomplish this, RoboJam's network uses a mixture density layer to predict appropriate touch interaction locations in space and time. In this paper, we describe the design and implementation of RoboJam's network and how it has been integrated into a touchscreen music app. A preliminary evaluation analyses the system in terms of training, musical generation and user interaction

Designing relational pedagogies with jam2jamXO

This paper examines the affordances of the philosophy and practice of open source and the application of it in developing music education software. In particular I will examine the parallels inherent in the ‘openness’ of pragmatist philosophy in education (Dewey 1916, 1989) such as group or collaborative learning, discovery learning (Bruner 1966) and learning through creative activity with computers (Papert 1980, 1994). Primarily I am interested in ‘relational pedagogies’ (Ruthmann and Dillon In Press) which is in a real sense about the ethics of the transaction between student and teacher in an ecology where technology plays a more significant role. In these contexts relational pedagogies refers to how the music teacher manages their relationships with students and evaluates the affordances of open source technology in that process. It is concerned directly with how the relationship between student and teacher is affected by the technological tools, as is the capacity for music making and learning. In particular technologies that have agency present the opportunity for a partnership between user and technology that enhances the capacity for expressive music making, productive social interaction and learning. In this instance technologies with agency are defined as ones that enhance the capacity to be expressive and perform tasks with virtuosity and complexity where the technology translates simple commands and gestures into complex outcomes. The technology enacts a partnership with the user that becomes both a cognitive and performative amplifier. Specifically we have used this term to describe interactions with generative technologies that use procedural invention as a creative technique to produce music and visual media

Generative Design in Minecraft (GDMC), Settlement Generation Competition

This paper introduces the settlement generation competition for Minecraft, the first part of the Generative Design in Minecraft challenge. The settlement generation competition is about creating Artificial Intelligence (AI) agents that can produce functional, aesthetically appealing and believable settlements adapted to a given Minecraft map - ideally at a level that can compete with human created designs. The aim of the competition is to advance procedural content generation for games, especially in overcoming the challenges of adaptive and holistic PCG. The paper introduces the technical details of the challenge, but mostly focuses on what challenges this competition provides and why they are scientifically relevant.Comment: 10 pages, 5 figures, Part of the Foundations of Digital Games 2018 proceedings, as part of the workshop on Procedural Content Generatio

Multi-agent evolutionary systems for the generation of complex virtual worlds

Modern films, games and virtual reality applications are dependent on convincing computer graphics. Highly complex models are a requirement for the successful delivery of many scenes and environments. While workflows such as rendering, compositing and animation have been streamlined to accommodate increasing demands, modelling complex models is still a laborious task. This paper introduces the computational benefits of an Interactive Genetic Algorithm (IGA) to computer graphics modelling while compensating the effects of user fatigue, a common issue with Interactive Evolutionary Computation. An intelligent agent is used in conjunction with an IGA that offers the potential to reduce the effects of user fatigue by learning from the choices made by the human designer and directing the search accordingly. This workflow accelerates the layout and distribution of basic elements to form complex models. It captures the designer's intent through interaction, and encourages playful discovery

Post-industrial robotics: the new tendency of digital fabrication for exploring responsive forms and materials through performance

The contribution proposes the experimental results of research on robotics manufacturing issues for the realization of informed architectural organisms on a 1:1 scale. The pavilions Fusta Robotics and Digital Urban Orchard and the technological system In.Flux represent the results of tests in which material, environmental and structural performance inform the computational process and the consequent materialization. The two pavilions, both wooden, constitute the physical implementation of different functional programs realised through a collaboration with industrial partners. Fusta Robotics is the result of a collaboration between industry and universities for the tectonic experimentation derived from the use of local non-engineered material. Digital Urban Orchard is the formal expression of a complex functional program arising from the relationship amongst form (shape), function and context for a new concept of socialization space and food production within the agenda at the self-sufficiency in Barcelona. Finally, through the In.Flux prototype, we investigated the relationship among formal generation, structural analysis and robotic manufacturing for the realization of concrete free-form structures. The analysis of the prototypes opens the debate on the role of IT in the post-digital era when the design process manifest through the control and management of the flow of information affecting the digital computation and fabrication and the material behaviour. The resulting theoretical assumption considers the architectural form as the result of a diagram of forces where the achievement of the performance is the driving parameter for the formal geometric exploration. The continuous variation resulting therefrom is informed by performance parameters that define a new aesthetic which represents together the manifestation of objectively measurable performance parameters and the power of the tool through which the form is generated