Pathfinder:a performance-game for the augmented drum-kit

Pathfinder is a performance-game for solo drummer, exploring the synergies between multiple contemporary creative practices. The work navigates between music composition, improvisation, projection/light art and game art. At its heart lies a bespoke electro-acoustic instrument, the augmented drum-kit, used not only to provide the sonic content of the work in real-time, but also as a highly expressive game controller that interacts with an instrument-specific game. The musical instrument offers a much wider range of expressive possibilities, control and tactile feedback in comparison to a traditional general-purpose game controller, and as a result it affords a more diverse and nuanced game play performance. Live electronics, lights, projections and the drum-kit all make up the performance-game’s universe, within which the performer has to explore, adapt, navigate and complete a journey

Pathfinder

Pathfinder is an audiovisual performance-game for solo drummer, exploring the synergies between multiple contemporary creative practices. The work navigates between music composition, improvisation, projection/light art and game art. At its heart lies a bespoke electroacoustic instrument, the augmented drum-kit, used not only to provide the sonic content of the work in real-time, but also as a highly expressive game controller that interacts with an instrument-specific game. The musical instrument offers a much wider range of expressive possibilities, control and tactile feedback in comparison to a traditional general-purpose game controller, and as a result it affords a more diverse and nuanced game play performance. Live electronics, lights, projections and the drum-kit all make up the performance-game’s universe, within which the performer has to explore, adapt, navigate and complete a journey. This research aimed to address a number of research questions such as: How can a musical game harness the expressivity of an instrumentalist’s performance practice, as opposed to commercial music games using conventional game controllers? How can improvisation, experimentation and failure be quantified and rewarded in an open world musical game scenario? The theoretical underpinnings of the work and findings are documented the upcoming article ‘Designing Musical Games for Electroacoustic Improvisation’, to be published on the upcoming Organised Sound Journal Vol.26, Number 1 Issue, by Cambridge University Press.Pathfinder was toured conferences, festivals and exhibitions internationally. These include the International Conference on Live Interfaces (2016), Brighton, New Interfaces for Musical Expression (2016), Brisbane, where the piece also won the conference’s best performance award, DiGRA/FDG (2016), Dundee, PRESS PLAY (2016), Dundee, Blank Arcade (2016), Dundee, Games Are for Everyone (2016), Edinburgh, Sonorities Festival of Contemporary Music (2016), Belfast, Edinburgh Science Festival (2017), Edinburgh, and the International Conference of Computer Music, Shanghai (2016).<br/

BitBox!:A case study interface for teaching real-time adaptive music composition for video games

Real-time adaptive music is now well-established as a popular medium, largely through its use in video game soundtracks. Commercial packages, such as fmod, make freely available the underlying technical methods for use in educational contexts, making adaptive music technologies accessible to students. Writing adaptive music, however, presents a significant learning challenge, not least because it requires a different mode of thought, and tutor and learner may have few mutual points of connection in discovering and understanding the musical drivers, relationships and structures in these works. This article discusses the creation of ‘BitBox!’, a gestural music interface designed to deconstruct and explain the component elements of adaptive composition through interactive play. The interface was displayed at the Dare Protoplay games exposition in Dundee in August 2014. The initial proof-of- concept study proved successful, suggesting possible refinements in design and a broader range of applications

Virtual Reality Rhythm Game

Virtual reality headsets such as the HTC Vive and Oculus Rift bring robust virtual reality technology in the hands of consumers. However, virtual reality technology is still a very new and unexplored domain with a dearth of compelling software that takes advantage of what virtual reality has to offer. Current rhythm games on the virtual reality platform lack a sense of immersion for the player. These games also require players to remain stationary during gameplay. Our solution is a game where players have to hit musical notes that appear in a trail around them. The trail will move in different directions and players have to move and turn around accordingly in order to hit every note and pass a song

From ‘hands up’ to ‘hands on’: harnessing the kinaesthetic potential of educational gaming

Traditional approaches to distance learning and the student learning journey have focused on closing the gap between the experience of off-campus students and their on-campus peers. While many initiatives have sought to embed a sense of community, create virtual learning environments and even build collaborative spaces for team-based assessment and presentations, they are limited by technological innovation in terms of the types of learning styles they support and develop. Mainstream gaming development – such as with the Xbox Kinect and Nintendo Wii – have a strong element of kinaesthetic learning from early attempts to simulate impact, recoil, velocity and other environmental factors to the more sophisticated movement-based games which create a sense of almost total immersion and allow untethered (in a technical sense) interaction with the games’ objects, characters and other players. Likewise, gamification of learning has become a critical focus for the engagement of learners and its commercialisation, especially through products such as the Wii Fit. As this technology matures, there are strong opportunities for universities to utilise gaming consoles to embed levels of kinaesthetic learning into the student experience – a learning style which has been largely neglected in the distance education sector. This paper will explore the potential impact of these technologies, to broadly imagine the possibilities for future innovation in higher education

Evaluation of live human-computer music-making: Quantitative and qualitative approaches

NOTICE: this is the author’s version of a work that was accepted for publication in International Journal of Human-Computer Studies. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in International Journal of Human-Computer Studies, [VOL 67,ISS 11(2009)] DOI: 10.1016/j.ijhcs.2009.05.00

Spartan Daily, April 10, 2002

Volume 118, Issue 46https://scholarworks.sjsu.edu/spartandaily/10619/thumbnail.jp

Piano Genie

We present Piano Genie, an intelligent controller which allows non-musicians to improvise on the piano. With Piano Genie, a user performs on a simple interface with eight buttons, and their performance is decoded into the space of plausible piano music in real time. To learn a suitable mapping procedure for this problem, we train recurrent neural network autoencoders with discrete bottlenecks: an encoder learns an appropriate sequence of buttons corresponding to a piano piece, and a decoder learns to map this sequence back to the original piece. During performance, we substitute a user's input for the encoder output, and play the decoder's prediction each time the user presses a button. To improve the intuitiveness of Piano Genie's performance behavior, we impose musically meaningful constraints over the encoder's outputs.Comment: Published as a conference paper at ACM IUI 201