Applications of AI in the BSc CS degree

This talk presents a high-level overview of four case studies wherein we used AI technology in the UoL/ Goldsmiths BSc Computer Science degree. Study 1 used NLP and LLM techniques to analyse survey data. Study 2 used LLM embeddings to detect collusion in exams. Study 3 used LLMs to generate exam starter templates. Study 4 created but did not deploy a prototype of a student tutor model using local LLMs and RAG techniques. Ethical aspects of each system are discussed with the audience based on a framework of value alignment

Automatic sound synthesizer programming: techniques and applications

The aim of this thesis is to investigate techniques for, and applications of automatic sound synthesizer programming. An automatic sound synthesizer programmer is a system which removes the requirement to explicitly specify parameter settings for a sound synthesis algorithm from the user. Two forms of these systems are discussed in this thesis: tone matching programmers and synthesis space explorers. A tone matching programmer takes at its input a sound synthesis algorithm and a desired target sound. At its output it produces a configuration for the sound synthesis algorithm which causes it to emit a similar sound to the target. The techniques for achieving this that are investigated are genetic algorithms, neural networks, hill climbers and data driven approaches. A synthesis space explorer provides a user with a representation of the space of possible sounds that a synthesizer can produce and allows them to interactively explore this space. The applications of automatic sound synthesizer programming that are investigated include studio tools, an autonomous musical agent and a self-reprogramming drum machine. The research employs several methodologies: the development of novel software frameworks and tools, the examination of existing software at the source code and performance levels and user trials of the tools and software. The main contributions made are: a method for visualisation of sound synthesis space and low dimensional control of sound synthesizers; a general purpose framework for the deployment and testing of sound synthesis and optimisation algorithms in the SuperCollider language sclang; a comparison of a variety of optimisation techniques for sound synthesizer programming; an analysis of sound synthesizer error surfaces; a general purpose sound synthesizer programmer compatible with industry standard tools; an automatic improviser which passes a loose equivalent of the Turing test for Jazz musicians, i.e. being half of a man-machine duet which was rated as one of the best sessions of 2009 on the BBC's 'Jazz on 3' programme

Towards a Machine Somaesthete: Latent Modeling of EMG Signals in Viola Playing

Somaesthetic experiential qualities can provide a window into process of meaning-making, both human and machinic. We draw such qualities from viola performance into the design-in-progress of a novel interactive performance system. In doing so, we introduce the concept of a Machine Somaesthete that senses and makes sense of these qualities from a second-person perspective. Our system comprises electromyographic (EMG) muscle sensing and a Variational Autoencoder. With a novel dataset, we aim to encode latent representations of performance movement that are meaningful from a somaesthetic perspective. We present our model and our design process, then analyse latent trajectories to interrogate how our system can be considered a Machine Somaesthete, and the nature of its sensitivity to bodily experiences of viola playing. At the intersection of artificial intelligence, music performance and intra-action design, we take a sympoietic (together-making) view of knowledge creation. We and our practices are transformed as we design - and design with - machine learning systems

Studio report: sound synthesis with DDSP and network bending techniques

This paper reports on our experiences synthesizing sounds and building network bending functionality onto the Differentiable Digital Signal Processing (DDSP) system. DDSP is an extension to the TensorFlow API with which we can embed trainable signal processing nodes in neural networks. Comparing DDSP sound synthesis networks to preset finding networks and sample level synthesis networks, we argue that it offers a third mode of working, providing continuous control in real-time of high fidelity synthesizers using low numbers of control parameters. We describe two phases of our experimentation. Firstly we worked with a composer to explore different training datasets and parameters. Secondly, we extended DDSP models with network bending functionality, which allows us to feed additional control data into the network's hidden layers and achieve new timbral effects. We describe several possible network bending techniques and how they affect the sound

Social machines for education driven by feedback agents

The aim of this paper is to explain some of the ways in which multi agent system (MAS) theory can be used to describe, design and enhance social machines (also referred to as Socio-Cognitive Systems). We believe there is a really opportunity for the MAS community to engage with emerging theory and practice of designing such systems. Social machines - also referred to as Socio-Cognitive Systems from the MAS community - are terms used to refer to the recent breed of technological systems which allow human and computational agents to socially interact, typically on a large scale and sometimes towards achieving shared goals. Examples include social networking platforms and crowd sourced encyclopaedias. The discussion of social machines and MAS is taken from three perspectives. Firstly, the theoretical notion of an abstract social machine as a socio-cognitive system containing humans and agents is introduced. Secondly, a speci#12;c instance of a social machine which has been designed to enable social music learning supported by agents is described. Thirdly, an agent architecture which is designed for operation within educational social machines is discussed, with particular focus on what we believe is the core currency of these machines: feedback.Much of of this work was undertaken as part of the FP7 project in the Technology Enhanced Learning Program called Practice and Performance Analysis Inspiring Social Education (PRAISE) involving the 1st and 2nd authors. We acknowledge Harry Brenton, Marco Gillies Andreu Grimalt-Reynes, Jonathan James, Edgar Jones, Julian Padget and Harko Harko Verhagen who have helped in discussions. The second author received support from the European Network for Social Intelligence, SINTELNET (FET Open Coordinated Action FP7-ICT-2009-C Project No. 286370) for short term visits to the IIIA to work with the 3rd author.Peer Reviewe

The pop song generator: designing an online course to teach collaborative, creative AI

This article describes and evaluates a new online AI-creativity course. The course is based around three near-state-of-the-art AI models combined into a pop song generating system. A fine-tuned GPT-2 model writes lyrics, Music-VAE composes musical scores and instrumentation and Diffsinger synthesises a singing voice. We explain the decisions made in designing the course which is based on Piagetian, constructivist 'learning-by-doing'. We present details of the five-week course design with learning objectives, technical concepts, and creative and technical activities. We explain how we overcame technical challenges to build a complete pop song generator system, consisting of Python scripts, pre-trained models, and Javascript code that runs in a dockerised Linux container via a web-based IDE. A quantitative analysis of student activity provides evidence on engagement and a benchmark for future improvements. A qualitative analysis of a workshop with experts validated the overall course design, it suggested the need for a stronger creative brief and ethical and legal content

Beyond Motivation and Engagement: Students’ Voices on the Use of Game-Based Learning in a Bachelor Computer Science Online Degree

Video games have many potential uses beyond pure entertainment, including their use in educational contexts. Yet, it remains really challenging to put together guidelines to design effective game-like interventions in educational contexts. This study examines existing work relating to gamification, game-based learning, and serious games, and finds there is still limited qualitative work concerning the student perspective and limited work developing pedagogical guidelines for developers wishing to develop effective game-based learning experiences. The study focuses on the perception of students in regard to game-based learning activities in the context of a BSc Computer Science online degree. Students enrolled in the online degree were invited to fill in an online survey after their experience with a selection of game-based learning activities in the online degree. Reflexive Thematic Analysis was used to evaluate the open-ended responses from 55 participants. First, quantitative and qualitative results revealed insightful information along with four overarching themes (“Complementary to lectures on topics that are usually hard or too abstract to teach”, “Allow students to take on identities and learn from different angles and perspectives”, “Balanced challenge and context relevance to minimise students wasting their time”, and “Reward players for their effort with meaningful rewards and provide a safe space for failure”), suggesting that game-based learning interventions offer more than just motivation and engagement. Second, technical and pedagogical principles emerged from the data analysis, proposing guidelines for future designers of game-based learning activities in similar educational contexts. Finally, the study provides a selection of twelve open-source and browser-based game-based learning activities, the ones students encountered in the BSc Computer Science online degree

Progress report on the EAVI BCI toolkit for music: musical applications of algorithms for use with consumer brain computer interfaces

No description supplie

Sun Begins to Fade

A score written by Max de Wardener for the Call and Response projec