Neuroeducation: Learning, Arts, and the Brain

Excerpts presentations and discussions from a May 2009 conference on the intersection of cognitive neuroscience, the arts, and learning -- the effects of early arts education on other aspects of cognition and implications for policy and practice

Learning music similarity from relative user ratings

Computational modelling of music similarity is an increasingly important part of personalisation and optimisation in music information retrieval and research in music perception and cognition. The use of relative similarity ratings is a new and promising approach to modelling similarity that avoids well known problems with absolute ratings. In this article, we use relative ratings from the MagnaTagATune dataset with new and existing variants of state-of-the-art algorithms and provide the first comprehensive and rigorous evaluation of this approach. We compare metric learning based on support vector machines (SVMs) and metric-learning-to-rank (MLR), including a diagonal and a novel weighted variant, and relative distance learning with neural networks (RDNN). We further evaluate the effectiveness of different high and low level audio features and genre data, as well as dimensionality reduction methods, weighting of similarity ratings, and different sampling methods. Our results show that music similarity measures learnt on relative ratings can be significantly better than a standard Euclidian metric, depending on the choice of learning algorithm, feature sets and application scenario. MLR and SVM outperform DMLR and RDNN, while MLR with weighted ratings leads to no further performance gain. Timbral and music-structural features are most effective, and all features jointly are significantly better than any other combination of feature sets. Sharing audio clips (but not the similarity ratings) between test and training sets improves performance, in particular for the SVM-based methods, which is useful for some applications scenarios. A testing framework has been implemented in Matlab and made publicly available http://mi.soi.city.ac.uk/datasets/ir2012framework so that these results are reproducible

From rituals to magic: Interactive art and HCI of the past, present, and future

The connection between art and technology is much tighter than is commonly recognized. The emergence of aesthetic computing in the early 2000s has brought renewed focus on this relationship. In this article, we articulate how art and Human–Computer Interaction (HCI) are compatible with each other and actually essential to advance each other in this era, by briefly addressing interconnected components in both areas—interaction, creativity, embodiment, affect, and presence. After briefly introducing the history of interactive art, we discuss how art and HCI can contribute to one another by illustrating contemporary examples of art in immersive environments, robotic art, and machine intelligence in art. Then, we identify challenges and opportunities for collaborative efforts between art and HCI. Finally, we reiterate important implications and pose future directions. This article is intended as a catalyst to facilitate discussions on the mutual benefits of working together in the art and HCI communities. It also aims to provide artists and researchers in this domain with suggestions about where to go next

Composing with Matter: Interdisciplinary Explorations Between the Natural and the Artificial

This practice-based research, which includes a written thesis and a portfolio of creative practice, represents the interdisciplinary exploration of co-composition between natural and artificial matter as otherworldly phenomena. Accelerated by the application of recent technologies to control natural materials, matter has become merged between nature and artefacts, offering new potentials, where the boundaries are becoming increasingly blurred. This thesis presents a series of complementing sound art works, including transition [systemic], transition [characteristic], and moment, which were devised through co-composing towards a creative outcome that combines sonic and visual elements by integrating natural and artificial matter as co-authors and co-makers within the creative process to generate multiple perspectives. Raising questions of the boundary between nature and artificiality, it aims to consider a new methodology for sound art in the human-dominated, Anthropocene epoch. This research employs natural elements and processes to engage with sonic and visual anthropomorphism. It is focused on generative processes in the organisation of matter, here analysed and harnessed for sound expression, using acoustic phenomena including the inaudible range that can be perceived through matter. Through a laboratory-based study made in collaboration with scientists, three ‘life-like’ features of the generative processes of materials are discussed: 1) fusion and division, 2) network formation, and 3) pulse and rhythm. The practice explores these features to develop a new methodology of authorship and making, examining the following two questions. How can life-like behaviours of matter be portrayed through sonic and visual modes of expression? And in what ways might the expression of life-like behaviours be grasped by human perception? In conclusion, by integrating the agency of matter into the compositional processes, life-like features – as described by current theories in art, design, science, and philosophy – are made apparent

Pathway to Future Symbiotic Creativity

This report presents a comprehensive view of our vision on the development path of the human-machine symbiotic art creation. We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist (Turing Artists) to a Machine artist in its own right. We begin with an overview of the limitations of the Turing Artists then focus on the top two-level systems, Machine Artists, emphasizing machine-human communication in art creation. In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations. The rapid development of immersive environment and further evolution into the new concept of metaverse enable symbiotic art creation through unprecedented flexibility of bi-directional communication between artists and art manifestation environments. By examining the latest sensor and XR technologies, we illustrate the novel way for art data collection to constitute the base of a new form of human-machine bidirectional communication and understanding in art creation. Based on such communication and understanding mechanisms, we propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle rather than the traditional "end-to-end" dogma. By proposing a new form of inverse reinforcement learning model, we outline the platform design of machine artists, demonstrate its functions and showcase some examples of technologies we have developed. We also provide a systematic exposition of the ecosystem for AI-based symbiotic art form and community with an economic model built on NFT technology. Ethical issues for the development of machine artists are also discussed