Towards a computational- and algorithmic-level account of concept blending using analogies and amalgams

Concept blending–a cognitive process which allows for the combination of certain elements (and their relations) from originally distinct conceptual spaces into a new unified space combining these previously separate elements, and enables reasoning and inference over the combination–is taken as a key element of creative thought and combinatorial creativity. In this article, we summarise our work towards the development of a computational-level and algorithmic-level account of concept blending, combining approaches from computational analogy-making and case-based reasoning (CBR). We present the theoretical background, as well as an algorithmic proposal integrating higher-order anti-unification matching and generalisation from analogy with amalgams from CBR. The feasibility of the approach is then exemplified in two case studies

Evaluating computational creativity: a standardised procedure for evaluating creative systems and its application

This thesis proposes SPECS: a Standardised Procedure for Evaluating Creative Systems. No methodology has been accepted as standard for evaluating the creativity of a system in the field of computational creativity and the multi-faceted and subjective nature of creativity generates substantial definitional issues. Evaluative practice has developed a general lack of rigour and systematicity, hindering research progress. SPECS is a standardised and systematic methodology for evaluating computational creativity. It is flexible enough to be applied to a variety of different types of creative system and adaptable to specific demands in different types of creativity. In the three-stage process of evaluation, researchers are required to be specific about what creativity entails in the domain they work in and what standards they test a system’s creativity by. To assist researchers, definitional issues are investigated and a set of components representing aspects of creativity is presented, which was empirically derived using computational linguistics analysis. These components are recommended for use within SPECS, being offered as a general definition of creativity that can be customised to account for any specific priorities for creativity in a given domain. SPECS is applied in a case study for detailed comparisons of the creativity of three musical improvisation systems, identifying which systems are more creative than others and why. In a second case study, SPECS is used to capture initial impressions on the creativity of systems presented at a 2011 computational creativity research event. Five systems performing different creative tasks are compared and contrasted. These case studies exemplify the valuable information that can be obtained on a system’s strengths and weaknesses. SPECS gives researchers vital feedback for improving their systems’ creativity, informing further progress in computational creativity research

Evaluating computational creativity: a standardised procedure for evaluating creative systems and its application

This thesis proposes SPECS: a Standardised Procedure for Evaluating Creative Systems. No methodology has been accepted as standard for evaluating the creativity of a system in the field of computational creativity and the multi-faceted and subjective nature of creativity generates substantial definitional issues. Evaluative practice has developed a general lack of rigour and systematicity, hindering research progress. SPECS is a standardised and systematic methodology for evaluating computational creativity. It is flexible enough to be applied to a variety of different types of creative system and adaptable to specific demands in different types of creativity. In the three-stage process of evaluation, researchers are required to be specific about what creativity entails in the domain they work in and what standards they test a system’s creativity by. To assist researchers, definitional issues are investigated and a set of components representing aspects of creativity is presented, which was empirically derived using computational linguistics analysis. These components are recommended for use within SPECS, being offered as a general definition of creativity that can be customised to account for any specific priorities for creativity in a given domain. SPECS is applied in a case study for detailed comparisons of the creativity of three musical improvisation systems, identifying which systems are more creative than others and why. In a second case study, SPECS is used to capture initial impressions on the creativity of systems presented at a 2011 computational creativity research event. Five systems performing different creative tasks are compared and contrasted. These case studies exemplify the valuable information that can be obtained on a system’s strengths and weaknesses. SPECS gives researchers vital feedback for improving their systems’ creativity, informing further progress in computational creativity research

Blending under deconstruction

n/

Intrinsic Motivation in Computational Creativity Applied to Videogames

PhD thesisComputational creativity (CC) seeks to endow artificial systems with creativity. Although human creativity is known to be substantially driven by intrinsic motivation (IM), most CC systems are extrinsically motivated. This restricts their actual and perceived creativity and autonomy, and consequently their benefit to people. In this thesis, we demonstrate, via theoretical arguments and through applications in videogame AI, that computational intrinsic reward and models of IM can advance core CC goals. We introduce a definition of IM to contextualise related work. Via two systematic reviews, we develop typologies of the benefits and applications of intrinsic reward and IM models in CC and game AI. Our reviews highlight that related work is limited to few reward types and motivations, and we thus investigate the usage of empowerment, a little studied, information-theoretic intrinsic reward, in two novel models applied to game AI. We define coupled empowerment maximisation (CEM), a social IM model, to enable general co-creative agents that support or challenge their partner through emergent behaviours. Via two qualitative, observational vignette studies on a custom-made videogame, we explore CEM’s ability to drive general and believable companion and adversary non-player characters which respond creatively to changes in their abilities and the game world. We moreover propose to leverage intrinsic reward to estimate people’s experience of interactive artefacts in an autonomous fashion. We instantiate this proposal in empowerment-based player experience prediction (EBPXP) and apply it to videogame procedural content generation. By analysing think-aloud data from an experiential vignette study on a dedicated game, we identify several experiences that EBPXP could predict. Our typologies serve as inspiration and reference for CC and game AI researchers to harness the benefits of IM in their work. Our new models can increase the generality, autonomy and creativity of next-generation videogame AI, and of CC systems in other domains

Geometric Methods for Context Sensitive Distributional Semantics

PhDThis thesis describes a novel methodology, grounded in the distributional semantic paradigm, for building context sensitive models of word meaning, affording an empirical exploration of the relationship between words and concepts. Anchored in theoretical linguistic insight regarding the contextually specified nature of lexical semantics, the work presented here explores a range of techniques for the selection of subspaces of word co-occurrence dimensions based on a statistical analysis of input terms as observed within large-scale textual corpora. The relationships between word-vectors that emerge in the projected subspaces can be analysed in terms of a mapping between their geometric features and their semantic properties. The power of this modelling technique is its ability to generate ad hoc semantic relationships in response to an extemporaneous linguistic or conceptual situation. The product of this approach is a generalisable computational linguistic methodology, capable of taking input in various forms, including word groupings and sentential context, and dynamically generating output from a broad base model of word co-occurrence data. To demonstrate the versatility of the method, this thesis will present competitive empirical results on a range of established natural language tasks including word similarity and relatedness rating, metaphor and metonymy detection, and analogy completion. A range of techniques will be applied in order to explore the ways in which different aspects of projected geometries can be mapped to different semantic relationships, allowing for the discovery of a range of lexical and conceptual properties for any given input and providing a basis for an empirical exploration of distinctions between the semantic phenomena under analysis. The case made here is that the flexibility of these models and their ability to extend output to evaluations of unattested linguistic relationships constitutes the groundwork for a method for the extrapolation of dynamic conceptual relationships from large-scale textual corpora. This method is presented as a complement and a counterpoint to established distributional methods for generating lexically productive word-vectors. Where contemporary vector space models of distributional semantics have almost universally involved either the factorisation of co-occurrence matrices or the incremental learning of abstract representations using neural networks, the approach described in this thesis preserves the connection between the individual dimensions of word-vectors and statistics pertaining to observations in a textual corpus. The hypothesis tested here is that the maintenance of actual, interpretable information about underlying linguistic data allows for the contextual selection of non-normalised subspaces with more nuanced geometric features. In addition to presenting competitive results for various computational linguistic targets, the thesis will suggest that the transparency of its representations indicates scope for the application of this model to various real-world problems where an interpretable relationship between data and output is highly desirable. This, finally, demonstrates a way towards the productive application of the theory and philosophy of language to computational linguistic practice.Engineering and Physical Sciences Research Council of the UK, EP/L50483X/1

Immoral Programming: What can be done if malicious actors use language AI to launch ‘deepfake science attacks’?

The problem-solving and imitation capabilities of AI are increasing. In parallel, research addressing ethical AI design has gained momentum internationally. However, from a cybersecurity-oriented perspective in AI safety, it is vital to also analyse and counteract the risks posed by intentional malice. Malicious actors could for instance exploit the attack surface of already deployed AI, poison AI training data, sabotage AI systems at the pre-deployment stage or deliberately design hazardous AI. At a time when topics such as fake news, disinformation, deepfakes and, recently, fake science are affecting online debates in the population at large but also specifically in scientific circles, we thematise the following elephant in the room now and not in hindsight: what can be done if malicious actors use AI for not yet prevalent but technically feasible ‘deepfake science attacks’, i.e. on (applied) science itself? Deepfakes are not restricted to audio and visual phenomena, and deepfake text whose impact could be potentiated with regard to speed, scope, and scale may represent an underestimated avenue for malicious actors. Not only has the imitation capacity of AI improved dramatically, e.g. with the advent of advanced language AI such as GPT-3 (Brown et al., 2020), but generally, present-day AI can already be abused for goals such as (cyber)crime (Kaloudi and Li, 2020) and information warfare (Hartmann and Giles, 2020). Deepfake science attacks on (applied) science and engineering – which belong to the class of what we technically denote as scientific and empirical adversarial (SEA) AI attacks (Aliman and Kester, 2021) – could be instrumental in achieving such aims due to socio-psycho-technological intricacies against which science might not be immune. But if not immunity, could one achieve resilience? This chapter familiarises the reader with a complementary solution to this complex issue: a generic ‘cyborgnetic’ defence (GCD) against SEA AI attacks. As briefly introduced in Chapter 4, the term cyborgnet (which is much more general than and not to be confused with the term ‘cyborg’) stands for a generic, substrate-independent and hybrid functional unit which is instantiated e.g. in couplings of present-day AIs and humans. Amongst many others, GCD uses epistemology, cybersecurity, cybernetics, and creativity research to tailor 10 generic strategies to the concrete exemplary use case of a large language model such as GPT-3. GCD can act as a cognitively diverse transdisciplinary scaffold to defend against SEA AI attacks – albeit with specific caveats

A formal descriptive theory of software-based creative practice

PhDCreative artefacts, from concert posters to architectural plans, are often created in entirely software-based workflows. Software tools can be easily made to record all user interactions, thereby capturing the observable part of creative practice. Although recording software-based creative practice is easy, analysing it is much harder. This is especially true if one wishes to analyse the cognitive process that underlies the recorded creative practice. There are currently no clear methods for the analysis of recorded creative practice, nor are there any suitable theories of the cognition underlying creative practice that can serve as the basis for the development of such methods. This thesis develops a formal descriptive theory of the cognition underlying software-based creative practice, with the aim of informing the development of analysis of recorded creative practice. The theory, called the Software-based Creative Practice Framework (SbCPF), fits with extended and predictive views of cognition. It characterises creative practice as a process of iteratively working from an abstract idea to a concrete artefact, whereby the required lowlevel detail to decide on action is imagined in flight, during practice. Furthermore, it argues that this iterative just-in-time imagination is necessary, because of the predictive nature of the mind. The SbCPF was developed through the use of a novel method for the analysis of creative practice displayed in video tutorials. This method is based on Grounded Theory, Rhetorical Structure Theory, Gesture Theory, Category Theory, and a novel taxonomy describing the relation of action to speech. The method is applied to produce a grounded theory of the creative practice of 3D modelling and animation with the Blender software. The grounded theory forms the basis of the aforementioned formal theory. Finally, the formal theory is further illustrated, evaluated, and explored by way of implementing a computational model.Queen Mary University of London, and the EPSRC Centre for Doctoral Training in Media and Arts Technology EP/G03723X/

Usability guidelines informing knowledge visualisation in demonstrating learners' knowledge acquisition

There is growing evidence that knowledge co-creation and interactivity during learning interventions aid knowledge acquisition and knowledge transfer. However, learners have mostly been passive consumers and not co-creators of the knowledge visualisation aids created by teachers and instructional designers. As such, knowledge visualisation has been underutilised for allowing learners to construct, demonstrate and share what they have learned. The dearth of appropriate guidelines for the use of knowledge visualisation for teaching and learning is an obstacle to using knowledge visualisation in teaching and learning. This provides a rationale for this study, which aims to investigate usability-based knowledge visualisation guidelines for teaching and learning. The application context is that of Science teaching for high school learners in the Gauteng province of South Africa. Following a design-based research methodology, an artefact of usability-based knowledge visualisation guidelines was created. The artefact was evaluated by testing learners’ conformity to the visualisation guidelines. Qualitative and quantitative data was captured using questionnaires, interviews and observations. The findings indicate that the guidelines considered in this study had various degrees of impact on the visualisations produced by learners. While some made noticeable impact, for others it could be considered negligible. Within the context of high school learning, these results justify the prioritisation of usability-based knowledge visualisation guidelines. Integrating Human Computer Interaction usability principles and knowledge visualisation guidelines to create usability-based knowledge visualisation guidelines provide a novel theoretical contribution upon which scientific knowledge visualisation can be expanded.School of ComputingM. Sc. (Computing