Poetics of Artificial Intelligence in Art Practice: (Mis)apprehended Bodies Remixed as Language

With a focus on the last five years, art employing artificial intelligence (AI) has been defined by a spectrum of activity, from the deep learning explorations of neural network researchers to artists critiquing the broader social implications of AI technology. There is an emergence of and increasing access to new tools and techniques for repurposing and manipulating material in unprecedented ways in art. At the same time, there is a dearth of language outside the scientific domain with which to discuss it. A combination of contextual review, comparison of artistic approaches, and practical projects explores speculation that the conceptual repertoire for remix studies can open up to art enabled by AI and machine learning (ML). This research contributes a practical, conceptual and combinatorial approach for artists who do not necessarily have a grounding in engineering or computer science. A bricolage methodology—described by Annette Markham as combining serendipity, proximity and contingency—reveals the poetics of AI-enabled art in the form of an assemblage of techniques that understands poetics as active making (poiesis) as well as an approach to manipulating language. The poetic capacity of AI/ML is understood as an emergent form of remix technique, with the ML at its core functioning like a remix engine. This practice based research presents several projects founded on an interrelation of body, text Bruce Gilchrist. Poetics of Artificial Intelligence in Art Practice: (Mis)apprehended Bodies Remixed as Language. 3 and predictive technology enabled by a human-action-recognition algorithm combined with a natural language generator. A significant number of artistic works have been made around object and facial recognition, while very little (if any) artist activity has focused on human-action-recognition. For this reason, I concentrated my research there

MC2: MPEG-7 content modelling communities

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityThe use of multimedia content on the web has grown significantly in recent years. Websites such as Facebook, YouTube and Flickr cater for enormous amounts of multimedia content uploaded by users. This vast amount of multimedia content requires comprehensive content modelling otherwise retrieving relevant content will be challenging. Modelling multimedia content can be an extremely time consuming task that may seem impossible particularly when undertaken by individual users. However, the advent of Web 2.0 and associated communities, such as YouTube and Flickr, has shown that users appear to be more willing to collaborate in order to take on enormous tasks such as multimedia content modelling. Harnessing the power of communities to achieve comprehensive content modelling is the primary focus of this research. The aim of this thesis is to explore collaborative multimedia content modelling and in particular the effectiveness of existing multimedia content modelling tools, taking into account the key development challenges of existing collaborative content modelling research and the associated modelling tools. Four research objectives are pursued in order to achieve this; first, design a user experiment to study users’ tagging behaviour with existing multimedia tagging tools and identify any relationships between such user behaviour; second, design and develop a framework for MPEG-7 content modelling communities based on the results of the experiment; third, implement an online service as a proof of concept of the framework; fourth, validate the framework through the online service during a repeat of the initial user experiment. This research contributes first, a conceptual model of user behaviour visualised as a fuzzy cognitive map and, second, an MPEG-7 framework for multimedia content modelling communities (MC2) and its proof of concept as an online service. The fuzzy cognitive model embodies relationships between user tagging behaviour and context and provides an understanding of user priorities in the description of content features and the relationships that exist between them. The MC2 framework, developed based on the fuzzy cognitive model, is deep-rooted in user content modelling behaviour and content preferences. A proof of concept of the MC2 framework is implemented as an online service in which all metadata is modelled using MPEG-7. The online service is validated, first, empirically with the same group of users and through the same experiment that led to the development of the fuzzy cognitive model and, second, functionally against the folksonomy and MPEG-7 content modelling tools used in the initial experiment. The validation demonstrates that MC2 has the advantages without the shortcomings of existing multimedia tagging tools by harnessing the ease of use of folksonomy tools while producing comprehensive structured metadata.Supported by UK Engineering and Physical Sciences Research Council (EPSRC

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Proceedings of the 1st International Conference on Live Coding

Open Access peer reviewed papers on live coding published at the 1st International Conference on Live Coding (ICLC) in Leeds

The influence of social interactions on innovative endeavors in online communities

Online communities offer great potential for sourcing future innovations. While organizations search for inspiration and innovations outside their organizational boundaries to stay competitive, individuals innovate to solve their own needs and subsequently freely reveal these innovations. Online communities constitute a virtual space for individuals to share ideas, socially interact, collaborate, and build on others’ ideas. In this dissertation, I investigate how these social interactions influence the generation of ideas and the ongoing idea development in online communities. The three studies of this dissertation use two unique large datasets that allowed the investigation of social interactions and their contents. In doing so, topic modeling and social network analysis techniques build the methodical foundation to measure latent content representations of the information that is exchanged in online communities. Regarding the generation of new ideas, this dissertation includes two empirical studies that focus on the content that individuals access through their social peers. The first study reveals that the combination of redundant and non-redundant information favors idea newness. In particular, brokers accessing diverse social information benefit from redundant content for generating new ideas. In contrast, non-redundant contents have detrimental effects on brokers’ social non- redundancy regarding brokers’ idea newness. The second study takes a time-dependent view on social interactions and finds that a temporal separation between inspiration and focus on specific contents leads to more innovative outcomes of individuals engaging and innovating in online communities. By focusing on the ongoing collaborative idea development process in online communities, the third study investigates how social influences shape the trajectory ideas take after they got initially shared. The findings of the third study show that social impact theory helps explain how social influences affect the development directions of ideas in online communities. By taking different perspectives on innovative endeavors in online communities, this dissertation contributes to the literature on online communities, social networks, and user innovation. Specifically, this dissertation emphasizes the importance of social interactions for innovations and this relationships’ dependence on the actual content, timing, and social impact of social interactions

The art of bots: A practice-based study of the multiplicity, entanglements and figuration of sociocomputational assemblages

This thesis examines and analyses an emerging art practice known as artbots. Artbots are internet-based software applications that are imbued with character and configured to engage and entertain online audiences. This form of practice, and the community of practice leading it, was found to be underrepresented and misunderstood. I argue that this artform is original and warrants a more thorough understanding. This thesis develops a conceptual framework for understanding artbots that focuses on and enables questioning around pertinent aspects of the practice. A wide range of literature was reviewed to provide theoretical underpinnings towards this framework, including literature on algorithm studies, science and technology studies, and software architecture. The devised framework examines artbot case studies through the notions of multiplicity, entanglement, and figuration, having understood artbots as heterogenous sociocomputational assemblages comprised of software components and human intraactivity. The research followed a varied methodology that encompassed participant observation and my own practice-based experiments in producing artbots. The study resulted in several original works. In addition, a showcase titled Art of Bots brought together key proponents and artbots, further providing material that is analysed in this thesis. The study helped identify and discuss artbots with attention to how they utilise modular software components in novel arrangements, how normative human and nonhuman relations of interaction are being eschewed in favour of entangled interrelations, and how artbots challenge common narratives dictating technological constructs by inventing unique characters and figurations