Pathway to Future Symbiotic Creativity

This report presents a comprehensive view of our vision on the development path of the human-machine symbiotic art creation. We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist (Turing Artists) to a Machine artist in its own right. We begin with an overview of the limitations of the Turing Artists then focus on the top two-level systems, Machine Artists, emphasizing machine-human communication in art creation. In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations. The rapid development of immersive environment and further evolution into the new concept of metaverse enable symbiotic art creation through unprecedented flexibility of bi-directional communication between artists and art manifestation environments. By examining the latest sensor and XR technologies, we illustrate the novel way for art data collection to constitute the base of a new form of human-machine bidirectional communication and understanding in art creation. Based on such communication and understanding mechanisms, we propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle rather than the traditional "end-to-end" dogma. By proposing a new form of inverse reinforcement learning model, we outline the platform design of machine artists, demonstrate its functions and showcase some examples of technologies we have developed. We also provide a systematic exposition of the ecosystem for AI-based symbiotic art form and community with an economic model built on NFT technology. Ethical issues for the development of machine artists are also discussed

Multi-Sensory Interaction for Blind and Visually Impaired People

This book conveyed the visual elements of artwork to the visually impaired through various sensory elements to open a new perspective for appreciating visual artwork. In addition, the technique of expressing a color code by integrating patterns, temperatures, scents, music, and vibrations was explored, and future research topics were presented. A holistic experience using multi-sensory interaction acquired by people with visual impairment was provided to convey the meaning and contents of the work through rich multi-sensory appreciation. A method that allows people with visual impairments to engage in artwork using a variety of senses, including touch, temperature, tactile pattern, and sound, helps them to appreciate artwork at a deeper level than can be achieved with hearing or touch alone. The development of such art appreciation aids for the visually impaired will ultimately improve their cultural enjoyment and strengthen their access to culture and the arts. The development of this new concept aids ultimately expands opportunities for the non-visually impaired as well as the visually impaired to enjoy works of art and breaks down the boundaries between the disabled and the non-disabled in the field of culture and arts through continuous efforts to enhance accessibility. In addition, the developed multi-sensory expression and delivery tool can be used as an educational tool to increase product and artwork accessibility and usability through multi-modal interaction. Training the multi-sensory experiences introduced in this book may lead to more vivid visual imageries or seeing with the mind’s eye

Electronic Imaging & the Visual Arts. EVA 2019 Florence

The Publication is following the yearly Editions of EVA FLORENCE. The State of Art is presented regarding the Application of Technologies (in particular of digital type) to Cultural Heritage. The more recent results of the Researches in the considered Area are presented. Information Technologies of interest for Culture Heritage are presented: multimedia systems, data-bases, data protection, access to digital content, Virtual Galleries. Particular reference is reserved to digital images (Electronic Imaging & the Visual Arts), regarding Cultural Institutions (Museums, Libraries, Palace - Monuments, Archaeological Sites). The International Conference includes the following Sessions: Strategic Issues; New Science and Culture Developments & Applications; New Technical Developments & Applications; Cultural Activities – Real and Virtual Galleries and Related Initiatives, Access to the Culture Information. One Workshop regards Innovation and Enterprise. The more recent results of the Researches at national and international level are reported in the Area of Technologies and Culture Heritage, also with experimental demonstrations of developed Activities

From rule-based to learning-based image-conditional image generation

Visual contents, such as movies, animations, computer games, videos and photos, are massively produced and consumed nowadays. Most of these contents are the combination of materials captured from real-world and contents synthesized by computers. Particularly, computer-generated visual contents are increasingly indispensable in modern entertainment and production. The generation of visual contents by computers is typically conditioned on real-world materials, driven by the imagination of designers and artists, or a combination of both. However, creating visual contents manually are both challenging and labor intensive. Therefore, enabling computers to automatically or semi-automatically synthesize needed visual contents becomes essential. Among all these efforts, a stream of research is to generate novel images based on given image priors, e.g., photos and sketches. This research direction is known as image-conditional image generation, which covers a wide range of topics such as image stylization, image completion, image fusion, sketch-to-image generation, and extracting image label maps. In this thesis, a set of novel approaches for image-conditional image generation are presented. The thesis starts with an exemplar-based method for facial image stylization in Chapter 2. This method involves a unified framework for facial image stylization based on a single style exemplar. A two-phase procedure is employed, where the first phase searches a dense and semantic-aware correspondence between the input and the exemplar images, and the second phase conducts edge-preserving texture transfer. While this algorithm has the merit of requiring only a single exemplar, it is constrained to face photos. To perform generalized image-to-image translation, Chapter 3 presents a data-driven and learning-based method. Inspired by the dual learning paradigm designed for natural language translation [115], a novel dual Generative Adversarial Network (DualGAN) mechanism is developed, which enables image translators to be trained from two sets of unlabeled images from two domains. This is followed by another data-driven method in Chapter 4, which learns multiscale manifolds from a set of images and then enables synthesizing novel images that mimic the appearance of the target image dataset. The method is named as Branched Generative Adversarial Network (BranchGAN) and employs a novel training method that enables unconditioned generative adversarial networks (GANs) to learn image manifolds at multiple scales. As a result, we can directly manipulate and even combine latent manifold codes that are associated with specific feature scales. Finally, to provide users more control over image generation results, Chapter 5 discusses an upgraded version of iGAN [126] (iGANHD) that significantly improves the art of manipulating high-resolution images through utilizing the multi-scale manifold learned with BranchGAN

Artificial Intelligence in the Creative Industries: A Review

This paper reviews the current state of the art in Artificial Intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically Machine Learning (ML) algorithms, is provided including Convolutional Neural Network (CNNs), Generative Adversarial Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement Learning (DRL). We categorise creative applications into five groups related to how AI technologies are used: i) content creation, ii) information analysis, iii) content enhancement and post production workflows, iv) information extraction and enhancement, and v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, machine learning-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of machine learning in domains with fewer constraints, where AI is the `creator', remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human centric -- where it is designed to augment, rather than replace, human creativity

Protected at work but not at home: para-occupational ‘take-home’ herbicide residue exposure risks amongst forestry workers families in South Africa

Para-occupational 'take-home’ exposure amongst worker’s families in Low-and middle-income countries (LMICs) is not well characterised. This is concerning as research shows the association between long-term low-dose herbicide exposure and the development of adverse health effects. This study explored 'take-home’ herbicide residue exposure risks amongst the families of Working for Water (WfW) forestry workers in the Western Cape, South Africa using aspects of the community-based participatory research (CBPR) approach photovoice. In addition, a documentary review of the existing WfW programme policies and regulations was undertaken to assess whether required practices supported or prevented the risk of 'taking-home’ herbicide residues. The results of the documentary review revealed that workplace policies and regulations did not address 'take-home’ exposure risks. Photovoice findings highlighted low compliance to safety practices (e.g., not adhering to PPE requirements) at worksites, and this was identified as the main risk factor for 'take-home’ exposure amongst worker’s families. It was noted that the transient nature of forestry work impacted on worker’s ability to carry out hygiene practices as decontamination facilities were not available at worksites for worker’s to use before going home. As a result, all workers took their personal protective equipment (PPE) home. Worker’s after work behaviours (e.g., wearing PPE inside the home) and home hygiene practices (e.g., laundering PPE separately from household laundry) varied. That is, some worker’s carried out protective practices whilst others did not. This was largely attributed to the workplace policies and regulations which did not cover 'take-home’ exposure risks as informed by the national legislation which has not established standards and regulations related to 'take-home’ exposure risks. Evidence from this study demonstrated the existence of workers’ 'taking-home’ herbicide residue and exposing their families to potential health risks from low-dose exposures

I'll Tell the Story My Way! Multi-perspective, Multimodal Storytelling in an Elementary Classroom

The project investigated how elementary students engaged in creating 21st-century narratives with multiple perspective storylines enriched with images, videos and sound. The project was implemented as three substudies - Study A, B & C, in specific instructional contexts. Study A & B employed the use of digital technology and Study C did not. All were conducted in two Toronto elementary schools over a period of six months with a total of forty-four students (comprised of 31 girls and 13 boys) aged 10 to 12, and with the collaboration of their teachers. The data collection took the form of participant observation (Spradley,1980) and detailed writing analysis. The initial procedure used a pictographic map of the structure of Snow White from which 4 class-created stories were generated, resulting in 55 students stories told from the various characters perspectives. The results showed the projects activities empowered the students and engaged them. Perspective taking in their stories allowed students to examine the lives of others, emotionally process the story, and empathize with their story characters predicaments. Findings from the project were 1) students who wrote on computers produced more text and were more reflective of their thoughts; 2) students did not collaborate in their writing when using computers while students writing in a traditional way with pencils and paper notebooks did; 3) boys and girls had distinctive narrative styles even when using the same storys content and structure; and 4) students did not use the technology as a tool to enhance personal creativity but rather as a substitute. Perspective writing may be easy to implement into practice and there is an indication that the method is widely applicable