CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model

With the development of deep generative models, recent years have seen great success of Chinese landscape painting generation. However, few works focus on controllable Chinese landscape painting generation due to the lack of data and limited modeling capabilities. In this work, we propose a controllable Chinese landscape painting generation method named CCLAP, which can generate painting with specific content and style based on Latent Diffusion Model. Specifically, it consists of two cascaded modules, i.e., content generator and style aggregator. The content generator module guarantees the content of generated paintings specific to the input text. While the style aggregator module is to generate paintings of a style corresponding to a reference image. Moreover, a new dataset of Chinese landscape paintings named CLAP is collected for comprehensive evaluation. Both the qualitative and quantitative results demonstrate that our method achieves state-of-the-art performance, especially in artfully-composed and artistic conception. Codes are available at https://github.com/Robin-WZQ/CCLAP.Comment: 8 pages,13 figure

Artificial Intelligence as a Substitute for Human Creativity

Creativity has always been perceived as a human trait, even though the exact neural mechanisms remain unknown, it has been the subject of research and debate for a long time. The recent development of AI technologies and increased interest in AI has led to many projects capable of performing tasks that have been previously regarded as impossible without human creativity. Music composition, visual arts, literature, and science represent areas in which these technologies have started to both help and replace the creative human, with the question of whether AI can be creative and capable of creation more realistic than ever. This review aims to provide an extensive perspective over several state-of-the art technologies and applications based on AI which are currently being implemented into areas of interest closely correlated to human creativity, as well as the economic impact the development of such technologies might have on those domains

Learning of Art Style Using AI and Its Evaluation Based on Psychological Experiments

GANs (Generative adversarial networks) is a new AI technology that can perform deep learning with less training data and has the capability of achieving transformation between two image sets. Using GAN we have carried out a comparison between several art sets with different art style. We have prepared several image sets; a flower photo set (A), an art image set (B1) of Impressionism drawings, an art image set of abstract paintings (B2), an art image set of Chinese figurative paintings, (B3), and an art image set of abstract images (B4) created by Naoko Tosa, one of the authors. Transformation between set A to each of B was carried out using GAN and four image sets (B1, B2, B3, B4) was obtained. Using these four image sets we have carried out psychological experiment by asking subjects consisting of 23 students to fill in questionnaires. By analyzing the obtained questionnaires, we have found the followings. Abstract drawings and figurative drawings are clearly judged to be different. Figurative drawings in West and East were judged to be similar. Abstract images by Naoko Tosa were judged as similar to Western abstract images. These results show that AI could be used as an analysis tool to reveal differences between art genres

Controllable Multi-domain Semantic Artwork Synthesis

We present a novel framework for multi-domain synthesis of artwork from semantic layouts. One of the main limitations of this challenging task is the lack of publicly available segmentation datasets for art synthesis. To address this problem, we propose a dataset, which we call ArtSem, that contains 40,000 images of artwork from 4 different domains with their corresponding semantic label maps. We generate the dataset by first extracting semantic maps from landscape photography and then propose a conditional Generative Adversarial Network (GAN)-based approach to generate high-quality artwork from the semantic maps without necessitating paired training data. Furthermore, we propose an artwork synthesis model that uses domain-dependent variational encoders for high-quality multi-domain synthesis. The model is improved and complemented with a simple but effective normalization method, based on normalizing both the semantic and style jointly, which we call Spatially STyle-Adaptive Normalization (SSTAN). In contrast to previous methods that only take semantic layout as input, our model is able to learn a joint representation of both style and semantic information, which leads to better generation quality for synthesizing artistic images. Results indicate that our model learns to separate the domains in the latent space, and thus, by identifying the hyperplanes that separate the different domains, we can also perform fine-grained control of the synthesized artwork. By combining our proposed dataset and approach, we are able to generate user-controllable artwork that is of higher quality than existingComment: 15 pages, accepted by CVMJ, to appea

Content loss and conditional space relationship in conditional generative adversarial networks

In the machine learning community, generative models, especially generative adversarial networks (GANs) continue to be an attractive yet challenging research topic. Right after the invention of GAN, many GAN models have been proposed by the researchers with the same goal: creating better images. The first and foremost feature that a GAN model should have is that creating realistic images that cannot be distinguished from genuine ones. A large portion of the GAN models proposed to this end have a common approach which can be defined as factoring the image generation process into multiple states for decomposing the difficult task into several more manageable sub tasks. This can be realized by using sequential conditional/unconditional generators. Although images generated by sequential generators experimentally prove the effectiveness of this approach, visually inspecting the generated images are far away of being objective and it is not yet quantitatively showed in an objective manner. In this paper, we quantitatively show the effectiveness of shrinking the conditional space by using the sequential generators instead of utilizing single but large generator. At the light of the content loss we demonstrate that in sequential designs, each generator helps to shrink the conditional space, and therefore reduces the loss and the uncertainties at the generated images. In order to quantitatively validate this approach, we tried different combinations of connecting generators sequentially and/or increasing the capacity of generators and using single or multiple discriminators under four different scenarios applied to image-to-image translation tasks. Scenario-1 uses the conventional pix2pix GAN model which serves as the based line model for the rest of the scenarios. In Scenario-2, we utilized two generators connected sequentially. Each generator is identical to the one used in Scenario-1. Another possibility is just doubling the size of a single generator which is evaluated in the Scenario-3. In the last scenario, we used two different discriminators in order to train two sequentially connected generators. Our quantitative results support that simply increasing the capacity of one generator, instead of using sequential generators, does not help a lot to reduce the content loss which is used in addition to adversarial loss and hence does not create better images

Chapter Il fulmine e la “reazione nera”: disegno naturale e artificiale dei pattern tra Golgi e Simondon

The 43rd UID conference, held in Genova, takes up the theme of ‘Dialogues’ as practice and debate on many fundamental topics in our social life, especially in these complex and not yet resolved times. The city of Genova offers the opportunity to ponder on the value of comparison and on the possibilities for the community, naturally focused on the aspects that concern us, as professors, researchers, disseminators of knowledge, or on all the possibile meanings of the discipline of representation and its dialogue with ‘others’, which we have broadly catalogued in three macro areas: History, Semiotics, Science / Technology. Therefore, “dialogue” as a profitable exchange based on a common language, without which it is impossible to comprehend and understand one another; and the graphic sign that connotes the conference is the precise transcription of this concept: the title ‘translated’ into signs, derived from the visual alphabet designed for the visual identity of the UID since 2017. There are many topics which refer to three macro sessions: - Witnessing (signs and history) - Communicating (signs and semiotics) - Experimenting (signs and sciences) Thanks to the different points of view, an exceptional resource of our disciplinary area, we want to try to outline the prevailing theoretical-operational synergies, the collaborative lines of an instrumental nature, the recent updates of the repertoires of images that attest and nourish the relations among representation, history, semiotics, sciences