What was Monet seeing while painting? Translating artworks to photo-realistic images

State of the art Computer Vision techniques exploit the availability of large-scale datasets, most of which consist of images captured from the world as it is. This brings to an incompatibility between such methods and digital data from the artistic domain, on which current techniques under-perform. A possible solution is to reduce the domain shift at the pixel level, thus translating artistic images to realistic copies. In this paper, we present a model capable of translating paintings to photo-realistic images, trained without paired examples. The idea is to enforce a patch level similarity between real and generated images, aiming to reproduce photo-realistic details from a memory bank of real images. This is subsequently adopted in the context of an unpaired image-to-image translation framework, mapping each image from one distribution to a new one belonging to the other distribution. Qualitative and quantitative results are presented on Monet, Cezanne and Van Gogh paintings translation tasks, showing that our approach increases the realism of generated images with respect to the CycleGAN approach

Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation

The applicability of computer vision to real paintings and artworks has been rarely investigated, even though a vast heritage would greatly benefit from techniques which can understand and process data from the artistic domain. This is partially due to the small amount of annotated artistic data, which is not even comparable to that of natural images captured by cameras. In this paper, we propose a semantic-aware architecture which can translate artworks to photo-realistic visualizations, thus reducing the gap between visual features of artistic and realistic data. Our architecture can generate natural images by retrieving and learning details from real photos through a similarity matching strategy which leverages a weakly-supervised semantic understanding of the scene. Experimental results show that the proposed technique leads to increased realism and to a reduction in domain shift, which improves the performance of pre-trained architectures for classification, detection, and segmentation. Code is publicly available at: https://github.com/aimagelab/art2real

Home: A Process Based Visual Exploration Of The Sentiment Of Attachment

This thesis is a studio-based inquiry in which I explored the theme of home by developing a process research through art creation. I created a series of five paintings using water-based media and water collected from each place that I associate with the sentiments of home. Through research of artists, development of a process, and reflection on the process and the artworks, I not only gained deeper insight into my personal sentiments of home, but also drew implications for my teaching

Image-to-Image Translation to Unfold the Reality of Artworks: an Empirical Analysis

State-of-the-art Computer Vision pipelines show poor performances on artworks and data coming from the artistic domain, thus limiting the applicability of current architectures to the automatic understanding of the cultural heritage. This is mainly due to the difference in texture and low-level feature distribution between artistic and real images, on which state-of-the-art approaches are usually trained. To enhance the applicability of pre-trained architectures on artistic data, we have recently proposed an unpaired domain translation approach which can translate artworks to photo-realistic visualizations. Our approach leverages semantically-aware memory banks of real patches, which are used to drive the generation of the translated image while improving its realism. In this paper, we provide additional analyses and experimental results which demonstrate the effectiveness of our approach. In particular, we evaluate the quality of generated results in the case of the translation of landscapes, portraits and of paintings coming from four different styles using automatic distance metrics. Also, we analyze the response of pre-trained architecture for classification, detection and segmentation both in terms of feature distribution and entropy of prediction, and show that our approach effectively reduces the domain shift of paintings. As an additional contribution, we also provide a qualitative analysis of the reduction of the domain shift for detection, segmentation and image captioning

A Phenomenological approach to media art environments: The Immersive art experience and the Finnish art scene

This research focuses on immersive art, defined as a multimedia experience where visitors interact with artwork whilst immersed in a range of sensory experiences. In this dissertation, I investigate the immersive art experience from the perspective of art history, social theory, and media studies situated within a phenomenological theoretical framework. I present a comparative analysis of forms of immersive spatiality, including projected moving-image art, spatial environments, participatory installations, video art installations and interactive environments in the international art scene. One of my objectives is to emphasise the role of video art in the development of interactive and immersive art environments. The growing importance of spectators for giving meaning to the artwork allows immersivity to be analysed in relation to the notions of spectacle and spectatorship. I connect disciplines, practices and concepts by adopting principles from Maurice Merleau-Ponty’s phenomenological writings. Spatiality and motility are pivotal points in immersive experiences. Immersive art, as an embodied mutual experience, materialises the phenomenological concepts of spectatorship, corporeality, motility, porosity, chiasm, and encounter. I have selected a group of relevant Finnish artists from different generations to characterise the development of media art and, particularly, immersive media art in an international context. The group includes Eija-Liisa Ahtila, Lauri Astala, Laura Beloff, Hanna Haaslahti, Tuomas A. Laitinen, Erkka Nissinen, and Marjatta Oja. I examine the historical dissemination of phenomenology in Finland and a renewed interest in the 1990s which coincided with the spatialisation of video art and the emergence of immersivity. I also investigate the opening of Kiasma Museum of Contemporary Art and its impact on Finnish culture, and the recent Amos Rex Museum, specifically built for immersive exhibitions. Regarding the unstable nature of media art, I analyse the changes in displaying art collections and exhibitions, the new commitments of art museums and the innovative directions taken by media conservators. My examination of immersive art, with its performativity and transience, reveals environmentally friendly and sustainable aspects.Fenomenologinen tulokulma mediataideympäristöihin. Immersiivinen taidekokemus ja Suomen taidekenttä Tämä tutkimus käsittelee immersiivistä taidetta multimediaalisena kokemuksena. Immersiossa kävijät ovat erilaisten aistimellisten kokemusten ympäröiminä vuorovaikutuksessa taiteen kanssa. Tutkin väitöskirjassani immersiivistä taidekokemusta fenomenologisessa teoriakehyksessä taidehistorian, yhteiskuntateorian ja mediatutkimuksen näkökulmasta. Esitän vertailevan analyysin immersiivisistä tilallisuuden muodoista, joihin sisällytän liikkuvan kuvan projisoinnit, tilateokset, osallistavat installaatiot, videoinstallaatiot ja interaktiiviset ympäristöt kansainvälisen taidekentän ilmiöinä. Yhtenä pyrkimyksenäni on painottaa videotaiteen merkitystä interaktiivisen ja immersiivisen taiteen kehityksessä. Katsojien kasvava rooli taideteoksen merkityksen muodostuksessa tarjoaa perustan immersion analyysille nimenomaan spektaakkelin ja katsojuuden viitekehyksessä. Yhdistän eri tieteenaloja, käytäntöjä ja käsitteitä toisiinsa Maurice Merleau-Pontyn fenomenologisten kirjoitusten avulla. Tilallisuus ja liike ovat immersiivisten kokemusten ytimessä. Jaettuna ruumiillisena kokemuksena immersiivinen taide ilmentää materiaalisesti fenomenologisia katsojuuden, ruumiillisuuden, liikkeessä olemisen, huokoisuuden, kiasman ja kohtaamisen käsitteitä. Olen valinnut joukon eri sukupolvia edustavia suomalaistaiteilijoita hahmot-taakseni mediataiteen ja erityisesti immersiivisen mediataiteen kansainvälisiä kehityskulkuja. Heihin lukeutuvat Eija-Liisa Ahtila, Lauri Astala, Laura Beloff, Hanna Haaslahti, Tuomas A. Laitinen, Erkka Nissinen ja Marjatta Oja. Käsittelen fenomenologian saapumista Suomeen sekä siihen 1990-luvulla videotaiteen tilallistumisen ja immersion esiin nousun yhteydessä uudelleen virinnyttä mielenkiintoa. Tarkastelen myös Nykytaiteen museo Kiasman perustamista ja sen vaikutusta suomalaiseen kulttuuriin, samoin kuin vastikään avattua Amos Rex -taidemuseota, joka on rakennettu erityisesti immersiivisiä näyttelyitä silmällä pitäen. Analysoin muutoksia taidekokoelmien ja näyttelyiden esillepanossa, taidemuseoiden uudenlaisia sitoumuksia ja mediataiteen kuratoinnin uutta luovia suuntia suhteessa mediataiteen nopeasti muuttuvaan luonteeseen. Painottamalla performatiivisuutta ja hetkellisyyttä nostan immersiivisen taiteen analyysissani näkyville sen ympäristöystävällisiä ja kestäviä ulottuvuuksia

A sketch of Peirce’s Firstness and its significance to art

This essay treats the growth and development of Charles S. Peirce’s three categories, particularly studying the qualities of Peirce’s Firstness, a basic formula of “airy-nothingness” (CP: 6.455) serving as fragment to Secondness and Thirdness. The categories of feeling, willing, and knowing are not separate entities but work in interaction within the three interpretants. Interpretants are triadomaniac elements through the adopted, revised, or changed habits of belief. In works of art, the first glance of Firstness arouses the spontaneous responses of musement, expressing emotions without the struggle and resistance of factual Secondness, and not yet involving logical Thirdness. The essential qualities of a loose or vague word, color, or sound give the fugitive meanings in Firstness. The flavor, brush, timbre, color, point, line, tone or touch of the First qualities of an aesthetic object is too small a base to build the logic of aesthetic judgment. The genesis art is explained by Peirce’s undegeneracy growing into group and individual interpretants and building into the passages and whole forms of double and single forms of degeneracy. The survey of the flash of Firstness is exemplified in a variety of artworks in language, music, sculpture, painting, and film. This analysis is a preliminary aid to further studies of primary Firstness in the arts

The Drowning World : The visual culture of climate change

A challenging question today is how to understand and act on climate change. Previous analyses of the public outreach of the climate sciences have concluded that the urgent communication of climate change is inadequate. It is foremost the invisibility of carbon dioxide and the lack of a tangible relationship between current emissions and future effects that have been seen as the main challenge to visually represent. The Drowning World questions how the communication problem is articulated, and the analysis focuses on the supplementary images that come with this scientific communication, including cover images to reports, backgrounds to diagrams, or graphic design elements. The conclusion is that even if the scientific images might fail to communicate the complexity of the climate issue, the supplementary images, and the way the story of our changing world is told, manage to bring a feeling of change with them. Images of water are especially recurring, as are projects that use immersive environments like virtual reality, and these representations compete for attention in the media noise of modern society, a world that “drowns” the viewers in auditory and visual stimuli. Thus there are many reasons for the title of this thesis – The Drowning World

Aligning Figurative Paintings With Their Sources for Semantic Interpretation

This paper reports steps in probing the artistic methods of figurative painters through computational algorithms. We explore a comparative method that investigates the relation between the source of a painting, typically a photograph or an earlier painting, and the painting itself. A first crucial step in this process is to find the source and to crop, standardize and align it to the painting so that a comparison becomes possible. The next step is to apply different low-level algorithms to construct difference maps for color, edges, texture, brightness, etc. From this basis, various subsequent operations become possible to detect and compare features of the image, such as facial action units and the emotions they signify. This paper demonstrates a pipeline we have built and tested using paintings by a renowned contemporary painter Luc Tuymans. We focus in this paper particularly on the alignment process, on edge difference maps, and on the utility of the comparative method for bringing out the semantic significance of a painting

The Illustrated Masterpiece: Teaching the Artist Biography to Elementary School Children Using the Illustrations in Picture Books

Senior Project submitted to The Division of Arts of Bard College. Arts programs, particularly ones that focus on education, are constantly losing their funding. Even as Michelle Obama, the former First Lady of the United States, stated that “Arts education is not a luxury, it’s a necessity. It’s really the air many of these kids breathe[1]”, programs that benefit these kids are seeing their budgets slashed, and classroom educators are unable to get the training that they need to provide fulfilling educational experiences. In an ideal world, museums would be free to everyone who wants to learn about art, and teachers would be paid at least six figures a year. But since that doesn’t seem to be happening any time soon, it is important for us to find ways to provide kids with wholistic, culturally-rich art experiences within their very own classrooms. I think that picture books could be a way of doing this. There are a surprising amount of picture books that focus on elements of art history. In this case study, I am going to focus on books that tell a biographical story and books that contain illustrations rather than print reproductions of works, because I am interested in the choices that illustrators make to show specific aspects of the artwork, because the biography is one of the major methods of art history, and because I remember, as a child, enjoying illustrated books far more than books with print reproductions (the series of artists biographies by Mike Venezia[2] come to mind). Within this project, I plan on looking at and analyzing several books about four different artists, each a key member in the canon of western art history, and each am artist with a distinct style to learn from. Through looking at these books about Pablo Picasso, Henri Matisse, and Frida Kahlo, I hope to learn about the ways that the illustrators choose to introduce the artists and their unique styles to elementary school audiences, and the ways that these introductions can be implemented within classrooms, even by teachers that have not received extensive training in art history. [1] Evidence. Cultural Learning Alliance. February 16, 2018. Accessed March 22, 2018. https://culturallearningalliance.org.uk/evidence/. [2] The Worlds Greatest Artists. Mike Venezia. Accessed March 22, 2018. http://www.mikevenezia.com/artists/