Autoencoding beyond pixels using a learned similarity metric

We present an autoencoder that leverages learned representations to better measure similarities in data space. By combining a variational autoencoder with a generative adversarial network we can use learned feature representations in the GAN discriminator as basis for the VAE reconstruction objective. Thereby, we replace element-wise errors with feature-wise errors to better capture the data distribution while offering invariance towards e.g. translation. We apply our method to images of faces and show that it outperforms VAEs with element-wise similarity measures in terms of visual fidelity. Moreover, we show that the method learns an embedding in which high-level abstract visual features (e.g. wearing glasses) can be modified using simple arithmetic

Images Big and Soft: The Digital Archive Rendered Cinematic

 In his recent immersive media art project titled Machine Hallucinations, artist Refik Anadol collected over 100 million images of New York City from social media and, using machine learning, created a 30-minute immersive experimental cinema experience that visualized the database. On his website, Anadol explains that computation allows “a novel form of synesthetic storytelling through its multilayered manipulation of a vast visual archive beyond the conventional limits of the camera and the existing cinematographic techniques.” With this project, Anadol demonstrates a tendency shared by a group of contemporary media artists who work at the intersection of cinema and the digital archive and who use machine learning and generative adversarial networks to render specific somatic experiences in relation to thousands of images. This essay discusses this shared focus by examining projects by three artists who use computational processes to assemble, manipulate, and then exhibit an archive of images as a part of their practice and output, translating the archival into the cinematic. The projects are significant in their evocation of what has been named by Ingrid Hoelzl the “soft-image” or “post-image,” shifting from the single image as a solid, stable representation within a collection of similarly single images, to that of the distributed, in-process experiential image. Further, each example approaches the creation of the collection with varied intentions; and each presents the material in disparate modalities that, while deeply connected to the cinematic, produce very different sensory experiences. Together, the examples offer a perspective on the archive in our current moment’s transition from representation to computation