The politics of visual indeterminacy in abstract AI art

In Perception Engines and Synthetic Abstractions, two generative AI art projects begun in 2018, Tom White experiments with visual abstraction to explore the indeterminacy of perception, interpretation, and agency. White’s AI systems produce images that will be interpreted as abstract artworks by human viewers, but which also confront human audiences with the realization that what is here deliberately rendered indeterminable for them will remain near-perfectly legible for AI-powered image recognition systems. This difference in perceptual and interpretive agency foregrounds an underlying politics of visual indeterminacy. White’s projects thus increase awareness of how machine vision—for example in automated online filtering systems—can diminish the horizon of what human audiences can or cannot see in an AI-driven digital cultural landscape, and how, in the process, underlying biases are normalized and human viewers become habituated to the dramatic shrinking of perceivable/viewable online image content mediated by AI

Explaining CLIP through Co-Creative Drawings and Interaction

This paper analyses a visual archive of drawings produced by an interactive robotic art installation where audience members narrated their dreams into a system powered by CLIPdraw deep learning (DL) model that interpreted and transformed their dreams into images. The resulting archive of prompt-image pairs were examined and clustered based on concept representation accuracy. As a result of the analysis, the paper proposes four groupings for describing and explaining CLIP-generated results: clear concept, text-to-text as image, indeterminacy and confusion, and lost in translation. This article offers a glimpse into a collection of dreams interpreted, mediated and given form by Artificial Intelligence (AI), showcasing oftentimes unexpected, visually compelling or, indeed, the dream-like output of the system, with the emphasis on processes and results of translations between languages, sign-systems and various modules of the installation. In the end, the paper argues that proposed clusters support better understanding of the neural model

Identifying Latent Causal Content for Multi-Source Domain Adaptation

Multi-source domain adaptation (MSDA) learns to predict the labels in target domain data, under the setting that data from multiple source domains are labelled and data from the target domain are unlabelled. Most methods for this task focus on learning invariant representations across domains. However, their success relies heavily on the assumption that the label distribution remains consistent across domains, which may not hold in general real-world problems. In this paper, we propose a new and more flexible assumption, termed \textit{latent covariate shift}, where a latent content variable $\mathbf{z}_c$ and a latent style variable $\mathbf{z}_s$ are introduced in the generative process, with the marginal distribution of $\mathbf{z}_c$ changing across domains and the conditional distribution of the label given $\mathbf{z}_c$ remaining invariant across domains. We show that although (completely) identifying the proposed latent causal model is challenging, the latent content variable can be identified up to scaling by using its dependence with labels from source domains, together with the identifiability conditions of nonlinear ICA. This motivates us to propose a novel method for MSDA, which learns the invariant label distribution conditional on the latent content variable, instead of learning invariant representations. Empirical evaluation on simulation and real data demonstrates the effectiveness of the proposed method

Soft thought (in architecture and choreography)

This article is an introduction to and exploration of the concept of ‘soft thought’. What we want to propose through the definition of this concept is an aesthetic of digital code that does not necessarily presuppose a relation with the generative aspects of coding, nor with its sensorial perception and evaluation. Numbers do not have to produce something, and do not need to be transduced into colours and sounds, in order to be considered as aesthetic objects. Starting from this assumption, our main aim will be to reconnect the numerical aesthetic of code with a more ‘abstract’ kind of feeling, the feeling of numbers indirectly felt as conceptual contagions’, that are ‘conceptually felt but not directly sensed. The following pages will be dedicated to the explication and exemplification of this particular mode of feeling, and to its possible definition as ‘soft thought’

Deep Understanding of Technical Documents : Automated Generation of Pseudocode from Digital Diagrams & Analysis/Synthesis of Mathematical Formulas

The technical document is an entity that consists of several essential and interconnected parts, often referred to as modalities. Despite the extensive attention that certain parts have already received, per say the textual information, there are several aspects that severely under researched. Two such modalities are the utility of diagram images and the deep automated understanding of mathematical formulas. Inspired by existing holistic approaches to the deep understanding of technical documents, we develop a novel formal scheme for the modelling of digital diagram images. This extends to a generative framework that allows for the creation of artificial images and their annotation. We contribute on the field with the creation of a novel synthetic dataset and its generation mechanism. We propose the conversion of the pseudocode generation problem to an image captioning task and provide a family of techniques based on adaptive image partitioning. We address the mathematical formulas’ semantic understanding by conducting an evaluating survey on the field, published in May 2021. We then propose a formal synthesis framework that utilized formula graphs as metadata, reaching for novel valuable formulas. The synthesis framework is validated by a deep geometric learning mechanism, that outsources formula data to simulate the missing a priori knowledge. We close with the proof of concept, the description of the overall pipeline and our future aims