Emergent interfaces: vague, complex, bespoke and embodied interaction between humans and computers

Most human-computer interfaces are built on the paradigm of manipulating abstract representations. This can be limiting when computers are used in artistic performance or as mediators of social connection, where we rely on qualities of embodied thinking: intuition, context, resonance, ambiguity, fluidity. We explore an alternative approach to designing interaction that we call the emergent interface: interaction leveraging unsupervised machine learning to replace designed abstractions with contextually-derived emergent representations. The approach offers opportunities to create interfaces bespoke to a single individual, to continually evolve and adapt the interface in line with that individual’s needs and affordances, and to bridge more deeply with the complex and imprecise interaction that defines much of our non-digital communication. We explore this approach through artistic research rooted in music, dance and AI with the partially emergent system Sonified Body. The system maps the moving body into sound using an emergent representation of the body derived from a corpus of improvised movement from the first author. We explore this system in a residency with three dancers. We reflect on the broader implications and challenges of this alternative way of thinking about interaction, and how far it may help users avoid being limited by the assumptions of a system’s designer

Between Fact and Fabrication: How Visual Art Might Nurture Environmental Consciousness

Previous studies have highlighted the communicative limitations of artistic visualizations, which are often too conceptual or interpretive to enhance public understanding of (and volition to act upon) scientific climate information. This seems to suggest a need for greater factuality/concreteness in artistic visualization projects, which may indeed be the case. However, in this paper, we synthesize insights from environmental psychology, the psychology of art, and intermediate disciplines like eco-aesthetics, to argue that artworks—defined by their counterfactual qualities—can be effective for stimulating elements of environmental consciousness. We also argue that different artworks may yield different effects depending on how they combine counter/factual strategies. In so doing, we assert that effective artistic perceptualization—here expressed as affectivization—exceeds the faithful translation of facts from one mode to another, and cannot be encapsulated in a single example of un/successful art

Emergence in interactive artistic visualization

This research draws on theories of emergence to inform the creation of an artistic and direct visualization. This is an interactive artwork and drawing tool for creative participant experiences. Emergence is characteristically creative and many different models of emergence exist. It is therefore possible to effect creativity through the application of emergence mechanisms from these different disciplines. A review of theories of emergence and examples of visualization in the arts, is provided. An art project led by the author is then discussed in this context. This project, Iterative Intersections, is a collaboration with community artists from Cerebral Palsy League. It has resulted in a number of creative outcomes including the interactive art application, Of me with me. Analytical discussion of this work shows how its construction draws on aspects of experience design, fractal and emergent theory to effect perceptual emergence and creative experience as well as to facilitate self-efficacy