MoSculp: Interactive Visualization of Shape and Time

We present a system that allows users to visualize complex human motion via 3D motion sculptures---a representation that conveys the 3D structure swept by a human body as it moves through space. Given an input video, our system computes the motion sculptures and provides a user interface for rendering it in different styles, including the options to insert the sculpture back into the original video, render it in a synthetic scene or physically print it. To provide this end-to-end workflow, we introduce an algorithm that estimates that human's 3D geometry over time from a set of 2D images and develop a 3D-aware image-based rendering approach that embeds the sculpture back into the scene. By automating the process, our system takes motion sculpture creation out of the realm of professional artists, and makes it applicable to a wide range of existing video material. By providing viewers with 3D information, motion sculptures reveal space-time motion information that is difficult to perceive with the naked eye, and allow viewers to interpret how different parts of the object interact over time. We validate the effectiveness of this approach with user studies, finding that our motion sculpture visualizations are significantly more informative about motion than existing stroboscopic and space-time visualization methods.Comment: UIST 2018. Project page: http://mosculp.csail.mit.edu

Clayful Phenomenology and material engagement: explorations in contemporary cognitive archaeology

The thesis explores the phenomenology of creative cognition from the viewpoint of a contemporary ceramic workshop. Over five chapters, six clay sculptural projects track the development of a method I call ‘clayful phenomenology’. Informed by Material Engagement Theory (MET), the method takes sculptural development (normally understood to be the realization of an artist’s vision) and reformulates it so that a body of clay becomes a transient, diffuse, knowledge-producing assembly. The reformulation replaces subjective experience with a systemic, phenomenological proposal for extended sentience. Clayful Phenomenology turns material culture from an object of study into a method for investigating its own creative becoming. Videos, photos and written notes record the materialization and evolution of ideation as it is enacted in the gestural relationship between clay and hand, what Malafouris calls “creative thinging”. This account of sculpting by sculpting gives unique access to the three principles of MET and focuses on how: 1. thoughts, feelings and sensations establish themselves through gestural activity in the workshop rather than neural activity in the brain, thus extending the mind; 2. intention develops through the making and breaking of habitual practices imbuing material with agency; 3. signification, enacted and ideated through material change, enables each project to learn itself into existence. Clayful phenomenology gives reason to question the meaning of ‘cognitive’ in ‘cognitive archaeology’ by suggesting the discipline might move away from retrofitting cognitive science models to past human thinking and towards using the archaeological study of material culture to challenge neuro-centric conceptions of the mind. The first three chapters develop the method by elaborating five contemporary examples of creative thinging. The final two chapters introduce a form of experimental cognitive archaeology during which clayful phenomenology explores the enactive signification of a prehistoric artefact: a Jōmon flame pot. This diffractive analogical approach does not attempt to uncover past meanings but to make sense of the archaeological record by creating new experiences of its traces in the present. The thesis concludes with a review of extended sentience in relation to the ownership of feelings and letting go of affective intentions

The Simplicity of Material Objects

In my dissertation, I advance and develop an unorthodox account of ordinary material objects: Aristotelian Parts Nihilism. According to my theory, ordinary material objects, strictly speaking, do not have proper parts: they are extended simples sharing the exact location of their constituting portions of matter. The present construction has two main theoretical benefits. On the one hand, it preserves the modal intuition according to which hylomorphs and their constituting portions of matter are numerically different. As a nice consequence, it allows philosophers getting rid of counterparts to account for transworld identity of objects. On the other hand, and differently from the most part of multi-thingist theories, it is fully compatible with Classical Extensional Mereology. The dissertation is divided into four chapters. In the first chapter, I revise some arguments against counterpart theory, and thus give indirect reason to prefer a standard account of transworld genuine identity. In the second chapter, I revise the major multi-thingist theories available and find them incompatible with Classical Extensional Mereology. Mereological hylomorphism is safe from this criticism, but arguably falls prey of a circularity of dependence. Then, I advance and describe Aristotelian Parts Nihilism. In the third chapter, I explore the issues with persistence and location, and show that Aristotelian Parts Nihilism fits Transdurantism well. In the fourth and last chapter, I defend the theory by some potential issues of causal overdetermination and coincidence. Importantly, I also explain how we can say ordinary material objects to be complex while still lacking proper parts

‘A Swarm of Sound’ : Audiovisual Immersion in Björk’s VR Video ‘Family’

Author's accepted manuscript.This article explores the idea of audiovisual immersion through the portal of the virtual reality music video. Our focus falls on a close reading of Björk’s video, ‘Family’, which addresses questions of immersion in relation to user-experience, staging, and technological innovation. This article draws on the authors’ responses to the video by considering the implications of VR immersion in a new generation of music video productions. As part of the methodology on offer, a model for music analysis is devised for conceptualising virtual audiovisual space (VAVS) and the inextricable relationships between production and compositional design.acceptedVersio

DEMONSTRATION: MONSTROUS OPERATIONS IN SCULPTURAL PRODUCTION AND DISPLAY

Sculpture is an equivocal medium, at once occupying the abstract space of representation and the real space of the viewer. This dual nature often produces an uncanny or monstrous experience for the viewer who feels drawn into the space of the work, and who is met by its disruptive, but evasive presence. This monstrous condition is revealed in modem and contemporary sculptural practices that have sought to complicate the dynamics of the relationship that sculpture has had with the furniture used in its production and display. Manipulations of the pedestal and the workbench, in various degrees of integration with the work, demonstrate (a word connected to revealing and monstrosity) this monstrous condition by providing both a transition and a barrier at the borders of meaning. In fulfillment of the Project-Based Stream of the PhD in Art and Visual Culture, the material in this thesis consists of three parts. The first part is a written thesis that utilizes an image from the 1931 film version of Frankenstein as a model for looking at sculptural practices and the furniture of its production and display. The second is a record of my studio research, which is based in sculpture and is directly engaged in the questions described here. Workbench forms used to produce sculptural artifacts are then used as ‘pedestals’ in the context of an exhibition. This work culminates in an exhibition at the McIntosh Gallery, in London Ontario. The third part documents Parker Branch, an ongoing collaborative curatorial project of which I am a part. The project consists of a small museum space that mounts a rotating exhibition program of found objects, with an emphasis on lateral diversions in meaning engendered by manipulations of traditional taxonomic systems. What is shared among these projects is an engagement with material artifacts and the mechanisms by which they are displayed. Each project explores the ways in which those mechanisms shape the production of meaning through various corruptions in linear development

THINGNESS: COMPOSITIONAL STRATEGIES FOR EMERGING VIRTUAL SONIC OBJECTS

This selection of ten artworks represents the conclusion of my aesthetic and conceptual interests related to composition for the last three years. This research has been focused on the sound composition of immaterial vivid objects, as abstract articulated xeno entities which emerges from the listening. After introducing key concepts and influences from the repertoire and literature, this thesis explores an essential concept called ‘thingness’, which constitutes the main core in the full discourse of my artistic research. My definition of objects starts from this principle, which is unfolded in four main categories: thingness in relation with sound, the listener, time, and space. Thingness has been applied in two types of works: sound installations and mediabased pieces, and likewise extended to collaborations, challenging and strengthening my views within pieces of a wider aesthetic range

Interactive Placemaking: Creativity and User Experience at Urban Installations

Interactive placemaking offers many new opportunities for the consideration and design of urban place, particularly through offering engagement with the creativity and experiences of local populations. This thesis explores the recent area of interactive placemaking from analytical and creative perspectives in developing a framework for both describing behaviours and producing design guidelines. The thesis begins by drawing together and building on existing work in HCI and Design to provide an initial framework for the analysis of interactive place. The framework combines theoretical and practical threads to look at two central questions: how do people behave in interactive places, and how do we design such interactive places? The assumption underlying the thesis is that users' behaviours in public spaces can be observed, abstracted, and formalised to the extent that designers can use in designing and evaluating interactive public installations. In observational studies of four interactive installations the thesis progressively develops the original framework so that it is usable, accessible and understandable by designers. The thesis then goes on to evaluate how twenty design students used the framework to realise a number of public interactive installations in the city of Chicago. The thesis concludes by combining insights from existing research and insights from the studies conducted to present a comprehensive and robust framework structure. The main contribution of the thesis is thus a new generative and evaluative framework for interactive placemaking

Expanding the Bounds of Critique: Kant, Benjamin and the Coming Philosophy

This project attempts to trace a continuity which runs from Walter Benjamin’s early writings on language and the coming philosophy through to his Arcades Project. Benjamin’s early writings on the Kantian critical system stipulate the need to bring the clarity and consistency of the critical system into relation with time, ephemera and history. This project argues that Benjamin’s early demands are developed via his encounter with the literary techniques of surrealism and the artistic techniques of Baudelaire’s Monsieur G. Ultimately, this work contends that Benjamin’s Arcades Project attempts to synthesize both the techniques developed in Benjamin’s middle period and the goals put forward in his early writings

Transmedial Narration

This open access book is a methodical treatise on narration in different types of media. A theoretical rather than a historical study, Transmedial Narration is relevant for an understanding of narration in all times, including our own. By reconstructing the theoretical framework of transmedial narration, this book enables the inclusion of all kinds of communicative media forms on their own terms. The treatise is divided into three parts. Part I presents established and newly developed concepts that are vital for formulating a nuanced theoretical model of transmedial narration. Part II investigates the specific transmedial media characteristics that are most central for realizing narratives in a plenitude of different media types. Finally, Part III contains brief studies in which the narrative potentials of painting, instrumental music, mathematical equations, and guided tours are illuminated with the aid of the theoretical framework developed throughout the book. Suitable for advanced students and scholars, this book provides tools to disentangle the narrative potential of any form of communication