MetaSpace II: Object and full-body tracking for interaction and navigation in social VR

MetaSpace II (MS2) is a social Virtual Reality (VR) system where multiple users can not only see and hear but also interact with each other, grasp and manipulate objects, walk around in space, and get tactile feedback. MS2 allows walking in physical space by tracking each user's skeleton in real-time and allows users to feel by employing passive haptics i.e., when users touch or manipulate an object in the virtual world, they simultaneously also touch or manipulate a corresponding object in the physical world. To enable these elements in VR, MS2 creates a correspondence in spatial layout and object placement by building the virtual world on top of a 3D scan of the real world. Through the association between the real and virtual world, users are able to walk freely while wearing a head-mounted device, avoid obstacles like walls and furniture, and interact with people and objects. Most current virtual reality (VR) environments are designed for a single user experience where interactions with virtual objects are mediated by hand-held input devices or hand gestures. Additionally, users are only shown a representation of their hands in VR floating in front of the camera as seen from a first person perspective. We believe, representing each user as a full-body avatar that is controlled by natural movements of the person in the real world (see Figure 1d), can greatly enhance believability and a user's sense immersion in VR.Comment: 10 pages, 9 figures. Video: http://living.media.mit.edu/projects/metaspace-ii

Taking empathy online

Despite its long history of investigating sociality, phenomenology has, to date, said little about online sociality. The phenomenological tradition typically claims that empathy is the fundamental way in which we experience others and their experiences. While empathy is discussed almost exclusively in the context of face-to-face interaction, I claim that we can empathetically perceive others and their experiences in certain online situations. Drawing upon the phenomenological distinction between the physical, objective body and the expressive, lived body, I: (i) highlight that empathy involves perceiving the other’s expressive, lived body, (ii) show that the lived body is not tied to the physical body and that empathy can take place outside of face-to-face interactions, and (iii) argue that the lived body can enter online space and is empathetically available to others there. I explore two ways in which the other’s lived body enters online space and can be empathetically perceived: first, in cases where our face-to-face encounter is technologically-mediated over video link and, second, by showing how the other’s texts, as speech, can form part of the other’s lived body. Investigating empathy online not only furthers our understanding of online encounters but also leads to a refined conception of empathy more generally

Simulating activities: Relating motives, deliberation, and attentive coordination

Activities are located behaviors, taking time, conceived as socially meaningful, and usually involving interaction with tools and the environment. In modeling human cognition as a form of problem solving (goal-directed search and operator sequencing), cognitive science researchers have not adequately studied “off-task” activities (e.g., waiting), non-intellectual motives (e.g., hunger), sustaining a goal state (e.g., playful interaction), and coupled perceptual-motor dynamics (e.g., following someone). These aspects of human behavior have been considered in bits and pieces in past research, identified as scripts, human factors, behavior settings, ensemble, flow experience, and situated action. More broadly, activity theory provides a comprehensive framework relating motives, goals, and operations. This paper ties these ideas together, using examples from work life in a Canadian High Arctic research station. The emphasis is on simulating human behavior as it naturally occurs, such that “working” is understood as an aspect of living. The result is a synthesis of previously unrelated analytic perspectives and a broader appreciation of the nature of human cognition. Simulating activities in this comprehensive way is useful for understanding work practice, promoting learning, and designing better tools, including human-robot systems

Emotion-focused therapy

Emotion-focused therapy (EFT), also known as process-experiential therapy, integrates active therapeutic methods from gestalt and other humanistic therapies within the frame of a person-centred relationship (Elliott, Watson, Goldman & Greenberg, 2004). EFT updates person-centred and gestalt therapies by incorporating contemporary emotion theory and affective neuroscience, dialectical constructivism, and contemporary attachment theory. In this chapter, I review the current status of EFT, summarising its history, theory, practice, and outcome evidence

Spectators’ aesthetic experiences of sound and movement in dance performance

In this paper we present a study of spectators’ aesthetic experiences of sound and movement in live dance performance. A multidisciplinary team comprising a choreographer, neuroscientists and qualitative researchers investigated the effects of different sound scores on dance spectators. What would be the impact of auditory stimulation on kinesthetic experience and/or aesthetic appreciation of the dance? What would be the effect of removing music altogether, so that spectators watched dance while hearing only the performers’ breathing and footfalls? We investigated audience experience through qualitative research, using post-performance focus groups, while a separately conducted functional brain imaging (fMRI) study measured the synchrony in brain activity across spectators when they watched dance with sound or breathing only. When audiences watched dance accompanied by music the fMRI data revealed evidence of greater intersubject synchronisation in a brain region consistent with complex auditory processing. The audience research found that some spectators derived pleasure from finding convergences between two complex stimuli (dance and music). The removal of music and the resulting audibility of the performers’ breathing had a significant impact on spectators’ aesthetic experience. The fMRI analysis showed increased synchronisation among observers, suggesting greater influence of the body when interpreting the dance stimuli. The audience research found evidence of similar corporeally focused experience. The paper discusses possible connections between the findings of our different approaches, and considers the implications of this study for interdisciplinary research collaborations between arts and sciences

Experiential fantasies, prediction, and enactive minds

A recent surge of work on prediction-driven processing models--based on Bayesian inference and representation-heavy models--suggests that the material basis of conscious experience is inferentially secluded and neurocentrically brain bound. This paper develops an alternative account based on the free energy principle. It is argued that the free energy principle provides the right basic tools for understanding the anticipatory dynamics of the brain within a larger brain-body-environment dynamic, viewing the material basis of some conscious experiences as extensive--relational and thoroughly world-involving

Being-in-the-world-with: Presence Meets Social And Cognitive Neuroscience

In this chapter we will discuss the concepts of “presence” (Inner Presence) and “social presence” (Co-presence) within a cognitive and ecological perspective. Specifically, we claim that the concepts of “presence” and “social presence” are the possible links between self, action, communication and culture. In the first section we will provide a capsule view of Heidegger’s work by examining the two main features of the Heideggerian concept of “being”: spatiality and “being with”. We argue that different visions from social and cognitive sciences – Situated Cognition, Embodied Cognition, Enactive Approach, Situated Simulation, Covert Imitation - and discoveries from neuroscience – Mirror and Canonical Neurons - have many contact points with this view. In particular, these data suggest that our conceptual system dynamically produces contextualized representations (simulations) that support grounded action in different situations. This is allowed by a common coding – the motor code – shared by perception, action and concepts. This common coding also allows the subject for natively recognizing actions done by other selves within the phenomenological contents. In this picture we argue that the role of presence and social presence is to allow the process of self-identification through the separation between “self” and “other,” and between “internal” and “external”. Finally, implications of this position for communication and media studies are discussed by way of conclusion

The uncanny body: from medical to aesthetic abnormality

In this essay I explore a possibility of experiential synthesis of the medicalized abnormal body with its aesthetic images. A personal narrative about meeting extreme abnormality serves as an introduction into theorizing aesthetic abnormality. The essay builds its argument on the phenomenological grounds; I therefore approach corporeality with Edmund Husserl and Maurice Merleau-Ponty. In turn, Max Ernst introduces an aesthetic frame for the subsequent examination of uncanny surreality. Two exemplars of the surreal body, Joel Witkin’s “Satiro” and Don DeLillo’s “Body Artist,” intend to substantiate the preceding theoretic. The study shows how the encounter with the abnormal embodiment may suspend normalized modes of constitution to provoke uncanny experience