Affordances, context and sociality

Affordances, i.e. the opportunity of actions offered by the environment, are one of the central research topics for the theoretical perspectives that view cognition as emerging from the interaction between the environment and the body. Being at the bridge between perception and action, affordances help to question a dichotomous view of perception and action. While Gibson’s view of affordances is mainly externalist, many contemporary approaches define affordances (and micro-affordances) as the product of long-term visuomotor associations in the brain. These studies have emphasized the fact that affordances are activated automatically, independently from the context and the previous intention to act: for example, affordances related to objects’ size would emerge even if the task does not require focusing on size. This emphasis on the automaticity of affordances has led to overlook their flexibility and contextual-dependency. In this contribution I will outline and discuss recent perspectives and evidence that reveal the flexibility and context-dependency of affordances, clarifying how they are modulated by the physical, cultural and social context. I will focus specifically on social affordances, i.e. on how perception of affordances might be influenced by the presence of multiple actors having different goals

The Effect of Anthropometric Properties of Self-Avatars on Action Capabilities in Virtual Reality

The field of Virtual Reality (VR) has seen a steady exponential uptake in the last decade and is being continuously incorporated into areas of popular interest like healthcare, training, recreation and gaming. This steady upward trend and prolonged popularity has resulted in numerous extravagant virtual environments, some that aim to mimic real-life experiences like combat training, while others intend to provide unique experiences that may otherwise be difficult to recreate like flying over ancient Egypt as a bird. These experiences often showcase highly realistic graphics, intuitive interactions and unique avatar embodiment scenarios with the help of various tracking sensors, high definition graphic displays, sound systems, etc. The literature suggests that estimates and affordance judgments in VR scenarios such as the ones described above are affected by the properties and the nature of the avatar embodied by the user. Therefore, to provide users with the finest experiences it is crucial to understand the interaction between the embodied self and the action capabilities afforded by it in the surrounding virtual environment. In a series of studies aimed at exploring the effect of gender matched body-scaled self-avatars on the user\u27s perception, we investigate the effect of self-avatars on the perception of size of objects in an immersive virtual environment (IVE) and how this perception affects the actions one can perform as compared to the real world. In the process, we make use of newer tracking technology and graphic displays to investigate the perceived differences between real world environments and their virtual counterparts to understand how the spatial properties of the environment and the embodied self-avatars affect affordances by means of passability judgments. We describe techniques for creation and mapping VR environments onto their real world counterparts and the creation of gender matched body-scaled self-avatars that provides real time full-body tracking. The first two studies investigate how newer graphical displays and off-the-shelf tracking devices can be utilized to create salient gender matched body-scaled self-avatars and their effect on the judgment of passability as a result of the embodied body schema. The study involves creating complex scripts that automate the process of mapping virtual worlds onto their real world counterparts within a 1cm margin of error and the creation of self-avatars that match height, limb proportions and shoulder width of the participant using tracking sensors. The experiment involves making judgments about the passability of an adjustable doorway in the real world and in a virtual to-scale replica of the real world environment. The results demonstrated that the perception of affordances in IVEs is comparable to the real world but the behavior leading to it differs in VR. Also, the body-scaled self-avatars generated provide salient information yielding performance similar to the real world. Several insights and guidelines related to creating veridical virtual environments and realistic self-avatars were achieved from this effort. The third study investigates how the presence of body-scaled self-avatars affects the perception of size of virtual handheld objects and the influence of the person-plus-virtual-object system created by lifting the said virtual object on passability. This is crucial to understand as VR simulations now often utilize self-avatars that carry objects while maneuvering through the environment. How they interact with these handheld objects can influence what they do in critical scenarios where split second decisions can change the outcome like combat training, role-playing games, first person shooting, thrilling rides, physiotherapy, etc. It has also been reported that the avatar itself can influence the perception of size of virtual objects, in turn influencing action capabilities. There is ample research on different interaction techniques to manipulate objects in a virtual world but the question about how the objects affect our action capabilities upon interaction remains unanswered, especially when the haptic feedback associated with holding a real object is mismatched or missing. The study investigates this phenomenon by having participants interact with virtual objects of different sizes and making frontal and lateral passability judgments to an adjustable aperture similar to the first experiment. The results suggest that the presence of self-avatars significantly affects affordance judgments. Interestingly, frontal and lateral judgments in IVEs seem to similar unlike the real world. Investigating the concept of embodied body schema and its influence on action-capabilities further, the fourth study looks at how embodying self-avatars that may vary slightly from your real world body affect performance and behavior in dynamic affordance scenarios. In this particular study, we change the eye height of the participants in the presence or absence of self-avatars that are either bigger, smaller or the same size as the participant. We then investigate how this change in eye height and anthropometric properties of the self-avatar affects their judgments when crossing streets with oncoming traffic in virtual reality. We also evaluate any changes in the perceived walking speed as a result of embodying altered self-avatars. The findings suggest that the presence of self-avatars results in safer crossing behavior, however scaling the eye height or the avatar does not seem to affect the perceived walking speed. A detailed discussion on all the findings can be found in the manuscript

Posture Affects Affordance Perception of Reachability in Virtual Reality

Tasks such as standing and reaching require differing levels of postural stability. Postural equilibrium is necessary to perceive the location of objects (Lee, Pacheco, & Newell, 2018). Visual perception of whether an object is within reach was investigated while standing upright. Participants viewed a 3D virtual reality (VR) environment with a stimulus object (red ball) placed at different egocentric distances. Participants made affordance judgements while in a standard stance condition as well as two separate active balance conditions (yoga tree pose, and toe-to-heel pose). Feedback on accuracy was not provided, and participants were not allowed to attempt to reach. Response time, affordance judgments (reachable, not reachable), and head movements were recorded on each trial. Consistent with recent research on reaching ability (Weast & Proffitt, 2018), the perceived action boundary occurred around 120% of arm length, indicating overestimation of perceived reaching ability. Response times increased with distance, and were shortest for the most difficult yoga tree pose. Head movement amplitude increased with increases in balance demands. Surprisingly, the coefficient of variation was comparable in the two poses that had increased balance requirements, and was more extreme in a less constrained, ostensibly easier pose for the shortest and longest distances. More complex descriptors of postural sway (i.e. multifractality) were predictive of perception while in the tree pose and the toe-to-heel pose, as compared to control stance. This demonstrates that standard measures of central tendency are not sufficient for describing multiscale interactions of postural dynamics in functional tasks

Interface design of educational games for middle school students

The purpose of this report is to identify interface design elements in educational games for a middle school audience. The four games in this report were selected based on their availability to the general public and their reputable developers. The purpose in undertaking this research is to determine interface and interaction conventions in educational games for 11-14 year olds in order to make recommendations to designers and developers. This report identifies the ways each game approaches important features such as art direction, providing feedback, portraying significant events, indicating clickable items, and event timing. It discusses the games’ techniques to guide gameplay. This report also addresses how interface design supports pedagogical goals.Curriculum and Instructio

The Problem of Mental Action

In mental action there is no motor output to be controlled and no sensory input vector that could be manipulated by bodily movement. It is therefore unclear whether this specific target phenomenon can be accommodated under the predictive processing framework at all, or if the concept of “active inference” can be adapted to this highly relevant explanatory domain. This contribution puts the phenomenon of mental action into explicit focus by introducing a set of novel conceptual instruments and developing a first positive model, concentrating on epistemic mental actions and epistemic self-control. Action initiation is a functionally adequate form of self-deception; mental actions are a specific form of predictive control of effective connectivity, accompanied and possibly even functionally mediated by a conscious “epistemic agent model”. The overall process is aimed at increasing the epistemic value of pre-existing states in the conscious self-model, without causally looping through sensory sheets or using the non-neural body as an instrument for active inference

Investigation of a holistic human-computer interaction (HCI) framework to support the design of extended reality (XR) based training simulators

In recent years, the use of Extended Reality (XR) based simulators for training has increased rapidly. In this context, there is a need to explore novel HCI-based approaches to design more effective 3D training environments. A major impediment in this research area is the lack of an HCI-based framework that is holistic and serves as a foundation to integrate the design and assessment of HCI-based attributes such as affordance, cognitive load, and user-friendliness. This research addresses this need by investigating the creation of a holistic framework along with a process for designing, building, and assessing training simulators using such a framework as a foundation. The core elements of the proposed framework include the adoption of participatory design principles, the creation of information-intensive process models of target processes (relevant to the training activities), and design attributes related to affordance and cognitive load. A new attribute related to affordance of 3D scenes is proposed (termed dynamic affordance) and its role in impacting user comprehension in data-rich 3D training environments is studied. The framework is presented for the domain of orthopedic surgery. Rigorous user-involved assessment of the framework and simulation approach has highlighted the positive impact of the HCI-based framework and attributes on the acquisition of skills and knowledge by healthcare users

Positing a space mirror mechanism: intentional understanding without action?

Recent evidence regarding a novel functionality of the mirror neuron system (MNS), a so-called 'space mirror mechanism', seems to reinforce the central role of the MNS in social cognition. According to the space mirror hypothesis, neural mirroring accounts for understanding not just what an observed agent is doing, but also the range of potential actions that a suitably located object affords an observed agent in the absence of any motor behaviour. This paper aims to show that the advocate of this space mirror hypothesis faces a crippling dilemma. Either what observed agents can do remains underdetermined by space mirror representations, and no proper understanding of action potentiality is gained; or, if it is just understanding of potential motor acts that is achieved through the sensorimotor representations generated by shared object-related affordances, the very explanatory role of space mirroring is compromised

Examining the Effects of Altered Avatars on Perception-Action in Virtual Reality

In virtual reality avatars are animated graphical representation of a person embedded in a virtual environment. Previous research has illustrated the benefits of having an avatar when perceiving aspects of virtual reality. We studied the effect that a non-faithful, or altered, avatar had on the perception of one\u27s action capabilities in VR. In Experiment 1, one group of participants acted with a normal, or faithful, avatar and the other group of participants used an avatar with an extended arm, all in virtual reality. In Experiment 2, the same methodology and procedure was used as in Experiment 1, except only the calibration phase occurred in VR, while the remaining reaches were completed in the real world. All participants performed reaches to various distances. The results of these studies show that calibration to altered dimensions of avatars is possible after receiving feedback while acting with the altered avatar. Further, calibration occurred more quickly when feedback was initially used to transition from a normal avatar to an altered avatar than when later transitioning from the altered avatar arm back to the normal avatar arm without feedback. The implications of these findings for training in virtual reality simulations and transfer back to the real world are also discussed