What Can I Do Around Here? Deep Functional Scene Understanding for Cognitive Robots

For robots that have the capability to interact with the physical environment through their end effectors, understanding the surrounding scenes is not merely a task of image classification or object recognition. To perform actual tasks, it is critical for the robot to have a functional understanding of the visual scene. Here, we address the problem of localizing and recognition of functional areas from an arbitrary indoor scene, formulated as a two-stage deep learning based detection pipeline. A new scene functionality testing-bed, which is complied from two publicly available indoor scene datasets, is used for evaluation. Our method is evaluated quantitatively on the new dataset, demonstrating the ability to perform efficient recognition of functional areas from arbitrary indoor scenes. We also demonstrate that our detection model can be generalized onto novel indoor scenes by cross validating it with the images from two different datasets

Affordances, context and sociality

Affordances, i.e. the opportunity of actions offered by the environment, are one of the central research topics for the theoretical perspectives that view cognition as emerging from the interaction between the environment and the body. Being at the bridge between perception and action, affordances help to question a dichotomous view of perception and action. While Gibson’s view of affordances is mainly externalist, many contemporary approaches define affordances (and micro-affordances) as the product of long-term visuomotor associations in the brain. These studies have emphasized the fact that affordances are activated automatically, independently from the context and the previous intention to act: for example, affordances related to objects’ size would emerge even if the task does not require focusing on size. This emphasis on the automaticity of affordances has led to overlook their flexibility and contextual-dependency. In this contribution I will outline and discuss recent perspectives and evidence that reveal the flexibility and context-dependency of affordances, clarifying how they are modulated by the physical, cultural and social context. I will focus specifically on social affordances, i.e. on how perception of affordances might be influenced by the presence of multiple actors having different goals

Co-thought and Co-speech Gestures Are Generated by the Same Action Generation Process

We thank Lucy Foulkes, Rachel Furness, Valentina Lee, and Zeshu Shao for their help with data collection; Paraskevi Argyriou for her help with reliability checks of gesture coding; and Agnieszka Konopka and Josje Praamstra for their help with proofreading this article.Peer reviewedPostprin

Dynamic mapping strategies for interactive art installations: an embodied combined HCI HRI HHI approach

This paper proposes a theoretical framework for dealing with the paradigm of interactivity in new media art, and how the broad use of the term in different research fields can lead to some misunderstandings. The paper addresses a conceptual view on how we can implement interaction in new media art from an embodied approach that unites views from HCI, HRI and HHI. The focus is on an intuitive mapping of a multitude of sensor data and to extend upon this using the paradigm of (1) finite state machines (FSM) to address dynamic mapping strategies, (2) mediality to address aisthesis and (3) embodiment to address valid mapping strategies originated from natural body movements. The theory put forward is illustrated by a case study

Differential neural dynamics underling pragmatic and semantic affordance processing in macaque ventral premotor cortex

Premotor neurons play a fundamental role in transforming physical properties of observed objects, such as size and shape, into motor plans for grasping them, hence contributing to "pragmatic" affordance processing. Premotor neurons can also contribute to "semantic" affordance processing, as they can discharge differently even to pragmatically identical objects depending on their behavioural relevance for the observer (i.e. edible or inedible objects). Here, we compared the response of monkey ventral premotor area F5 neurons tested during pragmatic (PT) or semantic (ST) visuomotor tasks. Object presentation responses in ST showed shorter latency and lower object selectivity than in PT. Furthermore, we found a difference between a transient representation of semantic affordances and a sustained representation of pragmatic affordances at both the single neuron and population level. Indeed, responses in ST returned to baseline within 0.5 s whereas in PT they showed the typical sustained visual-to-motor activity during Go trials. In contrast, during No-go trials, the time course of pragmatic and semantic information processing was similar. These findings suggest that premotor cortex generates different dynamics depending on pragmatic and semantic information provided by the context in which the to-be-grasped object is presented

Affordances and Information Systems Research: Taking Stock and Moving Forward

The term affordance appears with increasing frequency in the Information Systems (IS) literature. Nevertheless, those who study information technologies/information systems (IT/IS) via the affordance lens often have different views about its origin, meaning, and appropriate application in IS research. In turn, not spelling out the related assumptions and boundaries inherent in these diverse views may have hindered a wider and more cumulative adoption of the affordance lens in IS research. This paper offers a potential solution by (1) synthesizing the ecological psychology literature to suggest five key modules of the affordance concept relevant to IS research and (2) taking stock of IS research that has employed the affordance concept and classifying it according to its focus on three key affordance elements: IT artifact, user, and context. Finally, this paper presents a set of challenges, opportunities, and recommendations regarding how IS researchers can advance affordance-based research in the field

Towards a framework for investigating tangible environments for learning

External representations have been shown to play a key role in mediating cognition. Tangible environments offer the opportunity for novel representational formats and combinations, potentially increasing representational power for supporting learning. However, we currently know little about the specific learning benefits of tangible environments, and have no established framework within which to analyse the ways that external representations work in tangible environments to support learning. Taking external representation as the central focus, this paper proposes a framework for investigating the effect of tangible technologies on interaction and cognition. Key artefact-action-representation relationships are identified, and classified to form a structure for investigating the differential cognitive effects of these features. An example scenario from our current research is presented to illustrate how the framework can be used as a method for investigating the effectiveness of differential designs for supporting science learning

The influence of patient absorptive capacity and perception of technology characteristics on patients' satisfaction with medical services in Guangzhou, China: an affordance theory based approach

Background: Based on the theory of Affordance, this thesis proposes two analytical dimensions of Subjective Affordance (SA) and Behavioral Affordance (BA).The research aims to explore the influence of patients’ Absorptive Capacity (AC) and Technology Characteristics Perception (TCP) on their SA and BA of the WeChat client-side app in community hospitals so as to further evaluate the SA and BA influence on patients’ satisfaction of medical service (PS). Research subjects are patients who have used the community hospital WeChat client-side information platform in 28 community health service centers in nine districts of Guangzhou in the South of China. A questionnaire with 4 sections and 44 questions was designed and distributed. The effective questionnaire data were checked for descriptive analysis and a path model with 5 observed variables (AC,TCP, SA,BA,PS). was developed. Results: 1. A total of 410 questionnaires was distributed, of which 400 have been collected and 387 of them were valid and analyzed. 2. There is a significant difference among variable scores of patients with different ages and educational background. 3. The group of patients who use the information platform more actively have higher scores in the 5 variables. 4. TCP has a direct influence on PS while BA plays an intermediary role. It was found that the mediation effect of SA is not valid. 5. AC has no direct or indirect influence on PS. Conclusions: Age, educational background, and TCP (evaluation of the innovation characteristics of technology) play an important role in the final implementation of technology application behavior (BA), and the effective application of technology can indeed improve satisfaction with the medical services (PS)Tendo por base a teoria da Affordance, esta tese analisa duas dimensões desta teoria: a Affordance subjetiva (AS) e a Affordance comportamental (AC). A investigação tem por objetivo estudar a influência da Capacidade de Absorção (CA) e da Percepção das Características Tecnológicas (PCT) na AS e na AC dos pacientes estudados em relação à utilização da aplicação Wechat como plataforma de informação e fornecimento de serviços em Centros de Saúde, a fim de avaliar em que medida a AS e a AC influenciam a sua satisfação com os serviços médicos (SP). Os sujeitos do estudo são pacientes que utilizaram a aplicação WeChat na sua interação com Centros de Saúde. Foi administrado um questionário contendo quatro partes e um total de 44 questões em 28 destes centros localizados em 9 distritos da cidade de Cantão no sul da China. Os dados foram depois analisados tendo sido concebido um modelo com base nas 5 variáveis observadas (AS, AC, CA, PCT e PS). Resultados: 1. Dos 410 questionários distribuídos recolheram-se 400 dos quais 387 foram considerados válidos para análise; 2. Os resultados revelaram que existe uma diferença significativa entre os pacientes de diferentes idades e diferente formação académica; 3. O grupo de pacientes que mais utiliza a aplicação tem valores mais altos nas cinco variáveis analisadas; 4. A PCT tem uma influência direta na SP enquanto que a CA desempenha um papel mediador. Os resultados demonstraram ainda que o efeito moderador da AS não se verifica; 5. A AC não tem qualquer influência direta ou indireta na SP. Conclusões: A idade, a formação académica e a PCT (percepção das características inovadoras da tecnologia) desempenham um papel importante na Affordance Comportamental (AC) e a utilização efetiva da tecnologia pode contribuir para melhorar a satisfação dos pacientes com os serviços médicos (SP)