Social relation recognition in egocentric photostreams

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper proposes an approach to automatically categorize the social interactions of a user wearing a photo-camera (2fpm), by relying solely on what the camera is seeing. The problem is challenging due to the overwhelming complexity of social life and the extreme intra-class variability of social interactions captured under unconstrained conditions. We adopt the formalization proposed in Bugental’s social theory, that groups human relations into five social domains with related categories. Our method is a new deep learning architecture that exploits the hierarchical structure of the label space and relies on a set of social attributes estimated at frame level to provide a semantic representation of social interactions. Experimental results on the new EgoSocialRelation dataset demonstrate the effectiveness of our proposal.Peer ReviewedPostprint (author's final draft

Social Relation Recognition in Egocentric Photostreams

This paper proposes an approach to automatically categorize the social interactions of a user wearing a photo-camera 2fpm, by relying solely on what the camera is seeing. The problem is challenging due to the overwhelming complexity of social life and the extreme intra-class variability of social interactions captured under unconstrained conditions. We adopt the formalization proposed in Bugental's social theory, that groups human relations into five social domains with related categories. Our method is a new deep learning architecture that exploits the hierarchical structure of the label space and relies on a set of social attributes estimated at frame level to provide a semantic representation of social interactions. Experimental results on the new EgoSocialRelation dataset demonstrate the effectiveness of our proposal.Comment: Accepted at ICIP 201

Social Distance Evaluation in Human Parietal Cortex

Across cultures, social relationships are often thought of, described, and acted out in terms of physical space (e.g. “close friends” “high lord”). Does this cognitive mapping of social concepts arise from shared brain resources for processing social and physical relationships? Using fMRI, we found that the tasks of evaluating social compatibility and of evaluating physical distances engage a common brain substrate in the parietal cortex. The present study shows the possibility of an analytic brain mechanism to process and represent complex networks of social relationships. Given parietal cortex's known role in constructing egocentric maps of physical space, our present findings may help to explain the linguistic, psychological and behavioural links between social and physical space

Consciousness as Recursive, Spatiotemporal Self-Location

At the phenomenal level, consciousness arises in a consistently coherent fashion as a singular, unified field of recursive self-awareness (subjectivity) with explicitly orientational characteristics—that of a subject located both spatially and temporally in an egocentrically-extended domain. Understanding these twin elements of consciousness begins with the recognition that ultimately (and most primitively), cognitive systems serve the biological self-regulatory regime in which they subsist. The psychological structures supporting self-located subjectivity involve an evolutionary elaboration of the two basic elements necessary for extending self-regulation into behavioral interaction with the environment: an orientative reference frame which consistently structures ongoing interaction in terms of controllable spatiotemporal parameters, and processing architecture that relates behavior to homeostatic needs via feedback. Over time, constant evolutionary pressures for energy efficiency have encouraged the emergence of anticipative feedforward processing mechanisms, and the elaboration, at the apex of the sensorimotor processing hierarchy, of self-activating, highly attenuated recursively-feedforward circuitry processing the basic orientational schema independent of external action output. As the primary reference frame of active waking cognition, this recursive self-locational schema processing generates a zone of subjective self-awareness in terms of which it feels like something to be oneself here and now. This is consciousness-as-subjectivity

Toddler-Inspired Visual Object Learning

Real-world learning systems have practical limitations on the quality and quantity of the training datasets that they can collect and consider. How should a system go about choosing a subset of the possible training examples that still allows for learning accurate, generalizable models? To help address this question, we draw inspiration from a highly efficient practical learning system: the human child. Using head-mounted cameras, eye gaze trackers, and a model of foveated vision, we collected first-person (egocentric) images that represents a highly accurate approximation of the "training data" that toddlers' visual systems collect in everyday, naturalistic learning contexts. We used state-of-the-art computer vision learning models (convolutional neural networks) to help characterize the structure of these data, and found that child data produce significantly better object models than egocentric data experienced by adults in exactly the same environment. By using the CNNs as a modeling tool to investigate the properties of the child data that may enable this rapid learning, we found that child data exhibit a unique combination of quality and diversity, with not only many similar large, high-quality object views but also a greater number and diversity of rare views. This novel methodology of analyzing the visual "training data" used by children may not only reveal insights to improve machine learning, but also may suggest new experimental tools to better understand infant learning in developmental psychology

Semiotic Dynamics Solves the Symbol Grounding Problem

Language requires the capacity to link symbols (words, sentences) through the intermediary of internal representations to the physical world, a process known as symbol grounding. One of the biggest debates in the cognitive sciences concerns the question how human brains are able to do this. Do we need a material explanation or a system explanation? John Searle's well known Chinese Room thought experiment, which continues to generate a vast polemic literature of arguments and counter-arguments, has argued that autonomously establishing internal representations of the world (called 'intentionality' in philosophical parlance) is based on special properties of human neural tissue and that consequently an artificial system, such as an autonomous physical robot, can never achieve this. Here we study the Grounded Naming Game as a particular example of symbolic interaction and investigate a dynamical system that autonomously builds up and uses the semiotic networks necessary for performance in the game. We demonstrate in real experiments with physical robots that such a dynamical system indeed leads to a successful emergent communication system and hence that symbol grounding and intentionality can be explained in terms of a particular kind of system dynamics. The human brain has obviously the right mechanisms to participate in this kind of dynamics but the same dynamics can also be embodied in other types of physical systems