SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

Studying free-standing conversational groups (FCGs) in unstructured social settings (e.g., cocktail party ) is gratifying due to the wealth of information available at the group (mining social networks) and individual (recognizing native behavioral and personality traits) levels. However, analyzing social scenes involving FCGs is also highly challenging due to the difficulty in extracting behavioral cues such as target locations, their speaking activity and head/body pose due to crowdedness and presence of extreme occlusions. To this end, we propose SALSA, a novel dataset facilitating multimodal and Synergetic sociAL Scene Analysis, and make two main contributions to research on automated social interaction analysis: (1) SALSA records social interactions among 18 participants in a natural, indoor environment for over 60 minutes, under the poster presentation and cocktail party contexts presenting difficulties in the form of low-resolution images, lighting variations, numerous occlusions, reverberations and interfering sound sources; (2) To alleviate these problems we facilitate multimodal analysis by recording the social interplay using four static surveillance cameras and sociometric badges worn by each participant, comprising the microphone, accelerometer, bluetooth and infrared sensors. In addition to raw data, we also provide annotations concerning individuals' personality as well as their position, head, body orientation and F-formation information over the entire event duration. Through extensive experiments with state-of-the-art approaches, we show (a) the limitations of current methods and (b) how the recorded multiple cues synergetically aid automatic analysis of social interactions. SALSA is available at http://tev.fbk.eu/salsa.Comment: 14 pages, 11 figure

The Visual Social Distancing Problem

One of the main and most effective measures to contain the recent viral outbreak is the maintenance of the so-called Social Distancing (SD). To comply with this constraint, workplaces, public institutions, transports and schools will likely adopt restrictions over the minimum inter-personal distance between people. Given this actual scenario, it is crucial to massively measure the compliance to such physical constraint in our life, in order to figure out the reasons of the possible breaks of such distance limitations, and understand if this implies a possible threat given the scene context. All of this, complying with privacy policies and making the measurement acceptable. To this end, we introduce the Visual Social Distancing (VSD) problem, defined as the automatic estimation of the inter-personal distance from an image, and the characterization of the related people aggregations. VSD is pivotal for a non-invasive analysis to whether people comply with the SD restriction, and to provide statistics about the level of safety of specific areas whenever this constraint is violated. We then discuss how VSD relates with previous literature in Social Signal Processing and indicate which existing Computer Vision methods can be used to manage such problem. We conclude with future challenges related to the effectiveness of VSD systems, ethical implications and future application scenarios.Comment: 9 pages, 5 figures. All the authors equally contributed to this manuscript and they are listed by alphabetical order. Under submissio

Designing multiplayer games to facilitate emergent social behaviours online

This paper discusses an exploratory case study of the design of games that facilitate spontaneous social interaction and group behaviours among distributed individuals, based largely on symbolic presence 'state' changes. We present the principles guiding the design of our game environment: presence as a symbolic phenomenon, the importance of good visualization and the potential for spontaneous self-organization among groups of people. Our game environment, comprising a family of multiplayer 'bumper-car' style games, is described, followed by a discussion of lessons learned from observing users of the environment. Finally, we reconsider and extend our design principles in light of our observations

F-formation Detection: Individuating Free-standing Conversational Groups in Images

Detection of groups of interacting people is a very interesting and useful task in many modern technologies, with application fields spanning from video-surveillance to social robotics. In this paper we first furnish a rigorous definition of group considering the background of the social sciences: this allows us to specify many kinds of group, so far neglected in the Computer Vision literature. On top of this taxonomy, we present a detailed state of the art on the group detection algorithms. Then, as a main contribution, we present a brand new method for the automatic detection of groups in still images, which is based on a graph-cuts framework for clustering individuals; in particular we are able to codify in a computational sense the sociological definition of F-formation, that is very useful to encode a group having only proxemic information: position and orientation of people. We call the proposed method Graph-Cuts for F-formation (GCFF). We show how GCFF definitely outperforms all the state of the art methods in terms of different accuracy measures (some of them are brand new), demonstrating also a strong robustness to noise and versatility in recognizing groups of various cardinality.Comment: 32 pages, submitted to PLOS On

How simple rules determine pedestrian behavior and crowd disasters

With the increasing size and frequency of mass events, the study of crowd disasters and the simulation of pedestrian flows have become important research areas. Yet, even successful modeling approaches such as those inspired by Newtonian force models are still not fully consistent with empirical observations and are sometimes hard to calibrate. Here, a novel cognitive science approach is proposed, which is based on behavioral heuristics. We suggest that, guided by visual information, namely the distance of obstructions in candidate lines of sight, pedestrians apply two simple cognitive procedures to adapt their walking speeds and directions. While simpler than previous approaches, this model predicts individual trajectories and collective patterns of motion in good quantitative agreement with a large variety of empirical and experimental data. This includes the emergence of self-organization phenomena, such as the spontaneous formation of unidirectional lanes or stop-and-go waves. Moreover, the combination of pedestrian heuristics with body collisions generates crowd turbulence at extreme densities-a phenomenon that has been observed during recent crowd disasters. By proposing an integrated treatment of simultaneous interactions between multiple individuals, our approach overcomes limitations of current physics-inspired pair interaction models. Understanding crowd dynamics through cognitive heuristics is therefore not only crucial for a better preparation of safe mass events. It also clears the way for a more realistic modeling of collective social behaviors, in particular of human crowds and biological swarms. Furthermore, our behavioral heuristics may serve to improve the navigation of autonomous robots.Comment: Article accepted for publication in PNA

Minds Online: The Interface between Web Science, Cognitive Science, and the Philosophy of Mind

Alongside existing research into the social, political and economic impacts of the Web, there is a need to study the Web from a cognitive and epistemic perspective. This is particularly so as new and emerging technologies alter the nature of our interactive engagements with the Web, transforming the extent to which our thoughts and actions are shaped by the online environment. Situated and ecological approaches to cognition are relevant to understanding the cognitive significance of the Web because of the emphasis they place on forces and factors that reside at the level of agent–world interactions. In particular, by adopting a situated or ecological approach to cognition, we are able to assess the significance of the Web from the perspective of research into embodied, extended, embedded, social and collective cognition. The results of this analysis help to reshape the interdisciplinary configuration of Web Science, expanding its theoretical and empirical remit to include the disciplines of both cognitive science and the philosophy of mind

From single steps to mass migration: the problem of scale in the movement ecology of the Serengeti wildebeest

A central question in ecology is how to link processes that occur over different scales. The daily interactions of individual organisms ultimately determine community dynamics, population fluctuations and the functioning of entire ecosystems. Observations of these multiscale ecological processes are constrained by various technological, biological or logistical issues, and there are often vast discrepancies between the scale at which observation is possible and the scale of the question of interest. Animal movement is characterized by processes that act over multiple spatial and temporal scales. Second-by-second decisions accumulate to produce annual movement patterns. Individuals influence, and are influenced by, collective movement decisions, which then govern the spatial distribution of populations and the connectivity of meta-populations. While the field of movement ecology is experiencing unprecedented growth in the availability of movement data, there remain challenges in integrating observations with questions of ecological interest. In this article, we present the major challenges of addressing these issues within the context of the Serengeti wildebeest migration, a keystone ecological phenomena that crosses multiple scales of space, time and biological complexity. This article is part of the theme issue ’Collective movement ecology’