The Visual Social Distancing Problem

One of the main and most effective measures to contain the recent viral outbreak is the maintenance of the so-called Social Distancing (SD). To comply with this constraint, workplaces, public institutions, transports and schools will likely adopt restrictions over the minimum inter-personal distance between people. Given this actual scenario, it is crucial to massively measure the compliance to such physical constraint in our life, in order to figure out the reasons of the possible breaks of such distance limitations, and understand if this implies a possible threat given the scene context. All of this, complying with privacy policies and making the measurement acceptable. To this end, we introduce the Visual Social Distancing (VSD) problem, defined as the automatic estimation of the inter-personal distance from an image, and the characterization of the related people aggregations. VSD is pivotal for a non-invasive analysis to whether people comply with the SD restriction, and to provide statistics about the level of safety of specific areas whenever this constraint is violated. We then discuss how VSD relates with previous literature in Social Signal Processing and indicate which existing Computer Vision methods can be used to manage such problem. We conclude with future challenges related to the effectiveness of VSD systems, ethical implications and future application scenarios.Comment: 9 pages, 5 figures. All the authors equally contributed to this manuscript and they are listed by alphabetical order. Under submissio

Motion prediction and interaction localisation of people in crowds

PhDThe ability to analyse and predict the movement of people in crowded scenarios can be of fundamental importance for tracking across multiple cameras and interaction localisation. In this thesis, we propose a person re-identification method that takes into account the spatial location of cameras using a plan of the locale and the potential paths people can follow in the unobserved areas. These potential paths are generated using two models. In the first, people’s trajectories are constrained to pass through a set of areas of interest (landmarks) in the site. In the second we integrate a goal-driven approach to the Social Force Model (SFM), initially introduced for crowd simulation. SFM models the desire of people to reach specific interest points (goals) in a site, such as exits, shops, seats and meeting points while avoiding walls and barriers. Trajectory propagation creates the possible re-identification candidates, on which association of people across cameras is performed using spatial location of the candidates and appearance features extracted around a person’s head. We validate the proposed method in a challenging scenario from London Gatwick airport and compare it to state-of-the-art person re-identification methods. Moreover, we perform detection and tracking of interacting people in a framework based on SFM that analyses people’s trajectories. The method embeds plausible human behaviours to predict interactions in a crowd by iteratively minimising the error between predictions and measurements. We model people approaching a group and restrict the group formation based on the relative velocity of candidate group members. The detected groups are then tracked by linking their centres of interaction over time using a buffered graph-based tracker. We show how the proposed framework outperforms existing group localisation techniques on three publicly available datasets

Analysing Crowd Behaviours using Mobile Sensing

PhDResearchers have examined crowd behaviour in the past by employing a variety of methods including ethnographic studies, computer vision techniques and manual annotation-based data analysis. However, because of the resources to collect, process and analyse data, it remains difficult to obtain large data sets for study. Mobile phones offer easier means for data collection that is easy to analyse and can preserve the user’s privacy. The aim of this thesis is to identify and model different qualities of social interactions inside crowds using mobile sensing technology. This Ph.D. research makes three main contributions centred around the mobile sensing and crowd sensing area. Firstly, an open-source licensed mobile sensing framework is developed, named SensingKit, that is capable of collecting mobile sensor data from iOS and Android devices, supporting most sensors available in modern smartphones. The framework has been evaluated in a case study that investigates the pedestrian gait synchronisation phenomenon. Secondly, a novel algorithm based on graph theory is proposed capable of detecting stationary social interactions within crowds. It uses sensor data available in a modern smartphone device, such as the Bluetooth Smart (BLE) sensor, as an indication of user proximity, and accelerometer sensor, as an indication of each user’s motion state. Finally, a machine learning model is introduced that uses multi-modal mobile sensor data extracted from Bluetooth Smart, accelerometer and gyroscope sensors. The validation was performed using a relatively large dataset with 24 participants, where they were asked to socialise with each other for 45 minutes. By using supervised machine learning based on gradient-boosted trees, a performance increase of 26.7% was achieved over a proximity-based approach. Such model can be beneficial to the design and implementation of in-the-wild crowd behavioural analysis, design of influence strategies, and algorithms for crowd reconfiguration.UK Defence Science & Technology Laboratory (DSTL

Multi-target tracking and performance evaluation on videos

PhDMulti-target tracking is the process that allows the extraction of object motion patterns of interest from a scene. Motion patterns are often described through metadata representing object locations and shape information. In the first part of this thesis we discuss the state-of-the-art methods aimed at accomplishing this task on monocular views and also analyse the methods for evaluating their performance. The second part of the thesis describes our research contribution to these topics. We begin presenting a method for multi-target tracking based on track-before-detect (MTTBD) formulated as a particle filter. The novelty involves the inclusion of the target identity (ID) into the particle state, which enables the algorithm to deal with an unknown and unlimited number of targets. We propose a probabilistic model of particle birth and death based on Markov Random Fields. This model allows us to overcome the problem of the mixing of IDs of close targets. We then propose three evaluation measures that take into account target-size variations, combine accuracy and cardinality errors, quantify long-term tracking accuracy at different accuracy levels, and evaluate ID changes relative to the duration of the track in which they occur. This set of measures does not require pre-setting of parameters and allows one to holistically evaluate tracking performance in an application-independent manner. Lastly, we present a framework for multi-target localisation applied on scenes with a high density of compact objects. Candidate target locations are initially generated by extracting object features from intensity maps using an iterative method based on a gradient-climbing technique and an isocontour slicing approach. A graph-based data association method for multi-target tracking is then applied to link valid candidate target locations over time and to discard those which are spurious. This method can deal with point targets having indistinguishable appearance and unpredictable motion. MT-TBD is evaluated and compared with state-of-the-art methods on real-world surveillanceThis work was supported by the EU, under the FP7 project APIDIS (ICT-216023) and the Artemis JU and TSB as part of the COPCAMS project (332913)

Detection and tracking of groups in crowd

We propose a method to detect and track interacting people by employing a framework based on a Social Force Model (SFM). The method embeds plausible human behaviors to predict interactions in a crowd by iteratively minimizing the error between predictions and measurements. We model people approaching a group and restrict the group formation based on the relative velocity of candidate group members. The detected groups are then tracked by linking their interaction centers over time using a buffered graph-based tracker. We show how the proposed framework outperforms existing group localization techniques on three publicly available datasets, with improvements of up to 13% on group detection. 1