Towards a Practical Pedestrian Distraction Detection Framework using Wearables

Pedestrian safety continues to be a significant concern in urban communities and pedestrian distraction is emerging as one of the main causes of grave and fatal accidents involving pedestrians. The advent of sophisticated mobile and wearable devices, equipped with high-precision on-board sensors capable of measuring fine-grained user movements and context, provides a tremendous opportunity for designing effective pedestrian safety systems and applications. Accurate and efficient recognition of pedestrian distractions in real-time given the memory, computation and communication limitations of these devices, however, remains the key technical challenge in the design of such systems. Earlier research efforts in pedestrian distraction detection using data available from mobile and wearable devices have primarily focused only on achieving high detection accuracy, resulting in designs that are either resource intensive and unsuitable for implementation on mainstream mobile devices, or computationally slow and not useful for real-time pedestrian safety applications, or require specialized hardware and less likely to be adopted by most users. In the quest for a pedestrian safety system that achieves a favorable balance between computational efficiency, detection accuracy, and energy consumption, this paper makes the following main contributions: (i) design of a novel complex activity recognition framework which employs motion data available from users' mobile and wearable devices and a lightweight frequency matching approach to accurately and efficiently recognize complex distraction related activities, and (ii) a comprehensive comparative evaluation of the proposed framework with well-known complex activity recognition techniques in the literature with the help of data collected from human subject pedestrians and prototype implementations on commercially-available mobile and wearable devices

Utilising Visual Attention Cues for Vehicle Detection and Tracking

Advanced Driver-Assistance Systems (ADAS) have been attracting attention from many researchers. Vision-based sensors are the closest way to emulate human driver visual behavior while driving. In this paper, we explore possible ways to use visual attention (saliency) for object detection and tracking. We investigate: 1) How a visual attention map such as a \emph{subjectness} attention or saliency map and an \emph{objectness} attention map can facilitate region proposal generation in a 2-stage object detector; 2) How a visual attention map can be used for tracking multiple objects. We propose a neural network that can simultaneously detect objects as and generate objectness and subjectness maps to save computational power. We further exploit the visual attention map during tracking using a sequential Monte Carlo probability hypothesis density (PHD) filter. The experiments are conducted on KITTI and DETRAC datasets. The use of visual attention and hierarchical features has shown a considerable improvement of $\approx$8\% in object detection which effectively increased tracking performance by $\approx$4\% on KITTI dataset.Comment: Accepted in ICPR202

The Visual Social Distancing Problem

One of the main and most effective measures to contain the recent viral outbreak is the maintenance of the so-called Social Distancing (SD). To comply with this constraint, workplaces, public institutions, transports and schools will likely adopt restrictions over the minimum inter-personal distance between people. Given this actual scenario, it is crucial to massively measure the compliance to such physical constraint in our life, in order to figure out the reasons of the possible breaks of such distance limitations, and understand if this implies a possible threat given the scene context. All of this, complying with privacy policies and making the measurement acceptable. To this end, we introduce the Visual Social Distancing (VSD) problem, defined as the automatic estimation of the inter-personal distance from an image, and the characterization of the related people aggregations. VSD is pivotal for a non-invasive analysis to whether people comply with the SD restriction, and to provide statistics about the level of safety of specific areas whenever this constraint is violated. We then discuss how VSD relates with previous literature in Social Signal Processing and indicate which existing Computer Vision methods can be used to manage such problem. We conclude with future challenges related to the effectiveness of VSD systems, ethical implications and future application scenarios.Comment: 9 pages, 5 figures. All the authors equally contributed to this manuscript and they are listed by alphabetical order. Under submissio

FOLT: Fast Multiple Object Tracking from UAV-captured Videos Based on Optical Flow

Multiple object tracking (MOT) has been successfully investigated in computer vision. However, MOT for the videos captured by unmanned aerial vehicles (UAV) is still challenging due to small object size, blurred object appearance, and very large and/or irregular motion in both ground objects and UAV platforms. In this paper, we propose FOLT to mitigate these problems and reach fast and accurate MOT in UAV view. Aiming at speed-accuracy trade-off, FOLT adopts a modern detector and light-weight optical flow extractor to extract object detection features and motion features at a minimum cost. Given the extracted flow, the flow-guided feature augmentation is designed to augment the object detection feature based on its optical flow, which improves the detection of small objects. Then the flow-guided motion prediction is also proposed to predict the object's position in the next frame, which improves the tracking performance of objects with very large displacements between adjacent frames. Finally, the tracker matches the detected objects and predicted objects using a spatially matching scheme to generate tracks for every object. Experiments on Visdrone and UAVDT datasets show that our proposed model can successfully track small objects with large and irregular motion and outperform existing state-of-the-art methods in UAV-MOT tasks.Comment: Accepted by ACM Multi-Media 202

Context Detection, Categorization and Connectivity for Advanced Adaptive Integrated Navigation

Context is the environment that a navigation system operates in and the behaviour of its host vehicle or user. The type and quality of signals and environmental features available for positioning varies with the environment. For example, GNSS provides high-quality positioning in open environments, low-quality positioning in dense urban environments and no solution at all deep indoors. The behaviour of the host vehicle (or pedestrian) is also important. For example, pedestrian, car and train navigation all require different map-matching techniques, different motion constraints to limit inertial navigation error growth, and different dynamic models in a navigation filter [1]. A navigation system design should therefore be matched to its context. However, the context can change, particularly for devices, such as smartphones, which move between indoor and outdoor environments and can be stationary, on a pedestrian, or in a vehicle. For best performance, a navigation system should therefore be able to detect its operating context and adapt accordingly; this is context-adaptive positioning [1]. Previous work on context-adaptive navigation and positioning has focused on individual subsystems. For example, there has been substantial research into determining the motion type and sensor location for pedestrian dead reckoning using step detection [2-4]. Researchers have also begun to investigate context-adaptive (or cognitive) GNSS [5-7]. However, this paper considers context adaptation across an integrated navigation system as a whole. The paper addresses three aspects of context-adaptive integrated navigation: context detection, context categorization and context connectivity. It presents experimental results showing how GNSS C/N0 measurements, frequency-domain MEMS inertial sensor measurements and Wi-Fi signal availability could be used to detect both the environmental and behavioural contexts. It then looks at how context information could be shared across the different components of an integrated navigation system. Finally, the concept of context connectivity is introduced to improve the reliability of context detection. GNSS C/N0 measurement distributions, obtained using a smartphone, and Wi-Fi reception data collected over a range of indoor, urban and open environments will be compared to identify suitable features from which the environmental context may be derived. In an open environment, strong GNSS signals will be received from all directions. In an urban environment, fewer strong signals will be received and only from certain directions. Inside a building, nearly all GNSS signals will be much weaker than outside. Wi-Fi signals essentially vary with the environment in the opposite way to GNSS. Indoors, more access points (APs) can be received at higher signal strengths and there is greater variation in RSS. In urban environments, large numbers of APs can still be received, but at lower signal strengths [6]. Finally, in open environments, few APs, if any, will be received. Behavioural context is studied using an IMU. Although an Xsens MEMS IMU is used in this study, smartphone inertial sensors are also suitable. Pedestrian, car and train data has been collected under a range of different motion types and will be compared to identify context-dependent features. Early indications are that, as well as detecting motion, it is also possible to distinguish nominally-stationary IMUs that are placed in a car, on a person or on a table from the frequency spectra of the sensor measurements. The exchange of context information between subsystems in an integrated navigation system requires agreement on the definitions of those contexts. As different subsystems are often supplied by different organisations, it is desirable to standardize the context definitions across the whole navigation and positioning community. This paper therefore proposes a framework upon which a “context dictionary” could be constructed. Environmental and behavioural contexts are categorized separately and a hierarchy of attributes is proposed to enable some subsystems to work with highly specific context categories and others to work with broader categories. Finally, the concept of context connectivity is introduced. This is analogous to the road link connectivity used in map matching [8]. As context detection involves the matching of measurement data to stored context profiles, there will always be occurrences of false or ambiguous context identification. However, these may be minimized by using the fact that it is only practical to transition directly between certain pairs of contexts. For example, it is not normally possible to move directly from an airborne to an indoor environment as an aircraft must land first. Thus, the air and land contexts are connected, as are the land and indoor contexts, but the air and indoor contexts are not. Thus, by only permitting contexts that are connected to the previous context, false and ambiguous context detection is reduced. Robustness may be further enhanced by considering location-dependent connectivity. For example, people normally board and leave trains at stations and fixed-wing aircraft typically require an airstrip to take off and land. / References [1] Groves, P. D., Principles of GNSS, inertial, and multi-sensor integrated navigation systems, Second Edition, Artech House, 2013. [2] Park, C. G., et al., “Adaptive Step Length Estimation with Awareness of Sensor Equipped Location for PNS,” Proc. ION GNSS 2007. [3] Frank, K., et al., “Reliable Real-Time Recognition of Motion Related Human Activities Using MEMS Inertial Sensors,” Proc. ION GNSS 2010. [4] Pei, L., et al., “Using Motion-Awareness for the 3D Indoor Personal Navigation on a Smartphone,” Proc. ION GNSS 2011. [5] Lin, T., C. O’Driscoll, and G. Lachapelle, “Development of a Context-Aware Vector-Based High-Sensitivity GNSS Software Receiver,” Proc. ION ITM 2011. [6] Shafiee, M., K., O’Keefe, and G. Lachapelle, “Context-aware Adaptive Extended Kalman Filtering Using Wi-Fi Signals for GPS Navigation,” Proc. ION GNSS 2011. [7] Shivaramaiah, N. C., and A. G. Dempster, “Cognitive GNSS Receiver Design: Concept and Challenges,” Proc. ION GNSS 2011. [8] Quddus, M. A., High Integrity Map Matching Algorithms for Advanced Transport Telematics Applications, PhD Thesis, Imperial College London, 2006