Exploring human mobility patterns based on geotagged Flickr photos

Predicting human mobility behaviour has long been a topic of scientific interest. Such studies generally rely on tracking human movements through a range of data collection methodologies such as using GPS trackers, cellular network data etc. Some of this data may be confidential or hard to acquire. This thesis explores if existing publicly available data on online photo sharing platforms can be used to determine human mobility patterns with reasonable accuracy. We choose the Flickr website as the data collection medium as it has an extensive user base actively sharing photos many of which, have geo tags embedded in them which are preserved by Flickr. Our analysis reveals that while the data from Flickr is sparse and discontinuous making it unsuitable for reliable mobility prediction, typical human mobility trends based on time of day, day of week and month of the year can still be extracted. Such interesting patterns could be potentially used in traffic engineering domains or for user profiling purposes. More specifically, we describe how to obtain a subset of frequent active users and their information from Flickr, and the sliding window mechanism to filter the active periods of the users. Later we explain the various statistical methods applied on the filtered subset of data to identify the categories in which users could be classified, mainly short distance travellers and long distance travellers. The short distance travellers are considered for mobility trends prediction

Toward Complex 3D Movement Detection to Analyze Human Behavior via Radio-Frequency Signals

A doctoral dissertation completed for the degree of Doctor of Science (Technology) to be defended, with the permission of the Aalto University School of Electrical Engineering, at a public examination held with remote technology on 15 September 2020 at 6:00pm. Zoom link: https://aalto.zoom.us/j/6549167529A driver's attention, parallel actions, and emotions directly influence driving behavior. Any secondary task, be it cognitive, visual, or manual, that diverts driver focus from the primary task of driving is a source of distraction. Longer response time, inability to scan the road, and missing visual cues can all lead to car crashes with serious consequences. Current research focuses on detecting distraction by means of vehicle-mounted video cameras or wearable sensors for tracking eye movements and head rotation. Facial expressions, speech, and physiological signals are also among the widely used indicators for detecting distraction. These approaches are accurate, fast, and reliable but come with a high installation cost, requirements related to lighting conditions, privacy intrusions, and energy consumption. Over the past decade, the use of radio signals has been investigated as a possible solution for the aforementioned limitations of today's technologies. Changes in radio-signal patterns caused by movements of the human body can be analyzed and thereby used in detecting humans' gestures and activities. Human behavior and emotions, in particular, are less explored in this regard and are addressed mostly with reference to physiological signals. The thesis exploited multiple wireless technologies (1.8~GHz, WiFi, and millimeter wave) and combinations thereof to detect complex 3D movements of a driver in a car. Upper-body movements are vital indicators of a driver's behavior in a car, and the information from these movements could be used to generate appropriate feedback, such as warnings or provision of directives for actions that would avoid jeopardizing safety. Existing wireless-system-based solutions focus primarily on either large or small movements, or they address well-defined activities. They do not consider discriminating large movements from small ones, let alone their directions, within a single system. These limitations underscore the requirement to address complex natural-behavior situations precisely such as that in a car, which demands not only isolating particular movements but also classifying and predicting them. The research to reach the attendant goals exploited physical properties of RF signals, several hardware-software combinations, and building of algorithms to process and detect body movements -- from the simple to the complex. Additionally, distinctive feature sets were addressed for machine-learning techniques to find patterns in data and predict states accordingly. The systems were evaluated by performing extensive real-world studies

RFexpress! - RF emotion recognition in the wild

We present RFexpress! the first-ever system to recognize emotion from body movements and gestures via Device-Free Activity Recognition (DFAR). We focus on the distinction between neutral and agitated states in realistic environments. In particular, the system is able to detect risky driving behaviour in a vehicular setting as well as spotting angry conversations in an indoor environment. In case studies with 8 and 5 subjects the system could achieve recognition accuracies of 82.9% and 64%. We study the effectiveness of DFAR emotion and activity recognition systems in real environments such as cafes, malls, outdoor and office spaces. We measure radio characteristics in these environments at different days and times and analyse the impact of variations in the Signal to Noise Ratio (SNR) on the accuracy of DFAR emotion and activity recognition. In a case study with 5 subjects, we then find critical SNR values under which activity and emotion recognition results are no longer reliable.Peer reviewe

WiBot! In-vehicle behaviour and gesture recognition using wireless network edge

Recent advancements in vehicular technology have meant that integrated wireless devices such as Wi-Fi access points or bluetooth are deployed in vehicles at an increasingly dense scale. These vehicular network edge devices, while enabling in car wireless connectivity and infotainment services, can also be exploited as sensors to improve environmental and behavioural awareness that in turn can provide better and more personalised driver feedback and improve road safety. We present WiBot! a network-edge based behaviour recognition and gesture based personal assistant system for cars. WiBot leverages the vehicular network edge to detect distracted behaviour based on unusual head turns and arm movements during driving situations by monitoring radio frequency fluctuation patterns in real-time. Additionally, WiBot can recognise known gestures from natural arm movements while driving and use such gestures for passenger-car interaction. A key element of WiBot design is its impulsive windowing approach that allows start and end of gestures to be accurately identified in a continuous stream of data. We validate the system in a realistic driving environment by conducting a non-choreographed continuous recognition study with 40 participants at BMW Group Research, New Technologies and Innovation centre. By combining impulsive windowing with a unique selection of features from peaks and subcarrier analysis of RF CSI phase information, the system is able to achieve 94.5% accuracy for head-vs. arm movement separation. We can further confidently differentiate relevant gestures from random arm and head movements, head turns and idle movement with 90.5% accuracy.Peer reviewe

A cloud-IoT platform for passive radio sensing

We propose a platform for the integration of passive radio sensing and vision technologies into a cloud-IoT framework that performs real-time channel quality information (CQI) time series processing and analytics. Radio sensing and vision technologies allow to passively detect and track objects or persons by using radio waves as probe signals that encode a 2D/3D view of the environment they propagate through. View reconstruction from the received radio signals, or CQI, is based on real-time data processing tools, that combine multiple radio measurements from possibly heterogeneous IoT networks. The proposed platform is designed to efficiently store and analyze CQI time series of different types and provides formal semantics for CQI data manipulation (ontology models). Post-processed data can be then accessible to third parties via JSON-REST calls. Finally, the proposed system supports the reconfiguration of CQI data collection based on the respective application. The performance of the proposed tools are evaluated through two experimental case studies that focus on assisted living applications in a smart-space environment and on driver behavior recognition for in-car control services. Both studies adopt and compare different CQI manipulation models and radio devices as supported by current and future (5G) standards.Peer reviewe

3D Head Motion Detection Using Millimeter-Wave Doppler Radar

In advanced driver assistance systems to conditional automation systems, monitoring of driver state is vital for predicting the driver's capacity to supervise or maneuver the vehicle in cases of unexpected road events and to facilitate better in-car services. The paper presents a technique that exploits millimeter-wave Doppler radar for 3D head tracking. Identifying the bistatic and monostatic geometry for antennas to detect rotational vs. translational movements, the authors propose the biscattering angle for computing a distinctive feature set to isolate dynamic movements via class memberships. Through data reduction and joint time-frequency analysis, movement boundaries are marked for creation of a simplified, uncorrelated, and highly separable feature set. The authors report movement-prediction accuracy of 92%. This non-invasive and simplified head tracking has the potential to enhance monitoring of driver state in autonomous vehicles and aid intelligent car assistants in guaranteeing seamless and safe journeys.Peer reviewe

Wireless Multifrequency Feature Set to Simplify Human 3D Pose Estimation

We present a multi-frequency feature set to detect driver's 3D head and torso movements from fluctuations in the Radio Frequency (RF) channel due to body movements. Current features used for movement detection are based on time-of-flight, received signal strength and channel state information, and come with the limitations of coarse tracking, sensitivity towards multipath effects and handling corrupted phase data, respectively. There is no standalone feature set which accurately detects small and large movements and determines the direction in 3D space. We resolve this problem by using two radio signals at widely separated frequencies in a monostatic configuration. By combining information about displacement, velocity, and direction of movements derived from the Doppler Effect at each frequency, we expand the number of existing features. We separate Pitch, Roll, and Yaw movements of head from torso and arm. The extracted feature set is used to train a K-Nearest Neighbor classification algorithm which could provide behavioral awareness to cars while being less invasive as compared to camera-based systems. The training results on a data from 4 participants reveal that at 1.8GHz, the classification accuracy is 77.4%, at 30GHz it is 87.4%, and multi-frequency feature set improves the accuracy to 92%.Peer reviewe