Probabilistic Analysis of Temporal and Sequential Aspects of Activities of Daily Living for Abnormal Behaviour Detection

This paper presents a probabilistic approach for the identification of abnormal behaviour in Activities of Daily Living (ADLs) from dense sensor data collected from 30 participants. The ADLs considered are related to preparing and drinking (i) tea, and (ii) coffee. Abnormal behaviour identified in the context of these activities can be an indicator of a progressive health problem or the occurrence of a hazardous incident. The approach presented considers the temporal and sequential aspects of the actions that are part of each ADL and that vary between participants. The average and standard deviation for the duration and number of steps of each activity are calculated to define the average time and steps and a range within which a behaviour could be considered as normal for each stage and activity. The Cumulative Distribution Function (CDF) is used to obtain the probabilities of abnormal behaviours related to the early and late completion of activities and stages within an activity in terms of time and steps. Analysis shows that CDF can provide precise and reliable results regarding the presence of abnormal behaviour in stages and activities that last over a minute or consist of many steps. Finally, this approach could be used to train machine learning algorithms for abnormal behaviour detection.status: publishe

Breaking the habit: measuring and predicting departures from routine in individual human mobility

Researchers studying daily life mobility patterns have recently shown that humans are typically highly predictable in their movements. However, no existing work has examined the boundaries of this predictability, where human behaviour transitions temporarily from routine patterns to highly unpredictable states. To address this shortcoming, we tackle two interrelated challenges. First, we develop a novel information-theoretic metric, called instantaneous entropy, to analyse an individual’s mobility patterns and identify temporary departures from routine. Second, to predict such departures in the future, we propose the first Bayesian framework that explicitly models breaks from routine, showing that it outperforms current state-of-the-art predictor

A context-sensitive conceptual framework for activity modeling

Human motion trajectories, however captured, provide a rich spatiotemporal data source for human activity recognition, and the rich literature in motion trajectory analysis provides the tools to bridge the gap between this data and its semantic interpretation. But activity is an ambiguous term across research communities. For example, in urban transport research activities are generally characterized around certain locations assuming the opportunities and resources are present in that location, and traveling happens between these locations for activity participation, i.e., travel is not an activity, rather a mean to overcome spatial constraints. In contrast, in human-computer interaction (HCI) research and in computer vision research activities taking place along the way, such as reading on the bus, are significant for contextualized service provision. Similarly activities at coarser spatial and temporal granularity, e.g., holidaying in a country, could be recognized in some context or domain. Thus the context prevalent in the literature does not provide a precise and consistent definition of activity, in particular in differentiation to travel when it comes to motion trajectory analysis. Hence in this paper, a thorough literature review studies activity from different perspectives, and develop a common framework to model and reason human behavior flexibly across contexts. This spatio-temporal framework is conceptualized with a focus on modeling activities hierarchically. Three case studies will illustrate how the semantics of the term activity changes based on scale and context. They provide evidence that the framework holds over different domains. In turn, the framework will help developing various applications and services that are aware of the broad spectrum of the term activity across contexts

Cognitive visual tracking and camera control

Cognitive visual tracking is the process of observing and understanding the behaviour of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision

Privacy-preserving human mobility and activity modelling

The exponential proliferation of digital trends and worldwide responses to the COVID-19 pandemic thrust the world into digitalization and interconnectedness, pushing increasingly new technologies/devices/applications into the market. More and more intimate data of users are collected for positive analysis purposes of improving living well-being but shared with/without the user's consent, emphasizing the importance of making human mobility and activity models inclusive, private, and fair. In this thesis, I develop and implement advanced methods/algorithms to model human mobility and activity in terms of temporal-context dynamics, multi-occupancy impacts, privacy protection, and fair analysis. The following research questions have been thoroughly investigated: i) whether the temporal information integrated into the deep learning networks can improve the prediction accuracy in both predicting the next activity and its timing; ii) how is the trade-off between cost and performance when optimizing the sensor network for multiple-occupancy smart homes; iii) whether the malicious purposes such as user re-identification in human mobility modelling could be mitigated by adversarial learning; iv) whether the fairness implications of mobility models and whether privacy-preserving techniques perform equally for different groups of users. To answer these research questions, I develop different architectures to model human activity and mobility. I first clarify the temporal-context dynamics in human activity modelling and achieve better prediction accuracy by appropriately using the temporal information. I then design a framework MoSen to simulate the interaction dynamics among residents and intelligent environments and generate an effective sensor network strategy. To relieve users' privacy concerns, I design Mo-PAE and show that the privacy of mobility traces attains decent protection at the marginal utility cost. Last but not least, I investigate the relations between fairness and privacy and conclude that while the privacy-aware model guarantees group fairness, it violates the individual fairness criteria.Open Acces

Towards Ubiquitous Semantic Metaverse: Challenges, Approaches, and Opportunities

In recent years, ubiquitous semantic Metaverse has been studied to revolutionize immersive cyber-virtual experiences for augmented reality (AR) and virtual reality (VR) users, which leverages advanced semantic understanding and representation to enable seamless, context-aware interactions within mixed-reality environments. This survey focuses on the intelligence and spatio-temporal characteristics of four fundamental system components in ubiquitous semantic Metaverse, i.e., artificial intelligence (AI), spatio-temporal data representation (STDR), semantic Internet of Things (SIoT), and semantic-enhanced digital twin (SDT). We thoroughly survey the representative techniques of the four fundamental system components that enable intelligent, personalized, and context-aware interactions with typical use cases of the ubiquitous semantic Metaverse, such as remote education, work and collaboration, entertainment and socialization, healthcare, and e-commerce marketing. Furthermore, we outline the opportunities for constructing the future ubiquitous semantic Metaverse, including scalability and interoperability, privacy and security, performance measurement and standardization, as well as ethical considerations and responsible AI. Addressing those challenges is important for creating a robust, secure, and ethically sound system environment that offers engaging immersive experiences for the users and AR/VR applications.Comment: 18 pages, 7 figures, 3 table

Latitude, longitude, and beyond:mining mobile objects' behavior

Rapid advancements in Micro-Electro-Mechanical Systems (MEMS), and wireless communications, have resulted in a surge in data generation. Mobility data is one of the various forms of data, which are ubiquitously collected by different location sensing devices. Extensive knowledge about the behavior of humans and wildlife is buried in raw mobility data. This knowledge can be used for realizing numerous viable applications ranging from wildlife movement analysis, to various location-based recommendation systems, urban planning, and disaster relief. With respect to what mentioned above, in this thesis, we mainly focus on providing data analytics for understanding the behavior and interaction of mobile entities (humans and animals). To this end, the main research question to be addressed is: How can behaviors and interactions of mobile entities be determined from mobility data acquired by (mobile) wireless sensor nodes in an accurate and efficient manner? To answer the above-mentioned question, both application requirements and technological constraints are considered in this thesis. On the one hand, applications requirements call for accurate data analytics to uncover hidden information about individual behavior and social interaction of mobile entities, and to deal with the uncertainties in mobility data. Technological constraints, on the other hand, require these data analytics to be efficient in terms of their energy consumption and to have low memory footprint, and processing complexity

Anticipatory Mobile Computing: A Survey of the State of the Art and Research Challenges

Today's mobile phones are far from mere communication devices they were ten years ago. Equipped with sophisticated sensors and advanced computing hardware, phones can be used to infer users' location, activity, social setting and more. As devices become increasingly intelligent, their capabilities evolve beyond inferring context to predicting it, and then reasoning and acting upon the predicted context. This article provides an overview of the current state of the art in mobile sensing and context prediction paving the way for full-fledged anticipatory mobile computing. We present a survey of phenomena that mobile phones can infer and predict, and offer a description of machine learning techniques used for such predictions. We then discuss proactive decision making and decision delivery via the user-device feedback loop. Finally, we discuss the challenges and opportunities of anticipatory mobile computing.Comment: 29 pages, 5 figure