Multimodal Human Group Behavior Analysis

Human behaviors in a group setting involve a complex mixture of multiple modalities: audio, visual, linguistic, and human interactions. With the rapid progress of AI, automatic prediction and understanding of these behaviors is no longer a dream. In a negotiation, discovering human relationships and identifying the dominant person can be useful for decision making. In security settings, detecting nervous behaviors can help law enforcement agents spot suspicious people. In adversarial settings such as national elections and court defense, identifying persuasive speakers is a critical task. It is beneficial to build accurate machine learning (ML) models to predict such human group behaviors. There are two elements for successful prediction of group behaviors. The first is to design domain-specific features for each modality. Social and Psychological studies have uncovered various factors including both individual cues and group interactions, which inspire us to extract relevant features computationally. In particular, the group interaction modality plays an important role, since human behaviors influence each other through interactions in a group. Second, effective multimodal ML models are needed to align and integrate the different modalities for accurate predictions. However, most previous work ignored the group interaction modality. Moreover, they only adopt early fusion or late fusion to combine different modalities, which is not optimal. This thesis presents methods to train models taking multimodal inputs in group interaction videos, and to predict human group behaviors. First, we develop an ML algorithm to automatically predict human interactions from videos, which is the basis to extract interaction features and model group behaviors. Second, we propose a multimodal method to identify dominant people in videos from multiple modalities. Third, we study the nervousness in human behavior by a developing hybrid method: group interaction feature engineering combined with individual facial embedding learning. Last, we introduce a multimodal fusion framework that enables us to predict how persuasive speakers are. Overall, we develop one algorithm to extract group interactions and build three multimodal models to identify three kinds of human behavior in videos: dominance, nervousness and persuasion. The experiments demonstrate the efficacy of the methods and analyze the modality-wise contributions

A Framework for Data-Driven Solutions with COVID-19 Illustrations

Data–driven solutions have long been keenly sought after as tools for driving the world’s fast changing business environment, with business leaders seeking to enhance decision making processes within their organisations. In the current era of Big Data, applications of data tools in addressing global, regional and national challenges have steadily grown in almost all fields across the globe. However, working in silos has continued to impede research progress, creating knowledge gaps and challenges across geographical borders, legislations, sectors and fields. There are many examples of the challenges the world faces in tackling global issues, including the complex interactions of the 17 Sustainable Development Goals (SDG) and the spatio–temporal variations of the impact of the on-going COVID–19 pandemic. Both challenges can be seen as non–orthogonal, strongly correlated and requiring an interdisciplinary approach to address. We present a generic framework for filling such gaps, based on two data-driven algorithms that combine data, machine learning and interdisciplinarity to bridge societal knowledge gaps. The novelty of the algorithms derives from their robust built–in mechanics for handling data randomness. Animation applications on structured COVID–19 related data obtained from the European Centre for Disease Prevention and Control (ECDC) and the UK Office of National Statistics exhibit great potentials for decision-support systems. Predictive findings are based on unstructured data–a large COVID–19 X–Ray data, 3181 image files, obtained from GitHub and Kaggle. Our results exhibit consistent performance across samples, resonating with cross-disciplinary discussions on novel paths for data-driven interdisciplinary research

Recent Progress in Urbanisation Dynamics Research

This book is dedicated to urbanization, which is observed every day, as well as the methods and techniques of monitoring and analyzing this phenomenon. In the 21st century, urbanization has gained momentum, and the awareness of the significance and influence of this phenomenon on our lives make us take a closer look at it not only with curiosity, but also great attention. There are numerous reasons for this, among which the economy is of special significance, but it also has many results, namely, economic, social, and environmental. First of all, it is a spatial phenomenon, as all of the aspects can be placed in space. We would therefore like to draw special attention to the results of urbanization seen on the Earth's surface and in the surrounding space. The urbanization–land relation seems obvious, but is also interesting and multi-layered. The development of science and technology provides a lot of new tools for observing urbanization, as well as the analyses and inference of the phenomenon in space. This book is devoted to in-depth analysis of past, present and future urbanization processes all over the world. We present the latest trends of research that use experience in the widely understood geography of the area. This book is focused on multidisciplinary phenomenon, i.e., urbanization, with the use of the satellite and photogrammetric observation technologies and GIS analyses

A study of the effects of ageing on the characteristics of handwriting and signatures

The work presented in this thesis is focused on the understanding of factors that are unique to the elderly and their use of biometric systems. In particular, an investigation is carried out with a focus on the handwritten signature as the biometric modality of choice. This followed on from an in-depth analysis of various biometric modalities such as voice, fingerprint and face. This analysis aimed at investigating the inclusivity of and the policy guiding the use of biometrics by the elderly. Knowledge gained from extracted features of the handwritten signatures of the elderly shed more light on and exposed the uniqueness of some of these features in their ability to separate the elderly from the young. Consideration is also given to a comparative analysis of another handwriting task, that of copying text both in cursive and block capitals. It was discovered that there are features that are unique to each task. Insight into the human perceptual capability in inspecting signatures, in assessing complexity and in judging imitations was gained by analysing responses to practical scenarios that applied human perceptual judgement. Features extracted from a newly created database containing handwritten signatures donated by elderly subjects allowed the possibility of analysing the intra-class variations that exist within the elderly population