Automatic recognition of multiparty human interactions using dynamic Bayesian networks

Relating statistical machine learning approaches to the automatic analysis of multiparty communicative events, such as meetings, is an ambitious research area. We have investigated automatic meeting segmentation both in terms of “Meeting Actions” and “Dialogue Acts”. Dialogue acts model the discourse structure at a fine grained level highlighting individual speaker intentions. Group meeting actions describe the same process at a coarse level, highlighting interactions between different meeting participants and showing overall group intentions. A framework based on probabilistic graphical models such as dynamic Bayesian networks (DBNs) has been investigated for both tasks. Our first set of experiments is concerned with the segmentation and structuring of meetings (recorded using multiple cameras and microphones) into sequences of group meeting actions such as monologue, discussion and presentation. We outline four families of multimodal features based on speaker turns, lexical transcription, prosody, and visual motion that are extracted from the raw audio and video recordings. We relate these lowlevel multimodal features to complex group behaviours proposing a multistreammodelling framework based on dynamic Bayesian networks. Later experiments are concerned with the automatic recognition of Dialogue Acts (DAs) in multiparty conversational speech. We present a joint generative approach based on a switching DBN for DA recognition in which segmentation and classification of DAs are carried out in parallel. This approach models a set of features, related to lexical content and prosody, and incorporates a weighted interpolated factored language model. In conjunction with this joint generative model, we have also investigated the use of a discriminative approach, based on conditional random fields, to perform a reclassification of the segmented DAs. The DBN based approach yielded significant improvements when applied both to the meeting action and the dialogue act recognition task. On both tasks, the DBN framework provided an effective factorisation of the state-space and a flexible infrastructure able to integrate a heterogeneous set of resources such as continuous and discrete multimodal features, and statistical language models. Although our experiments have been principally targeted on multiparty meetings; features, models, and methodologies developed in this thesis can be employed for a wide range of applications. Moreover both group meeting actions and DAs offer valuable insights about the current conversational context providing valuable cues and features for several related research areas such as speaker addressing and focus of attention modelling, automatic speech recognition and understanding, topic and decision detection

Passphrase and keystroke dynamics authentication: security and usability

It was found that employees spend a total 2.25 days within a 60 day period on password related activities. Another study found that over 85 days an average user will create 25 accounts with an average of 6.5 unique passwords. These numbers are expected to increase over time as more systems become available. In addition, the use of 6.5 unique passwords highlight that passwords are being reused which creates security concerns as multiple systems will be accessible by an unauthorised party if one of these passwords is leaked. Current user authentication solutions either increase security or usability. When security increases, usability decreases, or vice versa. To add to this, stringent security protocols encourage unsecure behaviours by the user such as writing the password down on a piece of paper to remember it. It was found that passphrases require less cognitive effort than passwords and because passphrases are stronger than passwords, they don’t need to be changed as frequently as passwords. This study aimed to assess a two-tier user authentication solution that increases security and usability. The proposed solution uses passphrases in conjunction with keystroke dynamics to address this research problem. The design science research approach was used to guide this study. The study’s theoretical foundation includes three theories. The Shannon entropy formula was used to calculate the strength of passwords, passphrases and keystroke dynamics. The chunking theory assisted in assessing password and passphrase memorisation issues and the keystroke-level model was used to assess password and passphrase typing issues. Two primary data collection methods were used to evaluate the findings and to ensure that gaps in the research were filled. A login assessment experiment collected data on user authentication and user-system interaction for passwords and passphrases. Plus, an expert review was conducted to verify findings and assess the research artefact in the form of a model. The model can be used to assist with the implementation of a two-tier user authentication solution which involves passphrases and keystroke dynamics. There are a number of components that need to be considered to realise the benefits of this solution and ensure successful implementation

Enhancing factoid question answering using frame semantic-based approaches

FrameNet is used to enhance the performance of semantic QA systems. FrameNet is a linguistic resource that encapsulates Frame Semantics and provides scenario-based generalizations over lexical items that share similar semantic backgrounds.Doctor of Philosoph

Holistic interpretation of visual data based on topology:semantic segmentation of architectural facades

The work presented in this dissertation is a step towards effectively incorporating contextual knowledge in the task of semantic segmentation. To date, the use of context has been confined to the genre of the scene with a few exceptions in the field. Research has been directed towards enhancing appearance descriptors. While this is unarguably important, recent studies show that computer vision has reached a near-human level of performance in relying on these descriptors when objects have stable distinctive surface properties and in proper imaging conditions. When these conditions are not met, humans exploit their knowledge about the intrinsic geometric layout of the scene to make local decisions. Computer vision lags behind when it comes to this asset. For this reason, we aim to bridge the gap by presenting algorithms for semantic segmentation of building facades making use of scene topological aspects. We provide a classification scheme to carry out segmentation and recognition simultaneously.The algorithm is able to solve a single optimization function and yield a semantic interpretation of facades, relying on the modeling power of probabilistic graphs and efficient discrete combinatorial optimization tools. We tackle the same problem of semantic facade segmentation with the neural network approach.We attain accuracy figures that are on-par with the state-of-the-art in a fully automated pipeline.Starting from pixelwise classifications obtained via Convolutional Neural Networks (CNN). These are then structurally validated through a cascade of Restricted Boltzmann Machines (RBM) and Multi-Layer Perceptron (MLP) that regenerates the most likely layout. In the domain of architectural modeling, there is geometric multi-model fitting. We introduce a novel guided sampling algorithm based on Minimum Spanning Trees (MST), which surpasses other propagation techniques in terms of robustness to noise. We make a number of additional contributions such as measure of model deviation which captures variations among fitted models

Developing a New Approach to Road Planning in Thailand: Application of Link & Place to a Whole Nation

National and local road planning in Thailand is based on a framework that has remained largely unchanged for decades, and originates from a time period when the focus was on designing for a general growth in private car traffic and freight traffic. Since then, there has been a growing focus on multi-modality, on sustainability and on recognising the role of roads in a wider urban and rural place-making and environmental context. The thesis critically assesses the continued relevance of the current national road classification system and reviews a wide range of alternative classifications that have been proposed or adopted in different countries. It concludes that an alternative road classification system, Link & Place, has the potential to provide the basis of a new planning framework in Thailand and sets out to explore the implications of adopting this approach, in road ownership, funding and scheme prioritisation, both conceptually and at a practical level. Part A reviews international approaches to road classification and to road planning, and then examines both current national and sub-national planning frameworks in Thailand, based on literature/document reviews and interviews with key professionals in different government agencies and at various spatial levels. It concludes with an assessment of the strengths and limitations of the current planning procedures, and identifies key research gaps that are addressed in the remainder of the thesis. Part B considers how Link & Place could address the problems identified in part A and examines implications for road ownership and funding arrangements, and for performance measurement, problem identification and scheme prioritisation. The approach is tested empirically in a case study area, and further professional interviews are conducted to obtain local and national views on the applicability of this proposed approach. The final chapter critically assess what has been achieved and makes research and policy recommendations. The findings show that the road classification method based on Link & Place could be applied in a national context. Link & Place also offers a coordinated approach that brings administrative arrangements, funding, and performance measurement together, which could help build institutional capacity in sustainable road planning that is generally much needed in the global south

Proceedings of the Second International Mobile Satellite Conference (IMSC 1990)

Presented here are the proceedings of the Second International Mobile Satellite Conference (IMSC), held June 17-20, 1990 in Ottawa, Canada. Topics covered include future mobile satellite communications concepts, aeronautical applications, modulation and coding, propagation and experimental systems, mobile terminal equipment, network architecture and control, regulatory and policy considerations, vehicle antennas, and speech compression