5 research outputs found
Exploring Natural Language Processing Methods for Interactive Behaviour Modelling
Analysing and modelling interactive behaviour is an important topic in
human-computer interaction (HCI) and a key requirement for the development of
intelligent interactive systems. Interactive behaviour has a sequential
(actions happen one after another) and hierarchical (a sequence of actions
forms an activity driven by interaction goals) structure, which may be similar
to the structure of natural language. Designed based on such a structure,
natural language processing (NLP) methods have achieved groundbreaking success
in various downstream tasks. However, few works linked interactive behaviour
with natural language. In this paper, we explore the similarity between
interactive behaviour and natural language by applying an NLP method, byte pair
encoding (BPE), to encode mouse and keyboard behaviour. We then analyse the
vocabulary, i.e., the set of action sequences, learnt by BPE, as well as use
the vocabulary to encode the input behaviour for interactive task recognition.
An existing dataset collected in constrained lab settings and our novel
out-of-the-lab dataset were used for evaluation. Results show that this natural
language-inspired approach not only learns action sequences that reflect
specific interaction goals, but also achieves higher F1 scores on task
recognition than other methods. Our work reveals the similarity between
interactive behaviour and natural language, and presents the potential of
applying the new pack of methods that leverage insights from NLP to model
interactive behaviour in HCI
On the Generality of Codebook Approach for Sensor-Based Human Activity Recognition
With the recent spread of mobile devices equipped with different sensors, it is possible to continuously recognise and monitor activities in daily life. This sensor-based human activity recognition is formulated as sequence classification to categorise sequences of sensor values into appropriate activity classes. One crucial problem is how to model features that can precisely represent characteristics of each sequence and lead to accurate recognition. It is laborious and/or difficult to hand-craft such features based on prior knowledge and manual investigation about sensor data. To overcome this, we focus on a feature learning approach that extracts useful features from a large amount of data. In particular, we adopt a simple but effective one, called codebook approach, which groups numerous subsequences collected from sequences into clusters. Each cluster centre is called a codeword and represents a statistically distinctive subsequence. Then, a sequence is encoded as a feature expressing the distribution of codewords. The extensive experiments on different recognition tasks for physical, mental and eye-based activities validate the effectiveness, generality and usability of the codebook approach