1 research outputs found
Detecting human comprehension from nonverbal behaviour using artificial neural networks
Every day, communication between humans is abundant with an array of
nonverbal behaviours. Nonverbal behaviours are signals emitted without using words
such as facial expressions, eye gaze and body movement. Nonverbal behaviours have
been used to identify a person’s emotional state in previous research. With nonverbal
behaviour being continuously available and almost unconscious, it provides a
potentially rich source of knowledge once decoded. Humans are weak decoders of
nonverbal behaviour due to being error prone, susceptible to fatigue and poor at
simultaneously monitoring numerous nonverbal behaviours.
Human comprehension is primarily assessed from written and spoken language.
Existing comprehension assessments tools are inhibited by inconsistencies and are
often time-consuming with feedback delay. Therefore, there is a niche for attempting
to detect human comprehension from nonverbal behaviour using artificially intelligent
computational models such as Artificial Neural Networks (ANN), which are inspired by
the structure and behaviour of biological neural networks such as those found within
the human brain.
This Thesis presents a novel adaptable system known as FATHOM, which has been
developed to detect human comprehension and non-comprehension from monitoring
multiple nonverbal behaviours using ANNs. FATHOM’s Comprehension Classifier ANN
was trained and validated on human comprehension detection using the errorbackpropagation
learning algorithm and cross-validation in a series of experiments
with nonverbal datasets extracted from two independent comprehension studies
where each participant was digitally video recorded: (1) during a mock informed
consent field study and (2) in a learning environment. The Comprehension Classifier
ANN repeatedly achieved averaged testing classification accuracies (CA) above 84% in
the first phase of the mock informed consent field study. In the learning environment
study, the optimised Comprehension Classifier ANN achieved a 91.385% averaged
testing CA. Overall, the findings revealed that human comprehension and noncomprehension
patterns can be automatically detected from multiple nonverbal
behaviours using ANNs