Evaluating embodied conversational agents in multimodal interfaces

Based on cross-disciplinary approaches to Embodied Conversational Agents, evaluation methods for such human-computer interfaces are structured and presented. An introductory systematisation of evaluation topics from a conversational perspective is followed by an explanation of social-psychological phenomena studied in interaction with Embodied Conversational Agents, and how these can be used for evaluation purposes. Major evaluation concepts and appropriate assessment instruments – established and new ones – are presented, including questionnaires, annotations and log-files. An exemplary evaluation and guidelines provide hands-on information on planning and preparing such endeavours

On the automaticity of language processing

People speak and listen to language all the time. Given this high frequency of use, it is often suggested that at least some aspects of language processing are highly overlearned and therefore occur “automatically”. Here we critically examine this suggestion. We first sketch a framework that views automaticity as a set of interrelated features of mental processes and a matter of degree rather than a single feature that is all-or-none. We then apply this framework to language processing. To do so, we carve up the processes involved in language use according to (a) whether language processing takes place in monologue or dialogue, (b) whether the individual is comprehending or producing language, (c) whether the spoken or written modality is used, and (d) the linguistic processing level at which they occur, that is, phonology, the lexicon, syntax, or conceptual processes. This exercise suggests that while conceptual processes are relatively non-automatic (as is usually assumed), there is also considerable evidence that syntactic and lexical lower-level processes are not fully automatic. We close by discussing entrenchment as a set of mechanisms underlying automatization

Adaptive Cognitive Interaction Systems

Adaptive kognitive Interaktionssysteme beobachten und modellieren den Zustand ihres Benutzers und passen das Systemverhalten entsprechend an. Ein solches System besteht aus drei Komponenten: Dem empirischen kognitiven Modell, dem komputationalen kognitiven Modell und dem adaptiven Interaktionsmanager. Die vorliegende Arbeit enthält zahlreiche Beiträge zur Entwicklung dieser Komponenten sowie zu deren Kombination. Die Ergebnisse werden in zahlreichen Benutzerstudien validiert

BRAIN COMPUTER INTERFACE (BCI) ON ATTENTION: A SCOPING REVIEW

Technological innovations are now an integral part of healthcare. Brain-computer interface (BCI) is a novel technological intervention system that is useful in restoring function to people disabled by neurological disorders such as attention deficit hyperactivity disorder (ADHD), amyotrophic lateral sclerosis (ALS), cerebral palsy, stroke, or spinal cord injury. This paper surveys the literature concerning the effectiveness of BCI on attention in subjects under various conditions. The findings of this scoping review are that studies have been made on ADHD, ALS, ASD subjects, and subjects recovering from brain and spinal cord injuries. BCI based neurofeedback training is seen to be effective in improving attention in these subjects. Some studies have also been made on healthy subjects.BCI based neurofeedback training promises neurocognitive improvement and EEG changes in the elderly. Different cognitive assessments have been tried on healthy adults.   From this review, it is evident that hardly any research has been done on using BCI for enhancing attention in post-stroke subjects. So there arises the necessity for making a study on the effects of BCI based attention training in post-stroke subjects, as attention is the key for learning motor skills that get impaired following a stroke. Currently, many researches are underway to determine the effects of a BCI based training program for the enhancement of attention in post-stroke subjects

Measuring cognitive load and cognition: metrics for technology-enhanced learning

This critical and reflective literature review examines international research published over the last decade to summarise the different kinds of measures that have been used to explore cognitive load and critiques the strengths and limitations of those focussed on the development of direct empirical approaches. Over the last 40 years, cognitive load theory has become established as one of the most successful and influential theoretical explanations of cognitive processing during learning. Despite this success, attempts to obtain direct objective measures of the theory's central theoretical construct – cognitive load – have proved elusive. This obstacle represents the most significant outstanding challenge for successfully embedding the theoretical and experimental work on cognitive load in empirical data from authentic learning situations. Progress to date on the theoretical and practical approaches to cognitive load are discussed along with the influences of individual differences on cognitive load in order to assess the prospects for the development and application of direct empirical measures of cognitive load especially in technology-rich contexts

Evaluating Information Presentation Strategies for Spoken Dialogue Systems

Institute for Communicating and Collaborative SystemsA common task for spoken dialogue systems (SDS) is to help users select a suitable option (e.g., flight, hotel, restaurant) from the set of options available. When the number of options is small, they can simply be presented sequentially. However, as the number of options increases, the system must have strategies for helping users browse the space of available options. In this thesis, I compare two approaches to information presentation in SDS: (1) the summarize and refine (SR) approach (Polifroni et al., 2003; Polifroni, 2008) in which the summaries are generated by clustering the options based on attributes that lead to the smallest number of clusters, and (2) the user-model based summarize and refine (UMSR) approach (Demberg, 2005; Demberg and Moore, 2006) which employs a user model to cluster options based on attributes that are relevant to the user and uses coherence markers (e.g., connectives, discourse cues, adverbials) to highlight the trade-offs among the presented items. Prior work has shown that users prefer approaches to information presentation that take the user’s preferences into account (e.g., Komatani et al., 2003;Walker et al., 2004; Demberg and Moore, 2006). However, due to the complexity of building a working end-to-end SDS, these studies employed an ”overhearer” evaluation methodology, in which participants read or listened to pre-prepared dialogues, thus limiting evaluation criteria to users’ perceptions (e.g., informativeness, overview of options, and so on). In order to examine whether users prefer presentations based on UMSR when they were actively interacting with a dialogue system, and to measure the effectiveness and efficiency of the two approaches, I compared them in a Wizard-of-Oz experiment. I found that in terms of both task success and dialogue efficiency the UMSR approach was superior to the SR approach. In addition, I found that users also preferred presentations based on UMSR in the interactive mode. SDS are typically developed for situations in which the user’s hands and eyes are busy. I hypothesized that the benefits of pointing out relationships among options (i.e., trade-offs) in information presentation messages outweighs the costs of processing more complex sentences. To test this hypothesis, I performed two dual task experiments comparing the two approaches to information presentation in terms of their effect on cognitive load. Again, participants performed better with presentations based on the UMSR algorithm in terms of both dialogue efficiency and task success, and I found no detrimental effect on performance of the primary task. Finally, I hypothesized that one of the main reasons why UMSR is more efficient is because it uses coherence markers to highlight relations (e.g., trade-offs) between options and attributes. To test this hypothesis, I performed an eye-tracking experiment in which participants read presentations with and without these linguistic devices, and answered evaluation and comparison questions to measure differences in item recall. In addition, I used reading times to examine comprehension differences between the two information presentation strategies. I found that the linguistic devices used in UMSR indeed facilitated item recall, with no penalty in terms of comprehension cost. Thus, in this thesis I showed that an approach to information presentation that employs a user model and uses linguistic devices such as coherence markers to highlight trade-offs among the presented items improves information browsing. User studies demonstrated that this finding also applies to situations where users are performing another demanding task simultaneously

Multimodal information presentation for high-load human computer interaction

This dissertation addresses the question: given an application and an interaction context, how can interfaces present information to users in a way that improves the quality of interaction (e.g. a better user performance, a lower cognitive demand and a greater user satisfaction)? Information presentation is critical to the quality of interaction because it guides, constrains and even determines cognitive behavior. A good presentation is particularly desired in high-load human computer interactions, such as when users are under time pressure, stress, or are multi-tasking. Under a high mental workload, users may not have the spared cognitive capacity to cope with the unnecessary workload induced by a bad presentation. In this dissertation work, the major presentation factor of interest is modality. We have conducted theoretical studies in the cognitive psychology domain, in order to understand the role of presentation modality in different stages of human information processing. Based on the theoretical guidance, we have conducted a series of user studies investigating the effect of information presentation (modality and other factors) in several high-load task settings. The two task domains are crisis management and driving. Using crisis scenario, we investigated how to presentation information to facilitate time-limited visual search and time-limited decision making. In the driving domain, we investigated how to present highly-urgent danger warnings and how to present informative cues that help drivers manage their attention between multiple tasks. The outcomes of this dissertation work have useful implications to the design of cognitively-compatible user interfaces, and are not limited to high-load applications

Advancing Pattern Recognition Techniques for Brain-Computer Interfaces: Optimizing Discriminability, Compactness, and Robustness

In dieser Dissertation formulieren wir drei zentrale Zielkriterien zur systematischen Weiterentwicklung der Mustererkennung moderner Brain-Computer Interfaces (BCIs). Darauf aufbauend wird ein Rahmenwerk zur Mustererkennung von BCIs entwickelt, das die drei Zielkriterien durch einen neuen Optimierungsalgorithmus vereint. Darüber hinaus zeigen wir die erfolgreiche Umsetzung unseres Ansatzes für zwei innovative BCI Paradigmen, für die es bisher keine etablierte Mustererkennungsmethodik gibt

The Effectiveness of Aural Instructions with Visualisations in E-Learning Environments

Based on Mayer’s (2001) model for more effective learning by exploiting the brain’s dual sensory channels for information processing, this research investigates the effectiveness of using aural instructions together with visualisation in teaching the difficult concepts of data structures to novice computer science students. A small number of previous studies have examined the use of audio and visualisation in teaching and learning environments but none has explored the integration of both technologies in teaching data structures programming to reduce the cognitive load on learners’ working memory. A prototype learning tool, known as the Data Structure Learning (DSL) tool, was developed and used first in a short mini study that showed that, used together with visualisations of algorithms, aural instructions produced faster student response times than did textual instructions. This result suggested that the additional use of the auditory sensory channel did indeed reduce the cognitive load. The tool was then used in a second, longitudinal, study over two academic terms in which students studying the Data Structures module were offered the opportunity to use the DSL approach with either aural or textual instructions. Their use of the approach was recorded by the DSL system and feedback was invited at the end of every visualisation task. The collected data showed that the tool was used extensively by the students. A comparison of the students’ DSL use with their end-of-year assessment marks revealed that academically weaker students had tended to use the tool most. This suggests that less able students are keen to use any useful and available instrument to aid their understanding, especially of difficult concepts. Both the quantitative data provided by the automatic recording of DSL use and an end-of-study questionnaire showed appreciation by students of the help the tool had provided and enthusiasm for its future use and development. These findings were supported by qualitative data provided by student written feedback at the end of each task, by interviews at the end of the experiment and by interest from the lecturer in integrating use of the tool with the teaching of the module. A variety of suggestions are made for further work and development of the DSL tool. Further research using a control group and/or pre and post tests would be particularly useful