Mitigating User Frustration through Adaptive Feedback based on Human-Automation Etiquette Strategies

The objective of this study is to investigate the effects of feedback and user frustration in human-computer interaction (HCI) and examine how to mitigate user frustration through feedback based on human-automation etiquette strategies. User frustration in HCI indicates a negative feeling that occurs when efforts to achieve a goal are impeded. User frustration impacts not only the communication with the computer itself, but also productivity, learning, and cognitive workload. Affect-aware systems have been studied to recognize user emotions and respond in different ways. Affect-aware systems need to be adaptive systems that change their behavior depending on users’ emotions. Adaptive systems have four categories of adaptations. Previous research has focused on primarily function allocation and to a lesser extent information content and task scheduling. However, the fourth approach, changing the interaction styles is the least explored because of the interplay of human factors considerations. Three interlinked studies were conducted to investigate the consequences of user frustration and explore mitigation techniques. Study 1 showed that delayed feedback from the system led to higher user frustration, anger, cognitive workload, and physiological arousal. In addition, delayed feedback decreased task performance and system usability in a human-robot interaction (HRI) context. Study 2 evaluated a possible approach of mitigating user frustration by applying human-human etiquette strategies in a tutoring context. The results of Study 2 showed that changing etiquette strategies led to changes in performance, motivation, confidence, and satisfaction. The most effective etiquette strategies changed when users were frustrated. Based on these results, an adaptive tutoring system prototype was developed and evaluated in Study 3. By utilizing a rule set derived from Study 2, the tutor was able to use different automation etiquette strategies to target and improve motivation, confidence, satisfaction, and performance using different strategies, under different levels of user frustration. This work establishes that changing the interaction style alone of a computer tutor can affect a user’s motivation, confidence, satisfaction, and performance. Furthermore, the beneficial effect of changing etiquette strategies is greater when users are frustrated. This work provides a basis for future work to develop affect-aware adaptive systems to mitigate user frustration

Supporting Focus and Context Awareness in 3D Modeling Using Multi-Layered Displays

Although advances in computer technology over the past few decades have made it possible to create and render highly realistic 3D models these days, the process of creating these models has remained largely unchanged over the years. Modern 3D modeling software provide a range of tools to assist users with creating 3D models, but the process of creating models in virtual 3D space is nevertheless still challenging and cumbersome. This thesis, therefore, aims to investigate whether it is possible to support modelers more effectively by providing them with alternative combinations of hardware and software tools to improve their 3D modeling tasks. The first step towards achieving this goal has been to better understand the type of problems modelers face in using conventional 3D modeling software. To achieve this, a pilot study of novice 3D modelers, and a more comprehensive study of professional modelers were conducted. These studies resulted in identifying a range of focus and context awareness problems that modelers face in creating complex 3D models using conventional modeling software. These problems can be divided into four categories: maintaining position awareness, identifying and selecting objects or components of interest, recognizing the distance between objects or components, and realizing the relative position of objects or components. Based on the above categorization, five focus and context awareness techniques were developed for a multi-layer computer display to enable modelers to better maintain their focus and context awareness while performing 3D modeling tasks. These techniques are: object isolation, component segregation, peeling focus, slicing, and peeling focus and context. A user study was then conducted to compare the effectiveness of these focus and context awareness techniques with other tools provided by conventional 3D modeling software. The results of this study were used to further improve, and evaluate through a second study, the five focus and context awareness techniques. The two studies have demonstrated that some of these techniques are more effective in supporting 3D modeling tasks than other existing software tools

Intuitive Interaktion durch videobasierte Gestenerkennung

Hinter der Forschung an videobasierter Handgestenerkennung steht die Vision, Interaktion zwischen Mensch und Computer losgelöst von klassischen Eingabegeräten wie Maus und Tastatur zu realisieren. Das Ziel dieser Arbeit ist die Entwicklung von echtzeitfähigen Verfahren, die eine robuste und fehlerarme Erkennung menschlicher Handgesten realisieren und so die Bedienung eines Computersystems auch für technisch unerfahrene Anwender nutzbar machen. In dieser Arbeit werden vier Verfahren entwickelt, die unterschiedliche Arten der Interaktion durch videobasierte Handgestenerkennung realisieren.The vision behind research on video based hand gesture recognition is to realise a new kind of interaction between humans and computer beyond the classical input devices such as mouse and keyboard. The aim of this thesis is to develop new video based realtime algorithms, which enable a robust and accurate recognition of human hand gestures and allow interaction with the computer even for technically unversed users. In this thesis four different algorithms are developed that can be used for intuitive interaction purposes depending on the demands and needs of different scenario applications