Narrative balance management in an Intelligent biosafety training application for improving user performance

The use of three-dimensional virtual environments in training applications supports the simulation of complex scenarios and realistic object behaviour. While these environments have the potential to provide an advanced training experience to students, it is difficult to design and manage a training session in real time due to the number of parameters to pay attention to: timing of events, difficulty, user’s actions and their consequences or eventualities are some examples. For that purpose, we have extended our virtual Bio-safety Laboratory application used for training biohazard procedures with a Narrative Manager. The Narrative Manager controls the simulation deciding which events will take place in the simulation, and when, by controlling the narrative balance of the session. Our hypothesis is that the Narrative Manager allows us to increase the number of tasks for the user to solve and, due to balancing difficulty and intensity, it keeps the user interested in training. When evaluating our system we observed that the Narrative Manager effectively introduces more tasks for the user to solve, and despite that, is accepted by the users as more interesting and not harder than an identical system without a Narrative Manager. Also, a knowledge test demonstrated better results in users’ interest and learning output in the narrative condition

Non-invasive keyboard fatigue monitoring system for improving user performance and reducing incidences of Repetitive Strain Injuries

Computers are ubiquitous in their application and deployment all over the world. Along with their universal appeal and versatility they also pose dangers to their various users in the form of ailments such as Repetitive Strain Injuries, Carpals Tunnel Syndrome, etc. which are all specifically related to keyboard use. The objective of this thesis was to explore the possibility of developing a deterministic and non-invasive method of detecting keyboard fatigue. A software application was developed which allowed us to reliably monitor this as a function of the latency between keyboard key-press and key-release events recorded by the resident operating system. The latency trends that were observed through testing on three volunteers proved that the average latency calculated increased steadily with the onset of fatigue. Hence by estimating a threshold condition it was possible to train the system to estimate the fatigue level of the users and warn them appropriately at a considerably early stage of the condition

Assisting Human Decisions in Document Matching

Many practical applications, ranging from paper-reviewer assignment in peer review to job-applicant matching for hiring, require human decision makers to identify relevant matches by combining their expertise with predictions from machine learning models. In many such model-assisted document matching tasks, the decision makers have stressed the need for assistive information about the model outputs (or the data) to facilitate their decisions. In this paper, we devise a proxy matching task that allows us to evaluate which kinds of assistive information improve decision makers' performance (in terms of accuracy and time). Through a crowdsourced (N=271 participants) study, we find that providing black-box model explanations reduces users' accuracy on the matching task, contrary to the commonly-held belief that they can be helpful by allowing better understanding of the model. On the other hand, custom methods that are designed to closely attend to some task-specific desiderata are found to be effective in improving user performance. Surprisingly, we also find that the users' perceived utility of assistive information is misaligned with their objective utility (measured through their task performance)

Improving Performance by Encouraging Users to Consider Different Levels of Action Identification (LAI)

Action Identification Theory proposes that individuals perceive their actions at different levels of abstraction and how this perception can significantly impact their behavior. The paper argues that prompting users to shift between different levels of action identification during their interaction with an information system can improve their performance. The experimental work includes a laboratory experiment and a think-aloud study that explores the effect of users\u27 attention to different levels of action identification and the cognitive fit between on-screen representation and mental models on performance improvement. The discussion analyzes the results and outlines future research plans and expected contributions to the field. This study highlights the importance of considering the user\u27s cognitive processes when designing information systems and suggests potential ways to enhance their performance

A usability study of online library systems: A case of Sultanah Bahiyah Library, Universiti Utara Malaysia

The purpose of this study was to investigate usability of online library systems in Universiti Utara Malaysia (UUM). This study evaluated the usability of Sultanah Bahiyah Library’s web based systems by investigating the aspects of simplicity, comfort, user friendliness, control, readability, information adequacy/task match, navigability, recognition, access time, relevancy, consistency and visual presentation. This study examined user’s views about the usability of digital libraries whereas current and perceived importance. A sample of 45 students of Master of Business Administration (MBA) has been chosen. The Sultanah Bahiyah Library’s web based systems is very important especially for students and academic staffs of Universiti Utara Malaysia. The usability of the Library’s web based systems makes students easy to connect and for that the website should be helpful and attractive within good contents. The result found that the parallel nature of the users’ current views about the usability of digital libraries and users’ perceived importance of digital library usability allows direct comparison of all usability properties. The overall results yielded significant difference for the variables of user’s current views and perceived importance

SoundBar: exploiting multiple views in multimodal graph browsing

In this paper we discuss why access to mathematical graphs is problematic for visually impaired people. By a review of graph understanding theory and interviews with visually impaired users, we explain why current non-visual representations are unlikely to provide effective access to graphs. We propose the use of multiple views of the graph, each providing quick access to specific information as a way to improve graph usability. We then introduce a specific multiple view system to improve access to bar graphs called SoundBar which provides an additional quick audio overview of the graph. An evaluation of SoundBar revealed that additional views significantly increased accuracy and reduced time taken in a question answering task

Endoscopic Targeting Tasks Simulator: An Approach Using Game Engines

The pervasiveness of simulators used in professions requiring the skilled control of expensive machinery such as is the case in the aviation, mining, construction, and naval industries raises an intriguing question about the relatively poor adoption within the field of medicine. Certain surgical procedures such as neuro-endoscopic and laparoscopic lend themselves well to the application of virtual reality based simulators. This is due to the innate ability to decom- pose these complex macro level procedures into a hierarchy of subtasks that can be modelled in a software simulator to augment existing teaching and training techniques. The research in this thesis is focused with the design and implementation of a targeting- based simulator having applications in the evaluation of clinically relevant procedures within the neuro-endoscopic and potentially laparoscopic domains. Existing commercially available surgical simulators within these domains are often associated with being expensive, narrowly focussed in the skills they train, and fail to show statistically significant results in the efficacy of improving user performance through repeated use. Development of a targeting tasks simulator is used to evaluate what methods can be applied to provide a robust, objective measure of human performance as it relates to targeting tasks. In addition to performance evaluation, further research is conducted to help understand the impact of different input modalities; focusing primarily on input from a gamepad style device and as well a newer, more natural user interface provided by the Leap Motion Controller

For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI

Counterfactual explanations (CFEs) are a popular approach in explainable artificial intelligence (xAI), highlighting changes to input data necessary for altering a model's output. A CFE can either describe a scenario that is better than the factual state (upward CFE), or a scenario that is worse than the factual state (downward CFE). However, potential benefits and drawbacks of the directionality of CFEs for user behavior in xAI remain unclear. The current user study (N=161) compares the impact of CFE directionality on behavior and experience of participants tasked to extract new knowledge from an automated system based on model predictions and CFEs. Results suggest that upward CFEs provide a significant performance advantage over other forms of counterfactual feedback. Moreover, the study highlights potential benefits of mixed CFEs improving user performance compared to downward CFEs or no explanations. In line with the performance results, users' explicit knowledge of the system is statistically higher after receiving upward CFEs compared to downward comparisons. These findings imply that the alignment between explanation and task at hand, the so-called regulatory fit, may play a crucial role in determining the effectiveness of model explanations, informing future research directions in xAI. To ensure reproducible research, the entire code, underlying models and user data of this study is openly available: https://github.com/ukuhl/DirectionalAlienZooComment: 22 pages, 3 figures This work has been accepted for presentation at the 1st World Conference on eXplainable Artificial Intelligence (xAI 2023), July 26-28, 2023 - Lisbon, Portuga