Exploring the Effect of Arousal and Valence on Mouse Interaction

Determining a user’s affective state can be an important element when trying to understand human-computer interactions. Accurately assessing affect during system use, however, can be very difficult, especially in a non-laboratory setting. Extensive previous research in neuroscience has shown that arousal and valence influence motor control. In this research, the prior relevant neuroscience findings inform the investigation of mouse movement behavior under conditions of low and high arousal as well as positive and negative valence. A controlled laboratory experiment was conducted, providing support for hypotheses predicting that arousal and valence may be inferred by monitoring for differences in the distance, speed, and trajectory of mouse movement. Implications of these results for future research and practice are explored

Understanding Unauthorized Access using Fine-Grained Human-Computer Interaction Data

Unauthorized Data Access (UDA) by an internal employee is a major threat to an organization. Regardless of whether the individuals engaged in UDA with malicious intent or not, real-time identification of UDA events and anomalous behaviors is extremely difficult. For example, various artificial intelligence methods for detecting insider threat UDA have become readily available; while useful, such methods rely on post hoc analysis of the past (e.g., unsupervised learning algorithms on access logs). This research-in-progress note reports on if the analysis of Human-Computer Interaction (HCI) behaviors, which have been empirically validated in various studies to reveal hidden cognitive state, can be utilized as a method to detect UDAs. To examine this, an experimental design was required that would grant the subjects an opportunity to engage in UDA events while tracking the HCI behaviors in an unobtrusive manner. Background, experimental design, study execution, preliminary results, and future research plans are presented

Hearth: A Game Supporting Non-Intrusive and Concurrent Tracking of Player Emotion and Mouse Usage

Empirical evidence has supported the idea that eSports players\u27 emotions could be reflected in their mouse usage. Still, findings from IS literature on the exact relationships between users\u27 mouse usage patterns and their emotional states have been mixed. Possible causes include adjustment effects and offsetting effects. To address these problems, this study proposes a self-developed game named Hearth, which supports non-intrusive and concurrent tracking of players\u27 emotions and mouse usage. The game design supports the examination of the two possible effects. Results show that negative emotion was positively associated with the total mouse movement distance in a game turn, average task-level distance, and average task-level speed. Moreover, the open-source game proposed in this study facilitates further data collection from natural experiments due to its triadic design that addresses reality, meaning, and play

Role of Emotions and Aesthetics in ICT Usage for Underserved Communities: A NeuroIS Investigation

Usability and efficiency has received lot of attention in terms of ICT usage and attitude however non instrumental factors like emotions and aesthetics and their impact on ICT usage attitude and performance has not been extensively tested. Further underserved communities are focused communities that have limitations in terms of formal and functional literacy and technology experience. Aesthetics have been shown to be an important predictor of usage but this has not been tested in underserved communities. Also positive emotions have been linked to greater ICT usage as well as aesthetic experience. Measurement of factors like emotions, aesthetic preferences and ICT usage has so far been restricted to questionnaires however we propose to use objective measures like brain imaging technique (EEG) to supplement existing methodologies. The current paper is a research in progress that addresses potential role of aesthetics and emotions for understanding aesthetic preferences and ICT usages in underserved communities

Thinking Fast or slow? Understanding Answering Behavior Using Dual-Process Theory through Mouse Cursor Movements

Users’ underlying cognitive states govern their behaviors online. For instance, an extreme cognitive burden during live system use would negatively influence important user behaviors such as using the system and purchasing a product. Thus, inferring the user's cognitive state has practical significance for the commercialized systems. We use Dual-Process Theory to explain how the mouse cursor movements can be an effective measure of cognitive load. In an experimental study with five hundred and thirty-four subjects, we induced cognitive burden then monitored mouse cursor movements when the participants answered questions in an online survey. We found that participants' mouse cursor movements slow down when they are engaged in cognitively demanding tasks. With the newly derived measures, we were able to infer the state of heightened cognitive load with an overall accuracy of 70.22%. The results enable researchers to measure users' cognitive load with more granularity and present a new, theoretically sound method to assess the user's cognitive state

Measuring Actual Behaviors in HCI Research – A call to Action and an Example

There have been repeated calls for studies in behavioral science and human-computer interaction (HCI) research to measure participants’ actual behaviors. HCI research studies often use multiple constructs as perceived measures of behavior, which are captured using participants’ self-reports on surveys. Response biases, however, are a widespread threat to the validity of self-report measures. To mitigate this threat to validity, we propose that studies in HCI measure actual behaviors in appropriate contexts rather than solely perceptions. We report an example of using movements that reflect both actual behavior and behavioral changes measured within a health care IS usage context, specifically the detection and alleviation of neuromuscular degenerative disease. We propose and test a method of monitoring mouse-cursor movements to detect hand tremors in real time when individuals are using websites. Our work suggests that analyzing hand movements as an actual (rather than perceptual) measure of usage could enrich other areas of IS research (e.g., technology acceptance, efficacy, fear, etc.), in which perceptions of states and behavior are measured post hoc to the interaction and subject to the threats of various forms of response bias

Investigating the Effect of Insurance Fraud on Mouse Usage in Human-Computer Interactions

The completion of online forms is the catalyst for many business and governmental processes. However, providing fraudulent information in such forms is pervasive, resulting in costly consequences for organizations and society. Furthermore, detecting fraudulent responses in online forms is often very difficult, time consuming, and expensive. This research proposes that analyzing users’ mouse movements may reveal when a person is being fraudulent. Namely, based on neuroscience and deception theory, the paper explains how deception may influence hand movements captured via the computer mouse. In an insurance fraud context, a study is conducted to explore these proposed relationships. The results suggest that being deceptive may increase the normalized distance of movement, decrease the speed of movement, increase the response time, and result in more left clicks. Implications for human-computer interaction research and practice are discussed

Design And Lab Experiment Of A Stress Detection Service Based On Mouse Movements

Workplace stress can negatively affect the health condition of employees and with it, the performance of organizations. Although there exist approaches to measure work-related stress, two major limitations are the low resolution of stress data and its obtrusive measurement. The current work applies design science research with the goal to design, implement and evaluate a Stress Detection Service (SDS) that senses the degree of work-related stress solely based on mouse movements of knowledge workers. Using van Gemmert and van Galen’s stress theory and Bakker and Demerouti’s Job Demands-Resource model as justificatory knowledge, we implemented a first SDS prototype that senses mouse movements and perceived stress levels. Experimental results indicate that two feature sets of mouse movements, i.e. average deviation from an optimal mouse trajectory and average mouse speed, can classify high versus low stress with an overall accuracy of 78%. Future work regarding a second build-and-evaluate loop of a SDS, then tailored to the field setting, is discussed

Sleight of Hand: Identifying Concealed Information by Monitoring Mouse-Cursor Movements

Organizational members who conceal information about adverse behaviors present a substantial risk to that organization. Yet the task of identifying who is concealing information is extremely difficult, expensive, error-prone, and time-consuming. We propose a unique methodology for identifying concealed information: measuring people’s mouse-cursor movements in online screening questionnaires. We theoretically explain how mouse-cursor movements captured during a screening questionnaire differ between people concealing information and truth tellers. We empirically evaluate our hypotheses using an experiment during which people conceal information about a questionable act. While people completed the screening questionnaire, we simultaneously collected mouse-cursor movements and electrodermal activity—the primary sensor used for polygraph examinations—as an additional validation of our methodology. We found that mouse-cursor movements can significantly differentiate between people concealing information and people telling the truth. Mouse-cursor movements can also differentiate between people concealing information and truth tellers on a broader set of comparisons relative to electrodermal activity. Both mouse-cursor movements and electrodermal activity have the potential to identify concealed information, yet mouse-cursor movements yielded significantly fewer false positives. Our results demonstrate that analyzing mouse-cursor movements has promise for identifying concealed information. This methodology can be automated and deployed online for mass screening of individuals in a natural setting without the need for human facilitators. Our approach further demonstrates that mouse-cursor movements can provide insight into the cognitive state of computer users