Measuring Actual Behaviors in HCI Research – A call to Action and an Example

There have been repeated calls for studies in behavioral science and human-computer interaction (HCI) research to measure participants’ actual behaviors. HCI research studies often use multiple constructs as perceived measures of behavior, which are captured using participants’ self-reports on surveys. Response biases, however, are a widespread threat to the validity of self-report measures. To mitigate this threat to validity, we propose that studies in HCI measure actual behaviors in appropriate contexts rather than solely perceptions. We report an example of using movements that reflect both actual behavior and behavioral changes measured within a health care IS usage context, specifically the detection and alleviation of neuromuscular degenerative disease. We propose and test a method of monitoring mouse-cursor movements to detect hand tremors in real time when individuals are using websites. Our work suggests that analyzing hand movements as an actual (rather than perceptual) measure of usage could enrich other areas of IS research (e.g., technology acceptance, efficacy, fear, etc.), in which perceptions of states and behavior are measured post hoc to the interaction and subject to the threats of various forms of response bias

Improving the Quality of Survey Data: Using Answering Behavior as an Alternative Method for Detecting Biased Respondents

Online surveys are used for collecting self-report data. Despite their prevalent use, data quality problems persist due to various response biases. Here, we demonstrate how participant answering behaviors can be used to identify biased responses. We administered an online survey where participants reported their personality dimensions of neuroticism and extraversion—two personality dimensions that have been previously shown to be correlated with a propensity to deceive—and were later presented with a scenario to exhibit deceptive behavior. We then generated models to predict deception using the neuroticism and extraversion constructs. Using respondents’ fine-grained mouse movement data when answering these questions, we generated time, behavior, and navigation-based metrics to identify biased participants. By removing these outliers, model performance improved by 93% for neuroticism and 10% for extraversion. This approach aids in gaining a clearer understanding of how some types of response biases influence model performanc

(Optimal) governance of research support by “Survey Methodology”

Survey research is an integral element of modern social science. The infrastructure in terms of research institute, surveys, conferences and journals has greatly improved over the past 20 years and recently several initiatives have gained momentum even on a European level. This development has brought about the need for an integrated theoretical concept in order to assess and evaluate the quality of surveys and survey estimates. In our view, survey methodology is an interdisciplinary body of knowledge and expertise that describes the “science of conducting and evaluating surveys”. It is a theory-driven empirical approach to assess the quality of survey research. Thus, it applies the principles of survey research to the development and assessment of this very method. Even though surveys have been conducted in a highly professional manner for decades, survey methodology offers the opportunity to use a universal theoretical approach when planning and assessing surveys and also a joint terminology. Both, the integrated theoretical concept and the joint terminology foster the professionalization of survey methods and stimulates methodological research on the improvement of survey methods. One key element of survey methodology is the total survey error framework. It shall be described in greater detail below (section 1). Afterwards we will discuss some limitations of this concept (section 2) and mechanisms ad organizational issues in order to promote the use of this concept (section 3).

Development and Maintenance of Self-Disclosure on Facebook: The Role of Personality Traits

This study explored the relationships between Facebook self-disclosure and personality traits in a sample of Italian users. The aim was to analyze the predictive role of Big Five personality traits on different parameters of breadth and depth of selfdisclosed behaviors online. Facebook users, aged between 18 and 64 years of age (Mage = 25.3 years, SD = 6.8; N = 958), of which 51% were female, voluntarily completed an online survey assessing personality traits and Facebook self-disclosure. Results at a series of hierarchical regression analyses significantly corroborated the hypotheses that high extroverted and openness people tend to disclose on Facebook a significant amount of personal information, whereas high consciousness and agreeableness users are less inclined to do it. Furthermore, more extroverts and agreeableness people develop less intimacy on Facebook, differently from those with high levels of openness. Results also corroborated the hypothesis of a full mediation of time usage in the relationship between personality factors such as extroversion and conscientiousness with breadth of Facebook self-disclosure. Overall, according to the findings of the current study, personality traits and Facebook self-disclosure become central both as predictive variables for depicting the different profiles of potential addicted and as variables to help educators, teachers, and clinicians to develop training or therapeutic programs aimed at preventing the risk of Internet addiction. Limitations of the study are discussed, and directions for future research are suggested

An Empirical Study Comparing Unobtrusive Physiological Sensors for Stress Detection in Computer Work.

Several unobtrusive sensors have been tested in studies to capture physiological reactions to stress in workplace settings. Lab studies tend to focus on assessing sensors during a specific computer task, while in situ studies tend to offer a generalized view of sensors' efficacy for workplace stress monitoring, without discriminating different tasks. Given the variation in workplace computer activities, this study investigates the efficacy of unobtrusive sensors for stress measurement across a variety of tasks. We present a comparison of five physiological measurements obtained in a lab experiment, where participants completed six different computer tasks, while we measured their stress levels using a chest-band (ECG, respiration), a wristband (PPG and EDA), and an emerging thermal imaging method (perinasal perspiration). We found that thermal imaging can detect increased stress for most participants across all tasks, while wrist and chest sensors were less generalizable across tasks and participants. We summarize the costs and benefits of each sensor stream, and show how some computer use scenarios present usability and reliability challenges for stress monitoring with certain physiological sensors. We provide recommendations for researchers and system builders for measuring stress with physiological sensors during workplace computer use

Evaluation Strategy for the Re-Development of the Displays and Visitor Facilities at the Museum and Art Gallery, Kelvingrove

No abstract available

Knowledge extraction from pointer movements and its application to detect uncertainty

This work was supported by the Doctoral Program NOVA I4H (Fundacao para a Ciencia e a Tecnologia) [grant PD/BDE/114561/2016].Pointer-tracking methods can capture a real-time trace at high spatio-temporal resolution of users' pointer interactions with a graphical user interface. This trace is potentially valuable for research on human-computer interaction (HCI) and for investigating perceptual, cognitive and affective processes during HCI. However, little research has reported spatio-temporal pointer features for the purpose of tracking pointer movements in on-line surveys. In two studies, we identified a set of pointer features and movement patterns and showed that these can be easily distinguished. In a third study, we explored the feasibility of using patterns of interactive pointer movements, or micro-behaviours, to detect response uncertainty. Using logistic regression and k-fold cross-validation in model training and testing, the uncertainty model achieved an estimated performance accuracy of 81%. These findings suggest that micro-behaviours provide a promising approach toward developing a better understanding of the relationship between the dynamics of pointer movements and underlying perceptual, cognitive and affective psychological mechanisms. Human-computer interaction; Pointer-tracking; Mouse movement dynamics; Decision uncertainty; On-line survey; Spatio-temporal features; Machine learningproofpublishe

Contours of Inclusion: Frameworks and Tools for Evaluating Arts in Education

This collection of essays explores various arts education-specific evaluation tools, as well as considers Universal Design for Learning (UDL) and the inclusion of people with disabilities in the design of evaluation instruments and strategies. Prominent evaluators Donna M. Mertens, Robert Horowitz, Dennie Palmer Wolf, and Gail Burnaford are contributors to this volume. The appendix includes the AEA Standards for Evaluation. (Contains 10 tables, 2 figures, 30 footnotes, and resources for additional reading.) This is a proceedings document from the 2007 VSA arts Research Symposium that preceded the American Evaluation Association's (AEA) annual meeting in Baltimore, MD