Predicting users’ behavior using mouse movement information: an information foraging theory perspective

The prediction of users’ behavior is essential for keeping useful information on the web. Previous studies have used mouse cursor information in web usability evaluation and designing user-oriented search interfaces. However, we know fairly to a small extent pertaining to user behavior, specifically clicking and navigating behavior, for prolonged search session illustrating sophisticated search norms. In this study, we perform extensive analysis on a mouse movement activities dataset to capture every users’ movement pattern using the effects of information foraging theory (IFT). The mouse cursor movement information dataset includes the timing and positioning information of mouse cursors collected from several users in different sessions. The tasks vary in two dimensions: (1) to determine the interactive elements (i.e., information episodes) of user interaction with the site; (2) adopt these findings to predict users’ behavior by exploiting the LSTM model. Our model is developed to find the main patterns of the user’s movement on the site and simulate the behavior of users’ mouse movement on any website. We validate our approach on a mouse movement dataset with a rich collection of time and position information of mouse pointers in which searchers and websites are annotated by web foragers and information patches, respectively. Our evaluation shows that the proposed IFT-based effects provide an LSTM model a more accurate interpretative exposition of all the patterns in the movement of the users’ mouse cursors across the screen

Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation

This paper proposes an approach to detect information relevance during decision-making from eye movements in order to enable user interface adaptation. This is a challenging task because gaze behavior varies greatly across individual users and tasks and groundtruth data is difficult to obtain. Thus, prior work has mostly focused on simpler target-search tasks or on establishing general interest, where gaze behavior is less complex. From the literature, we identify six metrics that capture different aspects of the gaze behavior during decision-making and combine them in a voting scheme. We empirically show, that this accounts for the large variations in gaze behavior and out-performs standalone metrics. Importantly, it offers an intuitive way to control the amount of detected information, which is crucial for different UI adaptation schemes to succeed. We show the applicability of our approach by developing a room-search application that changes the visual saliency of content detected as relevant. In an empirical study, we show that it detects up to 97% of relevant elements with respect to user self-reporting, which allows us to meaningfully adapt the interface, as confirmed by participants. Our approach is fast, does not need any explicit user input and can be applied independent of task and user.Comment: The first two authors contributed equally to this wor

Desmitificando el valor de las imágenes de celebridades en sitios web de establecimientos de alojamiento turístico

La presencia de celebridades en el diseño Web se considera valiosa por sus influenciasen el atractivo visual y la credibilidad percibidos, así como en el tiempo dedicado por losusuarios de la Generación Y a ver los sitios. La investigación y las evidencias empíricas alrespecto son escasas, lo que compromete la efectividad de su uso en páginas Web. Se hizouna investigación causal con un experimento verdadero y rastreo ocular en el que participaron126 sujetos. Los hallazgos refutan esas ideas y revelan que la presencia de celebridadesen esos contenidos puede ser contraproducente para las empresas

AdWords, images, and banner blindness: an eye-tracking study

Unlike most online advertising, Google opts for a purely textual model in the ads showed in its results pages. This paper presents an eye-tracking study conducted to determine if right-side graphic elements in Google AdWords  affect users’ visual behavior. AdWords  have been modified to contain logos and pictures. The analysis concludes that the appearance of images does not change user interaction with ads, which is attributed to the “banner blindness” effect

Theory and practice in visual interfaces for semi-structured document discovery and selection

With the increase in electronic publications, and indeed the availability of existing publications in digital form, as well as the encouragement of open access publication, comes a challenge. That challenge is to create assistive software to aid in the discovery and selection of relevant documents to one’s information need. Visual interfaces have begun to address the need of information seekers in finding publications and wading through the large result sets that are returned from search engines. There is currently, little evidence to suggest that these interfaces are based on systematic research on requirements. In this article, we examine areas which contribute to the theory and practice of visual interfaces directly relating to the discovery and selection of publications. We bring together work from different fields in a targeted approach to assist the future creation of these interfaces

A Study of Immediate Requery Behavior in Search

© Haotian Zhang, Mustafa Abualsaud and Mark D. Smucker, 2018. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive version was published in the Proceedings of the 2018 Conference on Human Information Interaction & Retrieval, (CHIIR'18), ACM. , https://doi.org/10.1145/3176349.3176400When search results fail to satisfy users' information needs, users often reformulate their search query in the hopes of receiving better results. In many cases, users immediately requery without clicking on any search results. In this paper, we report on a user study designed to investigate the rate at which users immediately reformulate at different levels of search quality. We had users search for answers to questions as we manipulated the placement of the only relevant document in a ranked list of search results. We show that as the quality of search results decreases, the probability of immediately requerying increases. We find that users can quickly decide to immediately reformulate, and the time to immediately reformulate appears to be independent of the quality of the search results.Finally, we show that there appears to be two types of users. One group has a high probability of immediately reformulating and the other is unlikely to immediately reformulate unless no relevant documents can be found in the search results. While requerying takes time, it is the group of users who are more likely to immediately requery that are able to able find answers to questions the fastest.Natural Sciences and Engineering Research Council of Canada (Grants CRDPJ 468812-14 and RGPIN-2014-03642), in part by Google, and in part by the University of Waterloo

Patterns of Search Result Examination: Query to First Action.

To determine key factors that affect a user's behavior with search results, we conducted a controlled eye-tracking study of users completing search tasks using both desktop and mobile devices. We focus our investigation on users' behavior from their query to the first action they take with the search engine results page (SERP): either a click on a search result or a reformulation of their query. We found that a user deciding to reformulate a query rather than click on a result is best understood as being caused by the user's examination pattern not including a relevant search result. If a user sees a relevant result, they are very likely to click it. Of note, users do not look at all search results and their examination may be influenced by other factors. The key factors we found to explain a user's examination pattern are: the rank of search results, the user type, and the query quality. While existing research has identified rank and user types as important factors affecting examination patterns, to our knowledge, query quality is a new discovery. We found that user queries can be understood as either of weak or strong quality. Weak queries are those that the user may believe are more likely to fail compared to a strong query, and as a result, we find that users modify their examination patterns to view fewer documents when they issue a weak query, i.e. they give up sooner.Natural Sciences and Engineering Research Council of Canada, Grant CRDPJ 468812-14 || Natural Sciences and Engineering Research Council of Canada, Grant RGPIN-2014-03642 || Google || University of Waterlo

Using eye tracking and heart-rate activity to examine crossmodal correspondences QoE in Mulsemedia

Different senses provide us with information of various levels of precision and enable us to construct a more precise representation of the world. Rich multisensory simulations are thus beneficial for comprehension, memory reinforcement, or retention of information. Crossmodal mappings refer to the systematic associations often made between different sensory modalities (e.g., high pitch is matched with angular shapes) and govern multisensory processing. A great deal of research effort has been put into exploring cross-modal correspondences in the field of cognitive science. However, the possibilities they open in the digital world have been relatively unexplored. Multiple sensorial media (mulsemedia) provides a highly immersive experience to the users and enhances their Quality of Experience (QoE) in the digital world. Thus, we consider that studying the plasticity and the effects of cross-modal correspondences in a mulsemedia setup can bring interesting insights about improving the human computer dialogue and experience. In our experiments, we exposed users to videos with certain visual dimensions (brightness, color, and shape), and we investigated whether the pairing with a cross-modal matching sound (high and low pitch) and the corresponding auto-generated vibrotactile effects (produced by a haptic vest) lead to an enhanced QoE. For this, we captured the eye gaze and the heart rate of users while experiencing mulsemedia, and we asked them to fill in a set of questions targeting their enjoyment and perception at the end of the experiment. Results showed differences in eye-gaze patterns and heart rate between the experimental and the control group, indicating changes in participants’ engagement when videos were accompanied by matching cross-modal sounds (this effect was the strongest for the video displaying angular shapes and high-pitch audio) and transitively generated cross-modal vibrotactile effects.<?vsp -1pt?

Driven to distraction : examining the influence of distractors on search behaviours, performance and experience

Advertisements, sponsored links, clickbait, in-house recommendations and similar elements pervasively shroud featured content. Such elements vie for people's attention, potentially distracting people from their task at hand. The effects of such "distractors" is likely to increase people's cognitive workload and reduce their performance as they need to work harder to discern the relevant from non-relevant. In this paper, we investigate how people of varying cognitive abilities (measured using Perceptual Speed and Cognitive Failure instruments) are affected by these different types of distractions when completing search tasks. We performed a crowdsourced within-subjects user study, where 102 participants completed four search tasks using our news search engine over four different interface conditions: (i) one with no additional distractors; (ii) one with advertisements; (iii) one with sponsored links; and (iv) one with in-house recommendations. Our results highlight a number of important trends and findings. Participants perceived the interface condition without distractors as significantly better across numerous dimensions. Participants reported higher satisfaction, lower workload, higher topic recall, and found it easier to concentrate. Behaviourally, participants issued queries faster and clicked results earlier when compared to the interfaces with distractors. When using the interfaces with distractors, one in ten participants clicked on a distractor—and despite engaging with a distractor for less than twenty seconds, their task time increased by approximately two minutes. We found that the effects were magnified depending on cognitive abilities—with a greater impact of distractors on participants with lower perceptual speed, and for those with a higher propensity of cognitive failures. Distractors—regardless of their type—have negative consequences on a user’s search experience and performance. As a consequence, interfaces containing visually distracting elements are creating poorer search experiences due to the "distractor tax" being placed on people's limited attention