Reflecting upon perceptual speed tests in information retrieval : limitations, challenges, and recommendations

Perceptual Speed (PS) is a cognitive ability defined by an individual’s accuracy and speed to scan information while completing visual search tasks. Prior studies using PS tests have demonstrated that PS affects multiple factors in Information Retrieval (IR), such as a user’s search performance, interaction with the system, time spent completing tasks, and subjective impression of their workload. With greater knowledge of PS, systems could be designed that accommodate users with low PS to improve their overall search experience. However, in this perspectives paper, we analyse how PS tests have been used in IR, and identify multiple uncertainties regarding PS content, administration, analysis, and reporting of findings. Consequently, we aim to stir discussion between IR researchers by drawing awareness to these issues. As a result, we further discuss challenges involved in advancing how future PS tests are used in IR. Finally, we propose recommendations that have the potential for enhancing the reliability and validity of current PS tests

Uma proposta metodológica para avaliação de bibliotecas digitais: usabilidade e comportamento de busca por informação na Biblioteca Digital da Puc-Minas

Atualmente, com o desenvolvimento da Internet, novos sistemas e fontes de informação têm sido criados, colocando o usuário em um novo contexto de busca por informação. Diante disso, autores da ciência da informação e da ciência da computação têm se dedicado a estudos sobre a interação dos usuários com os sistemas neste novo ambiente. Este artigo apresenta uma pesquisa realizada no curso de Mestrado da Escola de Ciência da Informação da Universidade Federal de Minas Gerais (UFMG), que propõe que os estudos das duas áreas sejam usados conjuntamente, complementando-se, com o intuito de se analisar de forma abrangente um sistema de informação. Para teste da metodologia proposta, foi usada a Biblioteca Digital da PUC-Minas (BDP). A metodologia é composta por cinco etapas: (1)questionário sobre o contexto do usuário; (2) teste de usabilidade em laboratório; (3) questionário de satisfação; (4) estudo do comportamento de busca dos usuários através do teste com busca livre; (5) análise do log. Por fim, algumas conclusões são apresentadas sobre a aplicação da metodologia proposta. Palavras-chave Usabilidade; Comportamento de busca por informação; Bibliotecas digitais A methodological proposal for evaluation of digital libraries: usability and behavior in seeking information at the Digital Library of PUC-Minas Abstract Recently, with the development of the Internet, new systems and information sources have been created, moving the user to a new context of information seeking. Considering this, authors of Information Science and Computer Science have been studying about interaction between users and systems in this new environment. Thus, this work proposes the complementary and simultaneous use of methodologies in both areas, aiming a better evaluation of an information system. To test the proposed methodology this research is conducted at the Digital Library of Theses and Dissertations of the Catholic University of Minas Gerais, Brazil. The data collecting was divided into five steps: (1) Demographic questionnaire; (2) Usability testing; (3) Satisfaction questionnaire; (4) Study of information seeking behavior using a free navigation test and (5) Log analysis. At last, some conclusions are presented about the proposed methodology. Keywords Usability; Information seeking behavior; Digital Libraries

User analysis in HCI-the historical lessons from individual differences research

User analysis is a crucial aspect of user-centered systems design , yet HumanComputer Interaction (HCI) has yet to formulate reliable and valid characterizations of users beyond gross distinctions based on task and experience . Individual dif ferences research from mainstream psychology has identified a stable set of characteristics that would appear to of fer potential application in the HCI arena . Furthermore , in its evolution over the last 100 years , research on individual dif ferences has faced many of the problems of theoretical status and applicability that are common to HCI . In the present paper , the relationship between work in cognitive and dif ferential psychology and current analyses of users in HCI is examined . It is concluded that HCI could gain significant predictive power if individual dif ferences research was related to the analysis of users in contemporary systems design

Driven to distraction : examining the influence of distractors on search behaviours, performance and experience

Advertisements, sponsored links, clickbait, in-house recommendations and similar elements pervasively shroud featured content. Such elements vie for people's attention, potentially distracting people from their task at hand. The effects of such "distractors" is likely to increase people's cognitive workload and reduce their performance as they need to work harder to discern the relevant from non-relevant. In this paper, we investigate how people of varying cognitive abilities (measured using Perceptual Speed and Cognitive Failure instruments) are affected by these different types of distractions when completing search tasks. We performed a crowdsourced within-subjects user study, where 102 participants completed four search tasks using our news search engine over four different interface conditions: (i) one with no additional distractors; (ii) one with advertisements; (iii) one with sponsored links; and (iv) one with in-house recommendations. Our results highlight a number of important trends and findings. Participants perceived the interface condition without distractors as significantly better across numerous dimensions. Participants reported higher satisfaction, lower workload, higher topic recall, and found it easier to concentrate. Behaviourally, participants issued queries faster and clicked results earlier when compared to the interfaces with distractors. When using the interfaces with distractors, one in ten participants clicked on a distractor—and despite engaging with a distractor for less than twenty seconds, their task time increased by approximately two minutes. We found that the effects were magnified depending on cognitive abilities—with a greater impact of distractors on participants with lower perceptual speed, and for those with a higher propensity of cognitive failures. Distractors—regardless of their type—have negative consequences on a user’s search experience and performance. As a consequence, interfaces containing visually distracting elements are creating poorer search experiences due to the "distractor tax" being placed on people's limited attention

An exploratory Study of IR Interaction for User Interface Design

Information seeking is an dynamic and interactive process. Factors like users' information needs, individual differences, goals and tasks, knowledge and cognitive abilities etc. influence the information seeking process, and need to be identified and supported in the user interface design. We adopt a user-centered approach to establish a link between research within the IR interaction perspective and the methods in HCI on how to evaluate information seeking interaction in a hypertext IR system (Dienst). Our purpose with this exploratory study is to identify, describe and acquire knowledge of characteristics of the user population, and finally, to make suggestions for supporting users in user interface design. For the evaluation task, we have applied HCI evaluation techniques to our IR evaluation to make a connection between the traditional IR evaluation and HCI evaluation, combining different qualitative and quantitative data collection and analyzing methods, implemented in an experimental real-world online WWW setting. This methodology combined online (WWW-based) questionnaires and database log statistics. Preliminary results revealed several "hidden" realities: a mismatch between what people said they wanted to do as opposed to what they actually did. We also observed that people initially expected a specific function, but when using the system, they did not use it. Finally, we established some group differences concerning variables like previous experience searching information in hypertext systems, IR knowledge and browsing/searching strategies

Relationships between user performance and spatial ability in using map-based software on pen-based devices

This thesis explored the association between individual differences and software user performance. The primary goal was to investigate the relationship between spatial ability and user performance in the context of map software. Two studies were conducted to investigate this goal. There were two hypotheses that applied to both user studies. First, we expected that performance measures such as time to complete a task or accuracy would be positively correlated with spatial ability, as measured by scores on cognitive assessments for sub-factors of spatial ability. Second, we hypothesized that differences between subjects with low spatial ability and subjects with high spatial ability would be more pronounced with complex tasks and interfaces, such as a map interface or a task to compare ground settings with a map, relative to simpler tasks, such as finding a button that matched a target color or using map software with guidance on how to use the software to execute the task protocol. Each user study also incorporated additional questions unique to the study. Our first user study investigated how the relationship between spatial ability and performance might be affected by the relevance of the button label and alternative button layouts. This research indicated that good button labeling helps to mitigate performance differences among subjects with varying spatial ability. Additionally, we found support that performance differences related to spatial ability were most apparent in complex conditions. The second study investigated whether the association between spatial ability and user performance was affected by offering guidance on performing a task and whether orientation and visualization were dissociable factors for a task involving ground to map comparisons. While there were no benefits found from the guidance, visualization and orientation skills were found to be dissociable from one another for accuracy and tool use. Findings from both studies underscore the importance of interface design in potentially reducing costs and burden for users of map-based software in a small screen computing environment

Usabilidad de un programa de rehabilitación neuropsicológica por ordenador "Gradior" en personas con enfermedad mental grave y prolongada

[ES]La usabilidad es un factor clave en el desarrollo de aplicaciones exitosas de software interactivo. Por ello la importancia de medir el nivel de usabilidad en un programa de rehabilitación neuropsicológica Gradior en personas con enfermedad mental grave y prolongada (EMGyP). El objetivo de este estudio es valorar la usabilidad, satisfacción y pedagogía del software de rehabilitación neuropsicológica por ordenador Gradior con la intención de obtener información relevante del impacto de los Tecnología de Información y Comunicación (TIC) en personas con EMGyP y poder realizar mejoras en nuevas versiones Gradior 4.0. Metodología El diseño de la investigación es correlacional; por medio del Test de Usuario que es el método heurístico complementario para valorar si el programa Gradior cumple con los estándares de usabilidad, y en nuestra caso, empleamos una encuesta para obtener información sobre los estándares de usabilidad y el grado satisfacción de los pacientes al emplear la herramienta. Seguidamente se seleccionamos una muestra la cual esta constituida por 83 personas con enfermedad mental grave y prolongada del Complejo Asistencial de la Unidad Clínica de Gestión de Salud Mental, (Área de Neuropsicología Clínica) del Complejo Asistencial de Zamora. El instrumento de medida para el Programa de Rehabilitación Neuropsicológico Gradior ha sido el cuestionario “Encuesta de Usabilidad” que diseñaron los 2 profesionales de Salud Mental Complejo Asistencial de Zamora y el Centro de Investigaciones en Tecnologías Gráficas de la Universidad Politécnica de Valencia mediante estructuras gramaticales y vocabulario sencillo y comprensible utilizando una escala Likert del 1: (totalmente en desacuerdo) ; al 5 : (totalmente desacuerdo); y también de manera más gráfica y visual se utilizaron emoticonos según la puntuación 1: (emoticono triste) a 5 : (emoticonos alegre), de forma que la persona pueda dar sus respuestas con facilidad. Igualmente, la encuesta se dividió en 1. Sistema de ayuda; 2. Estructura del programa; 3. Pruebas; 4. Contenido y 5. Apariencia del programa. Haciéndose un análisis de los siguientes componentes : a. interfaz, b. funcionalidad, c. accesibilidad, d. interacción y e. utilidad. Además se hará un análisis de la satisfacción de los usuarios a la hora de utilizar la herramienta. A partir de la finalización de las entrevistas individuales con los usuarios y recogida de la información se aplicará el SPSS 17 para cuantificar las respuestas y analizar la varianza por medio del ANOVA y análisis factorial . Entendemos que un proyecto de investigación con estas característicos puede ser viable con los apoyos institucionales ineludibles de la Universidad de Salamanca, Unidad de Gestión de Salud Mental, Servicio de Psiquiatría, Área de Neuropsicología Clínica del Complejo Asistencial de Zamora y la Fundación INTRAS. Ya se han establecido los contactos necesarios con estas instituciones que han expresado su compromiso de participación y realizar este estudio. Para llevar a cabo esta investigación en el Complejo Asistencial de Zamora, Unidad de Gestión de Salud Mental, Servicio de Psiquiatría del Hospital Provincial de Zamora se cuenta con los recursos materiales: biblioteca de consulta, diversos test y materiales de rehabilitación cognitiva; medios informáticos hardware y software, 3 diferentes materiales de oficina y el programa Gradior – y los recursos humanos: profesionales de la Unidad de Gestión de Salud Mental, Universidad de Salamanca y Fundación Intras. Resultados La media de edad de los 83 pacientes que participaron en el estudio es de 44 años las mujeres y 39 los hombres. La evaluación de los índices de usabilidad determinaron un: 83.9% utilidad, 83.1% Interfaz; la satisfacción el 82.2%; el diseño un 81.1%; accesibilidad el 80,7% ; la interacción el 78.3%; la funcionalidad el 77.8%. Además el 77.1% de los sujetos sentía que mejoraban su calidad de vida e independencia y el 83.7% mejoraban sus déficit cognitivos. Además el 81.2% aceptan la herramienta y el 84.3% se siente motivados con el uso de la herramienta. Discusión Los resultados al evaluar la usabilidad proporcionan información relevante del uso del programa de intervención neurocognitiva Gradior. También reportan un impacto positivo en el deterioro cognitivo, calidad de vida e independencia. Gradior tiene los estándares de usabilidad para ser usado con las personas con una enfermedad mental grave y prolongada

The impact of concept map visualizations on the information behavior, perceptions of performance, learning and use with novices in the information retrieval context

In examining undergraduate students in the information retrieval environment for the impact of computer generated concept maps, two primary research questions were considered: 1) what is the impact of display type on the novice searcher’s information behavior; and 2) what is the impact of different display types on the user’s perceptions of performance, knowledge and overall use of the system.Sixty participants in this experiment were given hypothetical information needs on two different medical topics (cholesterol, depression). Participants’ explored one of three interactive visualization displays using these medical topics, answered a pre- and post-test instrument and then completed a final questionnaire on their perceptions of the displays. Different types of inferential statistical tests were used to examine the research questions. When appropriate, factorial ANOVAs, mixed between-within ANOVAs, and chi square tests of independence were conducted.Five main findings resulted from this research: 1) for all display types (LIST, SOM, PFNET) there is an increase in the number of participant search terms and in the incorporation of MeSH terminology from the visualizations following exposure to those displays; 2) there is a relationship between the display type and the interface level from which PFNET participants chose terms; 3) searchers’ feelings of confidence, satisfaction, success, and relevance increased across all groups after system interaction; however, pretest feelings of confidence and satisfaction seem to be dependent upon the participant’s self-reported prior knowledge of the search topic; 4) feelings of confidence and satisfaction on the topic participants reported less pre-test knowledge on (cholesterol) shifted to match post-test ratings of confidence and satisfaction on the topic they had more pre-test knowledge on (depression); and 5) participants rated the PFNET system more visually appealing, easier to understand and more likely to be used in the future if given the option. Overall findings suggest that all displays were useful to the participants in this experiment and that the PFNET display was particularly useful for the novice searcher.Ph.D., Information Science -- Drexel University, 200