The effect of dyslexia on information retrieval: A pilot study

Purpose – The purpose of the paper is to resolve a gap in our knowledge of how people with dyslexia interact with Information Retrieval (IR) systems, specifically an understanding of their information searching behaviour. Very little research has been undertaken with this particular user group, and given the size of the group (an estimated 10% of the population) this lack of knowledge needs to be addressed. Design/Methodology/Approach - We use elements of the dyslexia cognitive profile to design a logging system recording the difference between two sets of participants: dyslexic and control users. We use a standard Okapi interface together with two standard TREC topics in order to record the information searching behaviour of these users. We gather evidence from various sources, including quantitative information on search logs, together with qualitative information from interviews and questionnaires. We record variables on queries, documents, relevance assessments and sessions in the search logs. We use this evidence to examine the difference in searching between the two sets of users, in order to understand the effect of dyslexia on the information searching behaviour. A topic analysis is also conducted on the quantitative data to show any effect on the results from the information need. Research limitations/implications – As this is a pilot study, only 10 participants were recruited for the study, 5 for each user group. Due to ethical issues, the number of topics per search was restricted to one topic only. The study shows that the methodology applied is useful for distinguishing between the two user groups, taking into account differences between topic. We outline further research on the back of this pilot study in four main areas. A different approach from the proposed methodology is needed to measure the effect on query variables, which takes account of topic variation. More details on users are needed such as reading abilities, speed of language processing and working memory to distinguish the user groups. Effect of topic on search interaction must be measured in order to record the potential impact on the dyslexic user group. Work is needed on relevance assessment and effect on precision and recall for users who may not read many documents. Findings – Using the log data, we establish the differences in information searching behaviour of control and dyslexic users i.e. in the way the two groups interact with Okapi, and that qualitative information collected (such as experience etc) may not be able to account for these differences. Evidence from query variables was unable to distinguish between groups, but differences on topic for the same variables were recorded. Users who view more documents tended to judge more documents as being relevant, either in terms of the user group or topic. Session data indicated that there may be an important difference between the number of iterations used in a search between the user groups, as there may be little effect from the topic on this variable. Originality/Value – This is the first study of the effect of dyslexia on information search behaviour, and provides some evidence to take the field forward

Measuring children's search behaviour on a large scale

Children often experience problems during information-seeking using traditional search interfaces and search technologies, that are designed for adults. This is because children engage with the world in fundamentally different ways than adults. To design search technologies that support children in effective and enjoyable information-seeking, more research is needed to examine children’s specific skills and needs concerning information-seeking. Therefore, we developed an application that can monitor children’s search behaviour on a large scale. In this paper, we present the steps taken to develop this application. The basis of the application is UsaProxy, an existing system that is used to monitor the user’s usage of websites. We have increased the accuracy of UsaProxy and have developed an application that is able to extract useful information from UsaProxy’s log files

Experimental Methods in IIR: The Tension between Rigour and Ethics in Studies Involving Users with Dyslexia

Designing user studies in the interactive information retrieval (IIR) paradigm on people with impairments may sometimes require different methodological considerations than for other users. Consequently, there may be a tension between what the community regards as being a rigorous methodology against what researchers can do ethically with their users. This paper discusses issues to consider when designing IIR studies involving people with dyslexia, such as sampling, informed consent and data collection. The conclusion is that conducting user studies on participants with dyslexia requires special considerations at all stages of the experimental design. The purpose of this paper is to raise awareness and understanding in the research community about experimental methods involving users with dyslexia, and addresses researchers, as well as editors and reviewers. Several of the issues raised do not only apply to people with dyslexia, but have implications when researching other groups, for instance elderly people and users with learning, cognitive, sensory or motor impairments

A Study of Snippet Length and Informativeness: Behaviour, Performance and User Experience

The design and presentation of a Search Engine Results Page (SERP) has been subject to much research. With many contemporary aspects of the SERP now under scrutiny, work still remains in investigating more traditional SERP components, such as the result summary. Prior studies have examined a variety of different aspects of result summaries, but in this paper we investigate the influence of result summary length on search behaviour, performance and user experience. To this end, we designed and conducted a within-subjects experiment using the TREC AQUAINT news collection with 53 participants. Using Kullback-Leibler distance as a measure of information gain, we examined result summaries of different lengths and selected four conditions where the change in information gain was the greatest: (i) title only; (ii) title plus one snippet; (iii) title plus two snippets; and (iv) title plus four snippets. Findings show that participants broadly preferred longer result summaries, as they were perceived to be more informative. However, their performance in terms of correctly identifying relevant documents was similar across all four conditions. Furthermore, while the participants felt that longer summaries were more informative, empirical observations suggest otherwise; while participants were more likely to click on relevant items given longer summaries, they also were more likely to click on non-relevant items. This shows that longer is not necessarily better, though participants perceived that to be the case - and second, they reveal a positive relationship between the length and informativeness of summaries and their attractiveness (i.e. clickthrough rates). These findings show that there are tensions between perception and performance when designing result summaries that need to be taken into account

Web navigation for individuals with dyslexia: An exploratory study

In this paper, we present an exploratory study of the web navigation experiences of dyslexic users. Findings indicate that dyslexics exhibit distinctive web navigation behaviour and preferences. We believe that the outcomes of this study add to our understanding of the particular needs of this web user population and have implications for the design of effective navigation structures

Evaluating usability of e-commerce sites by tracking eye movements

The majority of existing e-commerce design guidelines has been derived by conducting heuristic evaluations, without reporting the involvement of the users themselves. This research provides clarification on a number of existing web design guidelines for e-commerce sites based on empirical studies with users. Four studies were conducted and each study focused on a specific set of design guidelines as found in the literature. A combined qualitative and quantitative approach has been used, including a state-of-the-art technique, eye tracking. The eye movement data were complemented by user-profile data elicited through background questionnaires and user-perception data as captured through semi-structured interviews. The first study investigated users’ initial impressions of homepages of e-commerce sites. The second study examined users’ adaptability to persistent or varied placement of design elements. The third and fourth studies explored the effect of the presentation format of e-commerce web pages: the first in terms of the proportion of images, and the second in terms of how key icons related to an e-commerce transaction were presented. On the whole, the results of the studies corroborated existing design guidelines, but they also identified potential refinements. The thesis contributes both methodologically and empirically to Human-Computer Interaction. The combined methodological approach enables insight into the user experience that spans behavioural aspects such as visual search behaviour and visual search performance data, and subjective aspects such as user expectations and preferences. The empirical outcomes amplify the design guidelines from a user’s perspective

Collocating Interface Objects: Zooming into Maps

May, Dean and Barnard [10] used a theoretically based model to argue that objects in a wide range of interfaces should be collocated following screen changes such as a zoom-in to detail. Many existing online maps do not follow this principle, but move a clicked point to the centre of the subsequent display, leaving the user looking at an unrelated location. This paper presents three experiments showing that collocating the point clicked on a map so that the detailed location appears in the place previously occupied by the overview location makes the map easier to use, reducing eye movements and interaction duration. We discuss the benefit of basing design principles on theoretical models so that they can be applied to novel situations, and so designers can infer when to use and not use them