Experimental Methods in IIR: The Tension between Rigour and Ethics in Studies Involving Users with Dyslexia

Designing user studies in the interactive information retrieval (IIR) paradigm on people with impairments may sometimes require different methodological considerations than for other users. Consequently, there may be a tension between what the community regards as being a rigorous methodology against what researchers can do ethically with their users. This paper discusses issues to consider when designing IIR studies involving people with dyslexia, such as sampling, informed consent and data collection. The conclusion is that conducting user studies on participants with dyslexia requires special considerations at all stages of the experimental design. The purpose of this paper is to raise awareness and understanding in the research community about experimental methods involving users with dyslexia, and addresses researchers, as well as editors and reviewers. Several of the issues raised do not only apply to people with dyslexia, but have implications when researching other groups, for instance elderly people and users with learning, cognitive, sensory or motor impairments

Children’s information retrieval: beyond examining search strategies and interfaces

The study of children’s information retrieval is still for the greater part untouched territory. Meanwhile, children can become lost in the digital information world, because they are confronted with search interfaces, both designed by and for adults. Most current research on children’s information retrieval focuses on examining children’s search performance on existing search interfaces to determine what kind of interfaces are suitable for children’s search behaviour. However, to discover the true nature of children’s search behaviour, we state that research has to go beyond examining search strategies used with existing search interfaces by examining children’s cognitive processes during information-seeking. A paradigm of children’s information retrieval should provide an overview of all the components beyond search interfaces and search strategies that are part of children’s information retrieval process. Better understanding of the nature of children’s search behaviour can help adults design interfaces and information retrieval systems that both support children’s natural search strategies and help them find their way in the digital information world

The effect of dyslexia on information retrieval: A pilot study

Purpose – The purpose of the paper is to resolve a gap in our knowledge of how people with dyslexia interact with Information Retrieval (IR) systems, specifically an understanding of their information searching behaviour. Very little research has been undertaken with this particular user group, and given the size of the group (an estimated 10% of the population) this lack of knowledge needs to be addressed. Design/Methodology/Approach - We use elements of the dyslexia cognitive profile to design a logging system recording the difference between two sets of participants: dyslexic and control users. We use a standard Okapi interface together with two standard TREC topics in order to record the information searching behaviour of these users. We gather evidence from various sources, including quantitative information on search logs, together with qualitative information from interviews and questionnaires. We record variables on queries, documents, relevance assessments and sessions in the search logs. We use this evidence to examine the difference in searching between the two sets of users, in order to understand the effect of dyslexia on the information searching behaviour. A topic analysis is also conducted on the quantitative data to show any effect on the results from the information need. Research limitations/implications – As this is a pilot study, only 10 participants were recruited for the study, 5 for each user group. Due to ethical issues, the number of topics per search was restricted to one topic only. The study shows that the methodology applied is useful for distinguishing between the two user groups, taking into account differences between topic. We outline further research on the back of this pilot study in four main areas. A different approach from the proposed methodology is needed to measure the effect on query variables, which takes account of topic variation. More details on users are needed such as reading abilities, speed of language processing and working memory to distinguish the user groups. Effect of topic on search interaction must be measured in order to record the potential impact on the dyslexic user group. Work is needed on relevance assessment and effect on precision and recall for users who may not read many documents. Findings – Using the log data, we establish the differences in information searching behaviour of control and dyslexic users i.e. in the way the two groups interact with Okapi, and that qualitative information collected (such as experience etc) may not be able to account for these differences. Evidence from query variables was unable to distinguish between groups, but differences on topic for the same variables were recorded. Users who view more documents tended to judge more documents as being relevant, either in terms of the user group or topic. Session data indicated that there may be an important difference between the number of iterations used in a search between the user groups, as there may be little effect from the topic on this variable. Originality/Value – This is the first study of the effect of dyslexia on information search behaviour, and provides some evidence to take the field forward

Measuring cognitive load and cognition: metrics for technology-enhanced learning

This critical and reflective literature review examines international research published over the last decade to summarise the different kinds of measures that have been used to explore cognitive load and critiques the strengths and limitations of those focussed on the development of direct empirical approaches. Over the last 40 years, cognitive load theory has become established as one of the most successful and influential theoretical explanations of cognitive processing during learning. Despite this success, attempts to obtain direct objective measures of the theory's central theoretical construct – cognitive load – have proved elusive. This obstacle represents the most significant outstanding challenge for successfully embedding the theoretical and experimental work on cognitive load in empirical data from authentic learning situations. Progress to date on the theoretical and practical approaches to cognitive load are discussed along with the influences of individual differences on cognitive load in order to assess the prospects for the development and application of direct empirical measures of cognitive load especially in technology-rich contexts

Absence of spontaneous action anticipation by false belief attribution in children with autism spectrum disorder

Recently, a series of studies demonstrated false belief understanding in young children through completely nonverbal measures. These studies have revealed that children younger than 3 years of age, who consistently fail the standard verbal false belief test, can anticipate others' actions based on their attributed false beliefs. The current study examined whether children with autism spectrum disorder (ASD), who are known to have difficulties in the verbal false belief test, may also show such action anticipation in a nonverbal false belief test. We presented video stimuli of an actor watching an object being hidden in a box. The object was then displaced while the actor was looking away. We recorded children's eye movements and coded whether they spontaneously anticipated the actor's subsequent behavior, which could only have been predicted if they had attributed a false belief to her. Although typically developing children correctly anticipated the action, children with ASD failed to show such action anticipation. The results suggest that children with ASD have an impairment in false belief attribution, which is independent of their verbal ability

An athletic approach to studying perception-action integration: Does sport-specific training, and the impact of injury, influence how individuals visually guide navigation?

The objective of this thesis was to investigate perception-action integration capabilities of individuals during a choice navigation task. This task assessed navigation strategies in open space while individuals avoided colliding with two vertical obstacles that created a body-scaled, horizontal gap, at three varying obstacle distances from the starting location (3m, 5m, 7m). The two studies completed in this thesis employed the same paradigm to assess the hypothesized group differences. Gaze behaviours and kinematics of navigation strategies were compared between: 1) athletes specifically trained in navigating in open space versus non-athletes; and 2) athletes with post-concussion syndrome (PCS) versus non-concussed, specifically trained athletes. Specifically trained athletes have been identified as demonstrating more successful perception-action integration in discrete motor tasks related to their sport (Mann et al., 2007; Vickers, 2007). However, whether these abilities translate to the continuous motor task of obstacle avoidance in open space was unknown. The purpose of Study 1 was to identify the influence of sport-specific training on navigating in open space (i.e. navigational strategies of large field sport athletes) compared to age-matched, non-athletes. It was hypothesized that specifically-trained athletes would demonstrate fewer, longer fixations, suggesting a more successful perception-action integration strategy (as defined by Mann et al., 2007), and would employ more sport-specific navigation strategies than non-athletes by maintaining their straight trajectory toward the goal (Fajen & Warren, 2003). Athletes were found to make fewer, longer fixations than non-athletes. However, no differences were observed between navigation strategies of the two groups, nor were any kinematic measures found to differ between groups. It can be concluded that athletes and non-athletes differentially obtain visual information to perform the same actions, suggesting that athletes and non-athletes differentially perform perception-action integration when navigating in open space. Future studies are required to identify sport-specific nuances of navigation (moving obstacles, running) to better identify athletic-related navigation strategies. Although athletic training can enhance perception-action integration strategies, sport-related injuries can hinder this process. Following a concussion, individuals experience deficits of perception-action integration that persist well beyond 30 days of recovery, post-concussion (Baker and Cinelli, 2014; Slobounov et al., 2006). These perception-action integration deficits may also exist in individual with postconcussion syndrome (PCS). The purpose of the Study 2 was to identify whether perception-action integration deficits persist with the persistent physical symptoms of concussion characteristic of PCS. The current study revealed that athletes with PCS did not differ from non-concussed athletes on any measure of visual fixation strategy, nor were they found to differ on any kinematic measure assessed. These findings suggest that in the context of the current paradigm, athletes with PCS have no perception-action integration deficit. In that, athletes with PCS may have adapted perception-action integration strategies to navigate with equal efficiency as a specifically-trained group of athletes or that the paradigm was not sensitive enough to identify these differences. Such findings suggest that more research is required to assess what, if any, perception-action integration deficits persist with persisting physical symptoms of PCS to better benefit rehabilitative procedures and outcomes for these individuals. Together, these studies add to what was previously known about perception-action integration, as it relates to navigation. Both studies assessed perception-action integration in unique populations that add to understanding of behavioural dynamics in the sport setting. Study 1 builds on a line of research assessing affordance theory and behavioural dynamics in sport (Fajen, Riley, & Turvey, 2008). The findings of this study suggest that although navigation strategies did not differ between specifically trained athletes and non-athletes, visual search strategies employed in task did. Such findings add to the understanding that sport-specific training influences perception-action integration, through our understanding of how athletes obtain visual information to perform actions. This thesis did not identify perception-action integration deficits in athletes with PCS. These findings suggest that the individuals in the present study likely adapted to their injury as they demonstrated equal ability in gaze and navigation strategies to specifically-trained athletes. As such, further research is required to assess the cognitive, motor, and sensory-motor deficits that may persist with the persisting physical symptoms of PCS. As individuals with PCS do not demonstrate similar visuomotor integration deficits as individuals with acute concussions (Baker & Cinelli, 2014), such individuals must be assessed and researched as a separate population

Expertise in chess

This chapter provides an overview of research into chess expertise. After an historical background and a brief description of the game and the rating system, it discusses the information processes enabling players to choose good moves, and in particular the trade-offs between knowledge and search. Other topics include blindfold chess, talent, and the role of deliberate practice and tournament experience

Trail Making Test performance contributes to subjective judgment of visual efficiency in older adults

Introduction: The determinant factors that influence self-reported quality of vision have yet to be fully elucidated. This study evaluated a range of contextual information, established psychophysical tests, and in particular, a series of cognitive tests as potentially novel determinant factors.   Materials & Methods: Community dwelling adults (aged 50+) recruited to Wave 1 of The Irish Longitudinal Study on Ageing, excluding those registered blind, participated in this study (N = 5,021). Self-reports of vision were analysed in relation to visual acuity and contrast sensitivity, ocular pathology, visual (Choice Response Time task; Trail Making Test) and global cognition. Contextual factors such as having visited an optometrist and wearing glasses were also considered. Ordinal logistic regression was used to determine univariate and multivariate associations.   Results and Discussion: Poor Trail Making Test performance (Odds ratio, OR = 1.36), visual acuity (OR = 1.72) and ocular pathology (OR = 2.25) were determinant factors for poor versus excellent vision in self-reports. Education, wealth, age, depressive symptoms and general cognitive fitness also contributed to determining self-reported vision.   Conclusions: Trail Making Test contribution to self-reports may capture higher level visual processing and should be considered when using self-reports to assess vision and its role in cognitive and functional health

Applying psychological science to the CCTV review process: a review of cognitive and ergonomic literature

As CCTV cameras are used more and more often to increase security in communities, police are spending a larger proportion of their resources, including time, in processing CCTV images when investigating crimes that have occurred (Levesley & Martin, 2005; Nichols, 2001). As with all tasks, there are ways to approach this task that will facilitate performance and other approaches that will degrade performance, either by increasing errors or by unnecessarily prolonging the process. A clearer understanding of psychological factors influencing the effectiveness of footage review will facilitate future training in best practice with respect to the review of CCTV footage. The goal of this report is to provide such understanding by reviewing research on footage review, research on related tasks that require similar skills, and experimental laboratory research about the cognitive skills underpinning the task. The report is organised to address five challenges to effectiveness of CCTV review: the effects of the degraded nature of CCTV footage, distractions and interrupts, the length of the task, inappropriate mindset, and variability in people’s abilities and experience. Recommendations for optimising CCTV footage review include (1) doing a cognitive task analysis to increase understanding of the ways in which performance might be limited, (2) exploiting technology advances to maximise the perceptual quality of the footage (3) training people to improve the flexibility of their mindset as they perceive and interpret the images seen, (4) monitoring performance either on an ongoing basis, by using psychophysiological measures of alertness, or periodically, by testing screeners’ ability to find evidence in footage developed for such testing, and (5) evaluating the relevance of possible selection tests to screen effective from ineffective screener

Understanding Attention and Intermodal Processing Difficulties in Children with Autism Spectrum Disorders

Problems with social orienting, attention, and intermodal processing (IMP) involving social events are considered to be important deficits in children with autism spectrum disorder (ASD). The present study aimed to better understand how difficulties in attention are associated with IMP difficulties in children with ASD using a novel eye-tracking experimental task. Six- to 16-year- old children with ASD were compared to age- and cognitive ability- matched peers while engaging in a task involving intermodal processing and attentional experimental measures. Overall, results showed that attention may not be uniformly impaired in ASD in relation to intermodal processing abilities. However, for typically developing children, shifting attention performance and intermodal processing may be linked. Explaining the attentional differences in relation to intermodal processing difficulties will contribute to the field of ASD research in order to inform early interventionists, researchers, and clinicians in treating social deficits in this population