Report on the Information Retrieval Festival (IRFest2017)

The Information Retrieval Festival took place in April 2017 in Glasgow. The focus of the workshop was to bring together IR researchers from the various Scottish universities and beyond in order to facilitate more awareness, increased interaction and reflection on the status of the field and its future. The program included an industry session, research talks, demos and posters as well as two keynotes. The first keynote was delivered by Prof. Jaana Kekalenien, who provided a historical, critical reflection of realism in Interactive Information Retrieval Experimentation, while the second keynote was delivered by Prof. Maarten de Rijke, who argued for more Artificial Intelligence usage in IR solutions and deployments. The workshop was followed by a "Tour de Scotland" where delegates were taken from Glasgow to Aberdeen for the European Conference in Information Retrieval (ECIR 2017

Exploring how drivers perceive spatial earcons in automated vehicles

Automated vehicles seek to relieve the human driver from primary driving tasks, but this substantially diminishes the connection between driver and vehicle compared to manual operation. At present, automated vehicles lack any form of continual, appropriate feedback to re-establish this connection and offer a feeling of control. We suggest that auditory feedback can be used to support the driver in this context. A preliminary field study that explored how drivers respond to existing auditory feedback in manual vehicles was first undertaken. We then designed a set of abstract, synthesised sounds presented spatially around the driver, known as Spatial Earcons, that represented different primary driving sounds e.g. acceleration. To evaluate their effectiveness, we undertook a driving simulator study in an outdoor setting using a real vehicle. Spatial Earcons performed as well as Existing Vehicle Sounds during automated and manual driving scenarios. Subjective responses suggested Spatial Earcons produced an engaging driving experience. This paper argues that entirely new synthesised primary driving sounds, such as Spatial Earcons, can be designed for automated vehicles to replace Existing Vehicle Sounds. This creates new possibilities for presenting primary driving information in automated vehicles using auditory feedback, in order to re-establish a connection between driver and vehicle

Video test collection with graded relevance assessments

Relevance is a complex, but core, concept within the field of Information Retrieval. In order to allow system comparisons the many factors that influence relevance are often discarded to allow abstraction to a single score relating to relevance. This means that a great wealth of information is often discarded. In this paper we outline the creation of a video test collection with graded relevance assessments, to the best of our knowledge the first example of such a test collection for video retrieval. To directly address the shortcoming above we also gathered behavioural and perceptual data from assessors during the assessment process. All of this information along with judgements are available for download. Our intention is to allow other researchers to supplement the judgements to help create an adaptive test collection which contains supplementary information rather than a completely static collection with binary judgements

A comparison of artificial driving sounds for automated vehicles

As automated vehicles currently do not provide sufficient feedback relating to the primary driving task, drivers have no assurance that an automated vehicle has understood and can cope with upcoming traffic situations [16]. To address this we conducted two user evaluations to investigate auditory displays in automated vehicles using different types of sound cues related to the primary driving sounds: acceleration, deceleration/braking, gear changing and indicating. Our first study compared earcons, speech and auditory icons with existing vehicle sounds. Our findings suggested that earcons were an effective alternative to existing vehicle sounds for presenting information related to the primary driving task. Based on these findings a second study was conducted to further investigate earcons modulated by different sonic parameters to present primary driving sounds. We discovered that earcons containing naturally mapped sonic parameters such as pitch and timbre were as effective as existing sounds in a simulated automated vehicle

Towards quantifying the impact of non-uniform information access in collaborative information retrieval

The majority of research into Collaborative Information Retrieval (CIR) has assumed a uniformity of information access and visibility between collaborators. However in a number of real world scenarios, information access is not uniform between all collaborators in a team e.g. security, health etc. This can be referred to as Multi-Level Collaborative Information Retrieval (MLCIR). To the best of our knowledge, there has not yet been any systematic investigation of the effect of MLCIR on search outcomes. To address this shortcoming, in this paper, we present the results of a simulated evaluation conducted over 4 different non-uniform information access scenarios and 3 different collaborative search strategies. Results indicate that there is some tolerance to removing access to the collection and that there may not always be a negative impact on performance. We also highlight how different access scenarios and search strategies impact on search outcomes

The effect of thermal stimuli on the emotional perception of images

Thermal stimulation is a feedback channel that has the potential to influence the emotional response of people to media such as images. While previous work has demonstrated that thermal stimuli might have an effect on the emotional perception of images, little is understood about the exact emotional responses different thermal properties and presentation techniques can elicit towards images. This paper presents two user studies that investigate the effect thermal stimuli parameters (e.g. intensity) and timing of thermal stimuli presentation have on the emotional perception of images. We found that thermal stimulation increased valence and arousal in images with low valence and neutral to low arousal. Thermal augmentation of images also reduced valence and arousal in high valence and arousal images. We discovered that depending on when thermal augmentation is presented, it can either be used to create anticipation or enhance the inherent emotion an image is capable of evoking

Beyond actions : exploring the discovery of tactics from user logs

Search log analysis has become a common practice to gain insights into user search behaviour; it helps gain an understanding of user needs and preferences, as well as an insight into how well a system supports such needs. Currently, log analysis is typically focused on low-level user actions, i.e. logged events such as issued queries and clicked results, and often only a selection of such events are logged and analysed. However, types of logged events may differ widely from interface to interface, making comparison between systems difficult. Further, the interpretation of the meaning of and subsequent analysis of a selection of events may lead to conclusions out of context—e.g. the statistics of observed query reformulations may be influenced by the existence of a relevance feedback component. Alternatively, in lab studies user activities can be analysed at a higher level, such as search tactics and strategies, abstracted away from detailed interface implementation. Unfortunately, until now the required manual codings that map logged events to higher-level interpretations have prevented large-scale use of this type of analysis. In this paper, we propose a new method for analysing search logs by (semi-)automatically identifying user search tactics from logged events, allowing large-scale analysis that is comparable across search systems. In addition, as the resulting analysis is at a tactical level we reduce potential issues surrounding the need for interpretation of low-level user actions for log analysis. We validate the efficiency and effectiveness of the proposed tactic identification method using logs of two reference search systems of different natures: a product search system and a video search system. With the identified tactics, we perform a series of novel log analyses in terms of entropy rate of user search tactic sequences, demonstrating how this type of analysis allows comparisons of user search behaviours across systems of different nature and design. This analysis provides insights not achievable with traditional log analysis

A comparison of primary and secondary relevance judgements for real-life topics

The notion of relevance is fundamental to the field of Information Retrieval. Within the field a generally accepted conception of relevance as inherently subjective has emerged, with an individual’s assessment of relevance influenced by numerous contextual factors. In this paper we present a user study that examines in detail the differences between primary and secondary assessors on a set of “real-world” topics which were gathered specifically for the work. By gathering topics which are representative of the staff and students at a major university, at a particular point in time, we aim to explore differences between primary and secondary relevance judgements for real-life search tasks. Findings suggest that while secondary assessors may find the assessment task challenging in various ways (they generally possess less interest and knowledge in secondary topics and take longer to assess documents), agreement between primary and secondary assessors is high

Using thermal stimuli to enhance photo-sharing in social media

Limited work has been undertaken to show how the emotive ability of thermal stimuli can be used for interaction purposes. One potential application area is using thermal stimuli to influence emotions in images shared online such as social media platforms. This paper presents a two-part study, which examines how the documented emotive property of thermal stimuli can be applied to enhance social media images. Participants in part-one supplied images from their personal collection or social media profiles, and were asked to augment each image with thermal stimuli based on the emotions they wanted to enhance or reduce. Part-one participants were interviewed to understand the effects they wanted augmented images to have. In part-two, these augmented images were perceived by a different set of participants in a simulated social media interface. Results showed strong agreement between the emotions augmented images were designed to evoke and the emotions they actually evoked as perceived by part-two participants. Participants in part-one selected thermal stimuli augmentation intended to modulate valence and arousal in images as a way of enhancing the realism of the images augmented. Part-two results indicate this was achieved as participants perceived thermal stimuli augmentation reduced valence in negative images and modulated valence and arousal in positive images