Report on the Information Retrieval Festival (IRFest2017)

The Information Retrieval Festival took place in April 2017 in Glasgow. The focus of the workshop was to bring together IR researchers from the various Scottish universities and beyond in order to facilitate more awareness, increased interaction and reflection on the status of the field and its future. The program included an industry session, research talks, demos and posters as well as two keynotes. The first keynote was delivered by Prof. Jaana Kekalenien, who provided a historical, critical reflection of realism in Interactive Information Retrieval Experimentation, while the second keynote was delivered by Prof. Maarten de Rijke, who argued for more Artificial Intelligence usage in IR solutions and deployments. The workshop was followed by a "Tour de Scotland" where delegates were taken from Glasgow to Aberdeen for the European Conference in Information Retrieval (ECIR 2017

A comparison of artificial driving sounds for automated vehicles

As automated vehicles currently do not provide sufficient feedback relating to the primary driving task, drivers have no assurance that an automated vehicle has understood and can cope with upcoming traffic situations [16]. To address this we conducted two user evaluations to investigate auditory displays in automated vehicles using different types of sound cues related to the primary driving sounds: acceleration, deceleration/braking, gear changing and indicating. Our first study compared earcons, speech and auditory icons with existing vehicle sounds. Our findings suggested that earcons were an effective alternative to existing vehicle sounds for presenting information related to the primary driving task. Based on these findings a second study was conducted to further investigate earcons modulated by different sonic parameters to present primary driving sounds. We discovered that earcons containing naturally mapped sonic parameters such as pitch and timbre were as effective as existing sounds in a simulated automated vehicle

Video test collection with graded relevance assessments

Relevance is a complex, but core, concept within the field of Information Retrieval. In order to allow system comparisons the many factors that influence relevance are often discarded to allow abstraction to a single score relating to relevance. This means that a great wealth of information is often discarded. In this paper we outline the creation of a video test collection with graded relevance assessments, to the best of our knowledge the first example of such a test collection for video retrieval. To directly address the shortcoming above we also gathered behavioural and perceptual data from assessors during the assessment process. All of this information along with judgements are available for download. Our intention is to allow other researchers to supplement the judgements to help create an adaptive test collection which contains supplementary information rather than a completely static collection with binary judgements

Ranking Heterogeneous Search Result Pages using the Interactive Probability Ranking Principle

The Probability Ranking Principle (PRP) ranks search results based on their expected utility derived solely from document contents, often overlooking the nuances of presentation and user interaction. However, with the evolution of Search Engine Result Pages (SERPs), now comprising a variety of result cards, the manner in which these results are presented is pivotal in influencing user engagement and satisfaction. This shift prompts the question: How does the PRP and its user-centric counterpart, the Interactive Probability Ranking Principle (iPRP), compare in the context of these heterogeneous SERPs? Our study draws a comparison between the PRP and the iPRP, revealing significant differences in their output. The iPRP, accounting for item-specific costs and interaction probabilities to determine the ``Expected Perceived Utility" (EPU), yields different result orderings compared to the PRP. We evaluate the effect of the EPU on the ordering of results by observing changes in the ranking within a heterogeneous SERP compared to the traditional ``ten blue links''. We find that changing the presentation affects the ranking of items according to the (iPRP) by up to 48\% (with respect to DCG, TBG and RBO) in ad-hoc search tasks on the TREC WaPo Collection. This work suggests that the iPRP should be employed when ranking heterogeneous SERPs to provide a user-centric ranking that adapts the ordering based on the presentation and user engagement.Comment: To be presented as a full paper at ECIR 2024 in Glasgow, U

Using thermal stimuli to influence affect in different picture display sizes

The ability of images to evoke emotions in people has been well documented in previous research, as well as the differences in the emotional perception of images when viewed on different-sized screens and device types. The ability of thermal stimuli to evoke emotions in people when used for media augmentation has also been examined. However little is known about how thermal stimuli can be used to enhance or reduce affect in images with varying emotional properties displayed in different sizes or on different devices. To the best of our knowledge no work has been conducted to investigate if there is any difference in the effect thermal augmentation has on images displayed in different sizes on different device types. This paper presents two user studies to address this research gap. Study 1 explored the effect thermal stimulation has on images displayed in different sizes. Images were displayed in sizes corresponding to the full screen display of a laptop, tablet and mobile phone. In study 2 we examined whether the actual presentation device (tablet and mobile) plays a role in the emotional perception of images displayed on mobile devices. Results showed that thermal augmentation was most effective in modulating emotions in small-sized pictures (427x240 pixels display size) and pictures displayed on a mobile phone. Thermal stimuli also reduced emotions in medium display sizes (corresponding to the full screen display of a tablet)

Exploring how drivers perceive spatial earcons in automated vehicles

Automated vehicles seek to relieve the human driver from primary driving tasks, but this substantially diminishes the connection between driver and vehicle compared to manual operation. At present, automated vehicles lack any form of continual, appropriate feedback to re-establish this connection and offer a feeling of control. We suggest that auditory feedback can be used to support the driver in this context. A preliminary field study that explored how drivers respond to existing auditory feedback in manual vehicles was first undertaken. We then designed a set of abstract, synthesised sounds presented spatially around the driver, known as Spatial Earcons, that represented different primary driving sounds e.g. acceleration. To evaluate their effectiveness, we undertook a driving simulator study in an outdoor setting using a real vehicle. Spatial Earcons performed as well as Existing Vehicle Sounds during automated and manual driving scenarios. Subjective responses suggested Spatial Earcons produced an engaging driving experience. This paper argues that entirely new synthesised primary driving sounds, such as Spatial Earcons, can be designed for automated vehicles to replace Existing Vehicle Sounds. This creates new possibilities for presenting primary driving information in automated vehicles using auditory feedback, in order to re-establish a connection between driver and vehicle

The Influence of Presentation and Performance on User Satisfaction

The effectiveness of an IR system is gauged not just by its ability to retrieve relevant results but also by how it presents these results to users; an engaging presentation often correlates with increased user satisfaction. While existing research has delved into the link between user satisfaction, IR performance metrics, and presentation, these aspects have typically been investigated in isolation. Our research aims to bridge this gap by examining the relationship between query performance, presentation and user satisfaction. For our analysis, we conducted a between-subjects experiment comparing the effectiveness of various result card layouts for an ad-hoc news search interface. Drawing data from the TREC WaPo 2018 collection, we centered our study on four specific topics. Within each of these topics, we assessed six distinct queries with varying nDCG values. Our study involved 164 participants who were exposed to one of five distinct layouts containing result cards, such as "title'', "title+image'', or "title+image+summary''. Our findings indicate that while nDCG is a strong predictor of user satisfaction at the query level, there exists no linear relationship between the performance of the query, presentation of results and user satisfaction. However, when considering the total gain on the initial result page, we observed that presentation does play a significant role in user satisfaction (at the query level) for certain layouts with result cards such as, title+image or title+image+summary. Our results also suggest that the layout differences have complex and multifaceted impacts on satisfaction. We demonstrate the capacity to equalize user satisfaction levels between queries of varying performance by changing how results are presented. This emphasizes the necessity to harmonize both performance and presentation in IR systems, considering users' diverse preferences.Comment: To appear as a full paper at CHIIR 2024, Sheffield, U

Using thermal stimuli to enhance photo-sharing in social media

Limited work has been undertaken to show how the emotive ability of thermal stimuli can be used for interaction purposes. One potential application area is using thermal stimuli to influence emotions in images shared online such as social media platforms. This paper presents a two-part study, which examines how the documented emotive property of thermal stimuli can be applied to enhance social media images. Participants in part-one supplied images from their personal collection or social media profiles, and were asked to augment each image with thermal stimuli based on the emotions they wanted to enhance or reduce. Part-one participants were interviewed to understand the effects they wanted augmented images to have. In part-two, these augmented images were perceived by a different set of participants in a simulated social media interface. Results showed strong agreement between the emotions augmented images were designed to evoke and the emotions they actually evoked as perceived by part-two participants. Participants in part-one selected thermal stimuli augmentation intended to modulate valence and arousal in images as a way of enhancing the realism of the images augmented. Part-two results indicate this was achieved as participants perceived thermal stimuli augmentation reduced valence in negative images and modulated valence and arousal in positive images