1,761 research outputs found
Report on the Information Retrieval Festival (IRFest2017)
The Information Retrieval Festival took place in April 2017 in Glasgow. The focus of the workshop was to bring together IR researchers from the various Scottish universities and beyond in order to facilitate more awareness, increased interaction and reflection on the status of the field and its future. The program included an industry session, research talks, demos and posters as well as two keynotes. The first keynote was delivered by Prof. Jaana Kekalenien, who provided a historical, critical reflection of realism in Interactive Information Retrieval Experimentation, while the second keynote was delivered by Prof. Maarten de Rijke, who argued for more Artificial Intelligence usage in IR solutions and deployments. The workshop was followed by a "Tour de Scotland" where delegates were taken from Glasgow to Aberdeen for the European Conference in Information Retrieval (ECIR 2017
Abstracts and Abstracting in Knowledge Discovery
published or submitted for publicatio
Monitoring and the Risk Governance of Repository Development and Staged Closure:Exploratory Engagement Activity in Three European Countries.
This report is the product of research activity within the EC Seventh Framework Programme “Monitoring Developments for Safe Repository Operation and Staged Closure” (MoDeRn) Project. This project aims to further develop understanding of the role of monitoring in staged implementation of geological disposal to a level of description that is closer to the actual implementation of monitoring. It focuses on monitoring conducted to confirm the basis of the long term safety case and on monitoring conducted to inform on options available to manage the stepwise disposal process from construction to closure (including e.g. the option of waste retrieval). This report investigates the potential of citizen stakeholder engagement in the identification of monitoring objectives and the development of monitoring strategies for geological disposal of high level waste (HLW) or spent nuclear fuel (SNF). It builds on an earlier MoDeRn report describing monitoring the safe disposal of radioactive waste as a socio-technical activity (Bergmans, Elam, Simmons and Sundqvist 2012)
A topical approach to retrievability bias estimation
Retrievability is an independent evaluation measure that offers insights to an aspect of retrieval systems that performance and efficiency measures do not. Retrievability is often used to calculate the retrievability bias, an indication of how accessible a system makes all the documents in a collection. Generally, computing the retrievability bias of a system requires a colossal number of queries to be issued for the system to gain an accurate estimate of the bias. However, it is often the case that the accuracy of the estimate is not of importance, but the relationship between the estimate of bias and performance when tuning a systems parameters. As such, reaching a stable estimation of bias for the system is more important than getting very accurate retrievability scores for individual documents. This work explores the idea of using topical subsets of the collection for query generation and bias estimation to form a local estimate of bias which correlates with the global estimate of retrievability bias. By using topical subsets, it would be possible to reduce the volume of queries required to reach an accurate estimate of retrievability bias, reducing the time and resources required to perform a retrievability analysis. Findings suggest that this is a viable approach to estimating retrievability bias and that the number of queries required can be reduced to less than a quarter of what was previously thought necessary
An empirical analysis of pruning techniques performance, retrievability and bias
Prior work on using retrievability measures in the evaluation of information retrieval (IR) systems has laid out the foundations for investigating the relation between retrieval performance and retrieval bias. While various factors influencing retrievability have been examined, showing how the retrieval model may influence bias, no prior work has examined the impact of the index (and how it is optimized) on retrieval bias. Intuitively, how the documents are represented, and what terms they contain, will influence whether they are retrievable or not. In this paper, we investigate how the retrieval bias of a system changes as the inverted index is optimized for efficiency through static index pruning. In our analysis, we consider four pruning methods and examine how they affect performance and bias on the TREC GOV2 Collection. Our results show that the relationship between these factors is varied and complex-and very much dependent on the pruning algorithm. We find that more pruning results in relatively little change or a slight decrease in bias up to a point, and then a dramatic increase. The increase in bias corresponds to a sharp decrease in early precision such as NDCG@10 and is also indicative of a large decrease in MAP. The findings suggest that the impact of pruning algorithms can be quite varied-but retrieval bias could be used to guide the pruning process. Further work is required to determine precisely which documents are most affected and how this impacts upon performance
Evaluation of information retrieval systems using structural equation modeling
The interpretation of the experimental data collected by testing systems across input datasets and model parameters is of strategic importance for system design and implementation. In particular, finding relationships between variables and detecting the latent variables affecting retrieval performance can provide designers, engineers and experimenters with useful if not necessary information about how a system is performing. This paper discusses the use of Structural Equation Modeling (SEM) in providing an in-depth explanation of evaluation results and an explanation of failures and successes of a system; in particular, we focus on the case of evaluation of Information Retrieval systems
Monitoring the Safe Disposal of Radioactive Waste: a Combined Technical and Socio-Political Activity
Abstract Images Have Different Levels of Retrievability Per Reverse Image Search Engine
Much computer vision research has focused on natural images, but technical
documents typically consist of abstract images, such as charts, drawings,
diagrams, and schematics. How well do general web search engines discover
abstract images? Recent advancements in computer vision and machine learning
have led to the rise of reverse image search engines. Where conventional search
engines accept a text query and return a set of document results, including
images, a reverse image search accepts an image as a query and returns a set of
images as results. This paper evaluates how well common reverse image search
engines discover abstract images. We conducted an experiment leveraging images
from Wikimedia Commons, a website known to be well indexed by Baidu, Bing,
Google, and Yandex. We measure how difficult an image is to find again
(retrievability), what percentage of images returned are relevant (precision),
and the average number of results a visitor must review before finding the
submitted image (mean reciprocal rank). When trying to discover the same image
again among similar images, Yandex performs best. When searching for pages
containing a specific image, Google and Yandex outperform the others when
discovering photographs with precision scores ranging from 0.8191 to 0.8297,
respectively. In both of these cases, Google and Yandex perform better with
natural images than with abstract ones achieving a difference in retrievability
as high as 54\% between images in these categories. These results affect anyone
applying common web search engines to search for technical documents that use
abstract images.Comment: 20 pages; 7 figures; to be published in the proceedings of the
Drawings and abstract Imagery: Representation and Analysis (DIRA) Workshop
from ECCV 202
- …