Using Document-Quality Measures to Predict Web-Search Effectiveness

Abstract. The query-performance prediction task is estimating retrieval effectiveness in the absence of relevance judgments. The task becomes highly challenging over the Web due to, among other reasons, the effect of low quality (e.g., spam) documents on retrieval performance. To address this challenge, we present a novel prediction approach that utilizes queryindependent document-quality measures. While using these measures was shown to improve Web-retrieval effectiveness, this is the first study demonstrating the clear merits of using them for query-performance prediction. Evaluation performed with large scale Web collections shows that our methods post prediction quality that often surpasses that of state-of-the-art predictors, including those devised specifically for Web retrieval

La situación del patrimonio arqueológico subacuático en la cuenca extremeña del Tajo. Perspectivas de conservación, documentación y análisis

Trabajo presentado al I Congreso de Arqueología Náutica y Subacuática Española (ArNSe), sesión "Yacimientos en aguas continentales", celebrado en Cartagena (Murcia) del 14 al 16 de marzo de 2013.[ES]: Durante las décadas de 1960 y 1970 se acometieron obras de ingeniería hidráulica para la producción de energía eléctrica en el Tajo, lo que provocó que se sumergieran áreas con alto potencial arqueológico. Por su época de realización, apenas contaron con medidas correctoras de impacto, quedando sin realizar una documentación sistemática. En la actualidad hay cierta conciencia de la necesidad de intervenir sobre este patrimonio subacuático de aguas continentales, lo que debe generar un debate sobre cómo afrontar su conservación. La legislación no refleja la especificidad del patrimonio subacuático en aguas continentales, por lo que la consideración que reciben estos sitios es la habitual para la arqueología terrestre convencional. Parece evidente la necesidad de desarrollar técnicas para maximizar la recogida de información sobre los sitios arqueológicos y su entorno y al mismo tiempo, deben ser útiles para estudiar y modelar los agentes que provocan su deterioro.[EN]: During the 1960 and 1970 decades, several engineering works were performed along the Tagus River for producing hydroelectric energy; consequently areas with high potential archaeological were flooded. Since the heritage preservation polices were in an early stage, the works didn’t count with impact evaluation assesses, and a systematic documentation of heritage was never performed. Nowadays, there is awareness for engaging in activities directed to the underwater heritage that lies in continental waters, which should lead to a discussion about how to face its preservation. Current laws do not reflect the status of this specific underwater heritage, consequently, the sites are considered as conventional terrestrial archaeological heritage. It seems evident that is necessary to develop techniques to boost the gathering of information on the sites and their environments, which should be also dedicated to analyse and model the agents that aggravate its deterioration.Peer Reviewe

Information Needs, Queries, and Query Performance Prediction

The query performance prediction (QPP) task is to estimate the effectiveness of a search performed in response to a query with no relevance judgments. Existing QPP methods do not account for the effectiveness of a query in representing the underlying information need. We demonstrate the far-reaching implications of this reality using standard TREC-based evaluation of QPP methods: their relative prediction quality patterns vary with respect to the effectiveness of queries used to represent the information needs. Motivated by our findings, we revise the basic probabilistic formulation of the QPP task by accounting for the information need and its connection to the query. We further explore this connection by proposing a novel QPP approach that utilizes information about a set of queries representing the same information need. Predictors instantiated from our approach using a wide variety of existing QPP methods post prediction quality that substantially transcends that of applying these methods, as is standard, using a single query representing the information need. Additional in-depth empirical analysis of different aspects of our approach further attests to the crucial role of query effectiveness in QPP

IRIT-QFR: IRIT Query Feature Resource

International audienceIn this paper, we present a resource that consists of query features associated with TREC adhoc collections. We developed two types of query features: linguistics features that can be calculated from the query itself, prior to any search although some are collection-dependent and post-retrieval features that imply the query has been evaluated over the target collection. This paper presents the two types of features that we have estimated as well as their variants, and the resource produced. The total number of features with their variants that we have estimated is 258 where the number of pre-retrieval and post-retrieval features are 81 and 171, respectively. We also present the first analysis of this data that shows that some features are more relevant than others in IR applications. Finally, we present a few applications in which these resources could be used although the idea of making them available is to foster new usages for IR

Predicting the size of candidate document set for implicit web search result diversification

© Springer Nature Switzerland AG 2020.Implicit result diversification methods exploit the content of the documents in the candidate set, i.e., the initial retrieval results of a query, to obtain a relevant and diverse ranking. As our first contribution, we explore whether recently introduced word embeddings can be exploited for representing documents to improve diversification, and show a positive result. As a second improvement, we propose to automatically predict the size of candidate set on per query basis. Experimental evaluations using our BM25 runs as well as the best-performing ad hoc runs submitted to TREC (2009–2012) show that our approach improves the performance of implicit diversification up to 5.4% wrt. initial ranking

Navigating the user query space

Query performance prediction (QPP) aims to automatically estimate the performance of a query. Recently there have been many attempts to use these predictors to estimate whether a perturbed version of a query will outperform the original version. In essence, these approaches attempt to navigate the space of queries in a guided manner. In this paper, we perform an analysis of the query space over a substantial number of queries and show that (1) users tend to be able to extract queries that perform in the top 5% of all possible user queries for a specific topic, (2) that post-retrieval predictors outperform pre-retrieval predictors at the high end of the query space. And, finally (3), we show that some post retrieval predictors are better able to select high performing queries from a group of user queries for the same topic

Predicting the Performance of Recommender Systems: An Information Theoretic Approach

Proceedings of Third International Conference, ICTIR 2011, Bertinoro, Italy, September 12-14, 2011.The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-23318-0_5Performance prediction is an appealing problem in Recommender Systems, as it enables an array of strategies for deciding when to deliver or hold back recommendations based on their foreseen accuracy. The problem, however, has been barely addressed explicitly in the area. In this paper, we propose adaptations of query clarity techniques from ad-hoc Information Retrieval to define performance predictors in the context of Recommender Systems, which we refer to as user clarity. Our experiments show positive results with different user clarity models in terms of the correlation with single recommender’s performance. Empiric results show significant dependency between this correlation and the recommendation method at hand, as well as competitive results in terms of average correlation.This work was supported by the Spanish Ministry of Science and Innovation (TIN2008-06566-C04-02), University Autónoma de Madrid and the Community of Madrid (CCG10-UAM/TIC-5877

You can teach an old dog new tricks: Rank fusion applied to\ua0coordination level matching for\ua0ranking in systematic reviews

Coordination level matching is a ranking method originally proposed to rank documents given Boolean queries that is now several decades old. Rank fusion is a relatively recent method for combining runs from multiple systems into a single ranking, and has been shown to significantly improve the ranking. This paper presents a novel extension to coordination level matching, by applying rank fusion to each sub-clause of a Boolean query. We show that, for the tasks of systematic review screening prioritisation and stopping estimation, our method significantly outperforms the state-of-the-art learning to rank and bag-of-words-based systems for this domain. Our fully automatic, unsupervised method has (i) the potential for significant real-world cost savings (ii) does not rely on any intervention from the user, and (iii) is significantly better at ranking documents given only a Boolean query in the context of systematic reviews when compared to other approaches