The quest for information retrieval on the semantic web

Semantic search has been one of the motivations of the Semantic Web since it was envisioned. We propose a model for the exploitation of ontology-based KBs to improve search over large document repositories. The retrieval model is based on an adaptation of the classic vector-space model, including an annotation weighting algorithm, and a ranking algorithm. Semantic search is combined with keyword-based search to achieve tolerance to KB incompleteness. Our proposal has been tested on corpora of significant size, showing promising results with respect to keyword-based search, and providing ground for further analysis and research

An adaptation of the vector-space model for ontology based information retrieval

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. P. Castells, M. Fernández, D. Vallet. "An Adaptation of the Vector-Space Model for Ontology-Based Information Retrieval", IEEE Transactions on Knowledge and Data Engineering, vol. 19, no. 2, pp. 261-272, February 2007.Semantic search has been one of the motivations of the semantic Web since it was envisioned. We propose a model for the exploitation of ontology-based knowledge bases to improve search over large document repositories. In our view of information retrieval on the semantic Web, a search engine returns documents rather than, or in addition to, exact values in response to user queries. For this purpose, our approach includes an ontology-based scheme for the semiautomatic annotation of documents and a retrieval system. The retrieval model is based on an adaptation of the classic vector-space model, including an annotation weighting algorithm, and a ranking algorithm. Semantic search is combined with conventional keyword-based retrieval to achieve tolerance to knowledge base incompleteness. Experiments are shown where our approach is tested on corpora of significant scale, showing clear improvements with respect to keyword-based searchThis research was supported by the European Commission (FP6-027685—MESH), and the Spanish Ministry of Science and Education (TIN2005-06885)

Human Preferences as Dueling Bandits

© 2022 Association for Computing Machinery. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, http://dx.doi.org/10.1145/3477495.3531991The dramatic improvements in core information retrieval tasks engendered by neural rankers create a need for novel evaluation methods. If every ranker returns highly relevant items in the top ranks, it becomes difficult to recognize meaningful differences between them and to build reusable test collections. Several recent papers explore pairwise preference judgments as an alternative to traditional graded relevance assessments. Rather than viewing items one at a time, assessors view items side-by-side and indicate the one that provides the better response to a query, allowing fine-grained distinctions. If we employ preference judgments to identify the probably best items for each query, we can measure rankers by their ability to place these items as high as possible. We frame the problem of finding best items as a dueling bandits problem. While many papers explore dueling bandits for online ranker evaluation via interleaving, they have not been considered as a framework for offline evaluation via human preference judgments. We review the literature for possible solutions. For human preference judgments, any usable algorithm must tolerate ties, since two items may appear nearly equal to assessors, and it must minimize the number of judgments required for any specific pair, since each such comparison requires an independent assessor. Since the theoretical guarantees provided by most algorithms depend on assumptions that are not satisfied by human preference judgments, we simulate selected algorithms on representative test cases to provide insight into their practical utility. Based on these simulations, one algorithm stands out for its potential. Our simulations suggest modifications to further improve its performance. Using the modified algorithm, we collect over 10,000 preference judgments for pools derived from submissions to the TREC 2021 Deep Learning Track, confirming its suitability. We test the idea of best-item evaluation and suggest ideas for further theoretical and practical progress.We thank Mark Smucker, Gautam Kamath, and Ben Carterette for their feedback. This research was supported by the Natural Science and Engineering Research Council of Canada through its Discovery Grants program

Heuristicas y metaconocimiento en resolucion automatica de problemas de matematicas

Centro de Informacion y Documentacion Cientifica (CINDOC). C/Joaquin Costa, 22. 28002 Madrid. SPAIN / CINDOC - Centro de Informaciòn y Documentaciòn CientìficaSIGLEESSpai

Offline evaluation options for recommender systems

We undertake a detailed examination of the steps that make up offline experiments for recommender system evaluation, including the manner in which the available ratings are filtered and split into training and test; the selection of a subset of the available users for the evaluation; the choice of strategy to handle the background effects that arise when the system is unable to provide scores for some items or users; the use of either full or condensed output lists for the purposes of scoring; scoring methods themselves, including alternative top-weighted mechanisms for condensed rankings; and the application of statistical testing on a weighted-by-user or weighted-by-volume basis as a mechanism for providing confidence in measured outcomes. We carry out experiments that illustrate the impact that each of these choice points can have on the usefulness of an end-to-end system evaluation, and provide examples of possible pitfalls. In particular, we show that varying the split between training and test data, or changing the evaluation metric, or how target items are selected, or how empty recommendations are dealt with, can give rise to comparisons that are vulnerable to misinterpretation, and may lead to different or even opposite outcomes, depending on the exact combination of settings usedThe frst two authors were funded in part by Grant TIN2016-80630-P from the Spanish Ministry of Science, Innovation and Universitie

Un sistema de presentación dinámica hipermedia para representaciones

Los sistemas web adaptativos presentan información al usuario en documentos hipermedia generados dinámicamente de acuerdo con un modelo automáticamente actualizado del usuario. Por lo general los sistemas adaptativos existentes imponen un modelo fijo para representar el conocimiento que maneja la aplicación. Por otra parte, utilizan patrones invariables de diseño de página para los documentos generados, que no pueden ser libremente configurados por un diseñador humano. En este artículo proponemos un sistema genérico de presentación para sistemas hipermedia de enseñanza adaptativa altamente independiente de la representación del conocimiento del dominio y del mantenimiento del estado de la aplicación. La generalidad se consigue proporcionando un marco de aplicación para la definición de ontologías que mejor se ajustan a un dominio o un autor concreto. La presentación de las páginas a generar se describe en términos de clases y relaciones de la ontología. Un modelo explícito de la presentación, separado de los contenidos del curso, se utiliza para permitir a los diseñadores un extenso control sobre la generación de todos los aspectos de la presentación, con un coste de desarrollo moderado

Personalized content retrieval in context using ontological knowledge

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Vallet, D., Castells, P., Fernandez, M., Mylonas, P., Avrithis, Y. "Personalized Content Retrieval in Context Using Ontological Knowledge". IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, no. 3, pp. 336-346, March 2007Personalized content retrieval aims at improving the retrieval process by taking into account the particular interests of individual users. However, not all user preferences are relevant in all situations. It is well known that human preferences are complex, multiple, heterogeneous, changing, even contradictory, and should be understood in context with the user goals and tasks at hand. In this paper, we propose a method to build a dynamic representation of the semantic context of ongoing retrieval tasks, which is used to activate different subsets of user interests at runtime, in a way that out-of-context preferences are discarded. Our approach is based on an ontology-driven representation of the domain of discourse, providing enriched descriptions of the semantics involved in retrieval actions and preferences, and enabling the definition of effective means to relate preferences and context.This research was supported by the European Commission (FP6-001765 – aceMedia), and the Spanish Ministry of Science and Education (TIN2005-06885

Workshop on Learning and Evaluating Recommendations with Impressions (LERI)

This volume contains the papers presented at the Workshop on Learning and Evaluating Recommendations with Impressions (LERI), held in conjunction with the 17th ACM Conference on Recommender Systems (RecSys 2023). Recommender systems typically rely on past user interactions as the primary source of information for making predictions. However, although highly informative, past user interactions are strongly biased. Impressions, on the other hand, are a new source of information that indicate the items displayed on screen when the user interacted (or not) with them, and have the potential to impact the field of recommender systems in several ways. Early research on impressions was constrained by the limited availability of public datasets, but this is rapidly changing and, as a consequence, interest in impressions has increased. Impressions present new research questions and opportunities, but also bring new challenges. Several works propose to use impressions as part of recommender models in various ways and discuss their information content. Others explore their potential in off-policy-estimation and reinforcement learning. Overall, the interest of the community is growing, but efforts in this direction remain disconnected. Therefore, one of the aims of the LERI workshop is to bring the community togethe

Characterizing impression-aware recommender systems

Impression-aware recommender systems (IARS) are a type of recommenders that learn user preferences using their interactions and the recommendations (also known as impressions) shown to users. The community’s interest in this type of recommenders has steadily increased in recent years. To aid in characterizing this type of recommenders, we propose a theoretical framework to define IARS and classify the recommenders present in the state-of-the-art. We start this work by defining core concepts related to this type of recommenders, such as impressions and user feedback. Based on this theoretical framework, we identify and define three properties and three taxonomies that characterize IARS. Lastly, we undergo a systematic literature review where we discover and select papers belonging to the state-of-the-art. Our review analyzes papers under the properties and taxonomies we propose; we highlight the most and least common properties and taxonomies used in the literature, their relations, and their evolution over time, among others