4,067 research outputs found
Rhetorical relations for information retrieval
Typically, every part in most coherent text has some plausible reason for its
presence, some function that it performs to the overall semantics of the text.
Rhetorical relations, e.g. contrast, cause, explanation, describe how the parts
of a text are linked to each other. Knowledge about this socalled discourse
structure has been applied successfully to several natural language processing
tasks. This work studies the use of rhetorical relations for Information
Retrieval (IR): Is there a correlation between certain rhetorical relations and
retrieval performance? Can knowledge about a document's rhetorical relations be
useful to IR? We present a language model modification that considers
rhetorical relations when estimating the relevance of a document to a query.
Empirical evaluation of different versions of our model on TREC settings shows
that certain rhetorical relations can benefit retrieval effectiveness notably
(> 10% in mean average precision over a state-of-the-art baseline)
You can't see what you can't see: Experimental evidence for how much relevant information may be missed due to Google's Web search personalisation
The influence of Web search personalisation on professional knowledge work is
an understudied area. Here we investigate how public sector officials
self-assess their dependency on the Google Web search engine, whether they are
aware of the potential impact of algorithmic biases on their ability to
retrieve all relevant information, and how much relevant information may
actually be missed due to Web search personalisation. We find that the majority
of participants in our experimental study are neither aware that there is a
potential problem nor do they have a strategy to mitigate the risk of missing
relevant information when performing online searches. Most significantly, we
provide empirical evidence that up to 20% of relevant information may be missed
due to Web search personalisation. This work has significant implications for
Web research by public sector professionals, who should be provided with
training about the potential algorithmic biases that may affect their judgments
and decision making, as well as clear guidelines how to minimise the risk of
missing relevant information.Comment: paper submitted to the 11th Intl. Conf. on Social Informatics;
revision corrects error in interpretation of parameter Psi/p in RBO resulting
from discrepancy between the documentation of the implementation in R
(https://rdrr.io/bioc/gespeR/man/rbo.html) and the original definition
(https://dl.acm.org/citation.cfm?id=1852106) as per 20/05/201
Embedding Web-based Statistical Translation Models in Cross-Language Information Retrieval
Although more and more language pairs are covered by machine translation
services, there are still many pairs that lack translation resources.
Cross-language information retrieval (CLIR) is an application which needs
translation functionality of a relatively low level of sophistication since
current models for information retrieval (IR) are still based on a
bag-of-words. The Web provides a vast resource for the automatic construction
of parallel corpora which can be used to train statistical translation models
automatically. The resulting translation models can be embedded in several ways
in a retrieval model. In this paper, we will investigate the problem of
automatically mining parallel texts from the Web and different ways of
integrating the translation models within the retrieval process. Our
experiments on standard test collections for CLIR show that the Web-based
translation models can surpass commercial MT systems in CLIR tasks. These
results open the perspective of constructing a fully automatic query
translation device for CLIR at a very low cost.Comment: 37 page
A Cognitive-based scheme for user reliability and expertise assessment in Q&A social networks
Q&A social media has gained a great deal of attention during recent years. People rely on these sites to obtain information due to the number of advantages they offer as compared to conventional sources of knowledge (e.g., asynchronous and convenient access). However, for the same question one may find highly contradictory answers, causing ambiguity with respect to the correct information. This can be attributed to the presence of unreliable and/or non-expert users. In this work, we propose a novel approach for estimating the reliability and expertise of a user based on human cognitive traits. Every user can individually estimate these values based on local pairwise interactions. We examine the convergence performance of our algorithm and we find that it can accurately assess the reliability and the expertise of a user and can successfully react to the latter's behavior change. © 2011 IEEE
Good Applications for Crummy Entity Linkers? The Case of Corpus Selection in Digital Humanities
Over the last decade we have made great progress in entity linking (EL)
systems, but performance may vary depending on the context and, arguably, there
are even principled limitations preventing a "perfect" EL system. This also
suggests that there may be applications for which current "imperfect" EL is
already very useful, and makes finding the "right" application as important as
building the "right" EL system. We investigate the Digital Humanities use case,
where scholars spend a considerable amount of time selecting relevant source
texts. We developed WideNet; a semantically-enhanced search tool which
leverages the strengths of (imperfect) EL without getting in the way of its
expert users. We evaluate this tool in two historical case-studies aiming to
collect a set of references to historical periods in parliamentary debates from
the last two decades; the first targeted the Dutch Golden Age, and the second
World War II. The case-studies conclude with a critical reflection on the
utility of WideNet for this kind of research, after which we outline how such a
real-world application can help to improve EL technology in general.Comment: Accepted for presentation at SEMANTiCS '1
Characterizing Health-Related Information Needs of Domain Experts (regular paper)
International audienceIn information retrieval literature, understanding the users' intents behind the queries is critically important to gain a better insight of how to select relevant results. While many studies investigated how users in general carry out exploratory health searches in digital environments, a few focused on how are the queries formulated, specifically by domain expert users. This study intends to fill this gap by studying 173 health expert queries issued from 3 medical information retrieval tasks within 2 different evaluation compaigns. A statistical analysis has been carried out to study both variation and correlation of health-query attributes such as length, clarity and specificity of either clinical or non clinical queries. The knowledge gained from the study has an immediate impact on the design of future health information seeking systems
Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling
Uncertainty decomposition refers to the task of decomposing the total
uncertainty of a model into data (aleatoric) uncertainty, resulting from the
inherent complexity or ambiguity of the data, and model (epistemic)
uncertainty, resulting from the lack of knowledge in the model. Performing
uncertainty decomposition for large language models (LLMs) is an important step
toward improving the reliability, trustworthiness, and interpretability of
LLMs, but this research task is very challenging and remains unresolved. The
existing canonical method, Bayesian Neural Network (BNN), cannot be applied to
LLMs, because BNN requires training and ensembling multiple variants of models,
which is infeasible or prohibitively expensive for LLMs. In this paper, we
introduce an uncertainty decomposition framework for LLMs, called input
clarifications ensemble, which bypasses the need to train new models. Rather
than ensembling models with different parameters, our approach generates a set
of clarifications for the input, feeds them into the fixed LLMs, and ensembles
the corresponding predictions. We show that our framework shares a symmetric
decomposition structure with BNN. Empirical evaluations demonstrate that the
proposed framework provides accurate and reliable uncertainty quantification on
various tasks. Code will be made publicly available at
https://github.com/UCSB-NLP-Chang/llm_uncertainty .Comment: 15 pages, 3 figure
- âŠ