58 research outputs found
An Attention-Based Model for Predicting Contextual Informativeness and Curriculum Learning Applications
Both humans and machines learn the meaning of unknown words through
contextual information in a sentence, but not all contexts are equally helpful
for learning. We introduce an effective method for capturing the level of
contextual informativeness with respect to a given target word. Our study makes
three main contributions. First, we develop models for estimating contextual
informativeness, focusing on the instructional aspect of sentences. Our
attention-based approach using pre-trained embeddings demonstrates
state-of-the-art performance on our single-context dataset and an existing
multi-sentence context dataset. Second, we show how our model identifies key
contextual elements in a sentence that are likely to contribute most to a
reader's understanding of the target word. Third, we examine how our contextual
informativeness model, originally developed for vocabulary learning
applications for students, can be used for developing better training curricula
for word embedding models in batch learning and few-shot machine learning
settings. We believe our results open new possibilities for applications that
support language learning for both human and machine learner
An Axiomatic Analysis of Diversity Evaluation Metrics: Introducing the Rank-Biased Utility Metric
Many evaluation metrics have been defined to evaluate the effectiveness
ad-hoc retrieval and search result diversification systems. However, it is
often unclear which evaluation metric should be used to analyze the performance
of retrieval systems given a specific task. Axiomatic analysis is an
informative mechanism to understand the fundamentals of metrics and their
suitability for particular scenarios. In this paper, we define a
constraint-based axiomatic framework to study the suitability of existing
metrics in search result diversification scenarios. The analysis informed the
definition of Rank-Biased Utility (RBU) -- an adaptation of the well-known
Rank-Biased Precision metric -- that takes into account redundancy and the user
effort associated to the inspection of documents in the ranking. Our
experiments over standard diversity evaluation campaigns show that the proposed
metric captures quality criteria reflected by different metrics, being suitable
in the absence of knowledge about particular features of the scenario under
study.Comment: Original version: 10 pages. Preprint of full paper to appear at
SIGIR'18: The 41st International ACM SIGIR Conference on Research &
Development in Information Retrieval, July 8-12, 2018, Ann Arbor, MI, USA.
ACM, New York, NY, US
Searching as learning: Novel measures for information interaction research
There is growing recognition of the importance of learning as a search outcome and of the need to provide support for it. Yet, before we can consider learning as a part of search, we need to know how to assess it. This panel will focus on methods and measures for assessing learning in the context of search tasks and their outcomes. The panel will be interactive as the audience will be encouraged to engage in contributing their own experiences and ideas related to measures and methods to study learning as a part of search processes. Ideas and experiences with explicit and implicit indicators of learning and with evaluating learning outcomes will be shared during a dialogue between the audience and panelists. Outcomes from the panel discussions will contribute to formulating a research agenda for “search as learning.” The outcomes will be shared with the audience (and the wider ASIST community).Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/111136/1/meet14505101021.pd
Assessing learning outcomes in web searching: A comparison of tasks and query strategies
Users make frequent use of Web search for learning-related tasks, but little is known about how different Web search interaction strategies affect outcomes for learning-oriented tasks, or what implicit or explicit indicators could reliably be used to assess search-related learning on the Web. We describe a lab-based user study in which we investigated potential indicators of learning in web searching, effective query strategies for learning, and the relationship between search behavior and learning outcomes. Using questionnaires, analysis of written responses to knowledge prompts, and search log data, we found that searchers’ perceived learning outcomes closely matched their actual learning outcomes; that the amount searchers wrote in post-search questionnaire responses was highly correlated with their cognitive learning scores; and that the time searchers spent per document while searching was also highly and consistently correlated with higher-level cognitive learning scores. We also found that of the three query interaction conditions we applied, an intrinsically diverse presentation of results was associated with the highest percentage of users achieving combined factual and conceptual knowledge gains. Our study provides deeper insight into which aspects of search interaction are most effective for supporting superior learning outcomes, and the difficult problem of how learning may be assessed effectively during Web search.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/145733/1/Collins-Thompson Rieh CHIIR 2016.pd
Towards searching as a learning process: A review of current perspectives and future directions
We critically review literature on the association between searching and learning and contribute to the formulation of a research agenda for searching as learning. The paper begins by reviewing current literature that tends to characterize search systems as tools for learning. We then present a perspective on searching as learning that focuses on the learning that occurs during the search pro-cess, as well as search outputs and learning outcomes. The concept of ‘comprehensive search’ is proposed to describe iterative, reflec-tive and integrative search sessions that facilitate critical and creative learning beyond receptive learning. We also discuss how search interaction data can provide a rich source of implicit and explicit features through which to assess search-related learning. In conclu-sion, we summarize opportunities and challenges for future research with respect to four agendas: developing a search system that supports sense-making and enhances learning; supporting effective user interaction for searching as learning; providing an inquiry-based literacy tool within a search system; and assessing learning from online searching behaviour.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/145734/1/Rieh et al Towards searching as a learning process JIS2016.pd
Improved string matching under noisy channel conditions
Many document-based applications, including popular Web browsers, email viewers, and word processors, have a ‘Find on this Page ’ feature that allows a user to find every occurrence of a given string in the document. If the document text being searched is derived from a noisy process such as optical character recognition (OCR), the effectiveness of typical string matching can be greatly reduced. This paper describes an enhanced string-matching algorithm for degraded text that improves recall, while keeping precision at acceptable levels. The algorithm is more general than most approximate matching algorithms and allows string-to-string edits with arbitrary costs. We develop a method for evaluating our technique and use it to examine the relative effectiveness of each sub-component of the algorithm. Of the components we varied, we find that using confidence information from the recognition process lead to the largest improvements in matching accuracy
- …