14,564 research outputs found
On Horizontal and Vertical Separation in Hierarchical Text Classification
Hierarchy is a common and effective way of organizing data and representing
their relationships at different levels of abstraction. However, hierarchical
data dependencies cause difficulties in the estimation of "separable" models
that can distinguish between the entities in the hierarchy. Extracting
separable models of hierarchical entities requires us to take their relative
position into account and to consider the different types of dependencies in
the hierarchy. In this paper, we present an investigation of the effect of
separability in text-based entity classification and argue that in hierarchical
classification, a separation property should be established between entities
not only in the same layer, but also in different layers. Our main findings are
the followings. First, we analyse the importance of separability on the data
representation in the task of classification and based on that, we introduce a
"Strong Separation Principle" for optimizing expected effectiveness of
classifiers decision based on separation property. Second, we present
Hierarchical Significant Words Language Models (HSWLM) which capture all, and
only, the essential features of hierarchical entities according to their
relative position in the hierarchy resulting in horizontally and vertically
separable models. Third, we validate our claims on real-world data and
demonstrate that how HSWLM improves the accuracy of classification and how it
provides transferable models over time. Although discussions in this paper
focus on the classification problem, the models are applicable to any
information access tasks on data that has, or can be mapped to, a hierarchical
structure.Comment: Full paper (10 pages) accepted for publication in proceedings of ACM
SIGIR International Conference on the Theory of Information Retrieval
(ICTIR'16
On-the-fly Table Generation
Many information needs revolve around entities, which would be better
answered by summarizing results in a tabular format, rather than presenting
them as a ranked list. Unlike previous work, which is limited to retrieving
existing tables, we aim to answer queries by automatically compiling a table in
response to a query. We introduce and address the task of on-the-fly table
generation: given a query, generate a relational table that contains relevant
entities (as rows) along with their key properties (as columns). This problem
is decomposed into three specific subtasks: (i) core column entity ranking,
(ii) schema determination, and (iii) value lookup. We employ a feature-based
approach for entity ranking and schema determination, combining deep semantic
features with task-specific signals. We further show that these two subtasks
are not independent of each other and can assist each other in an iterative
manner. For value lookup, we combine information from existing tables and a
knowledge base. Using two sets of entity-oriented queries, we evaluate our
approach both on the component level and on the end-to-end table generation
task.Comment: The 41st International ACM SIGIR Conference on Research and
Development in Information Retrieva
Sensitive and Scalable Online Evaluation with Theoretical Guarantees
Multileaved comparison methods generalize interleaved comparison methods to
provide a scalable approach for comparing ranking systems based on regular user
interactions. Such methods enable the increasingly rapid research and
development of search engines. However, existing multileaved comparison methods
that provide reliable outcomes do so by degrading the user experience during
evaluation. Conversely, current multileaved comparison methods that maintain
the user experience cannot guarantee correctness. Our contribution is two-fold.
First, we propose a theoretical framework for systematically comparing
multileaved comparison methods using the notions of considerateness, which
concerns maintaining the user experience, and fidelity, which concerns reliable
correct outcomes. Second, we introduce a novel multileaved comparison method,
Pairwise Preference Multileaving (PPM), that performs comparisons based on
document-pair preferences, and prove that it is considerate and has fidelity.
We show empirically that, compared to previous multileaved comparison methods,
PPM is more sensitive to user preferences and scalable with the number of
rankers being compared.Comment: CIKM 2017, Proceedings of the 2017 ACM on Conference on Information
and Knowledge Managemen
- …