Equity of Attention: Amortizing Individual Fairness in Rankings

Rankings of people and items are at the heart of selection-making, match-making, and recommender systems, ranging from employment sites to sharing economy platforms. As ranking positions influence the amount of attention the ranked subjects receive, biases in rankings can lead to unfair distribution of opportunities and resources, such as jobs or income. This paper proposes new measures and mechanisms to quantify and mitigate unfairness from a bias inherent to all rankings, namely, the position bias, which leads to disproportionately less attention being paid to low-ranked subjects. Our approach differs from recent fair ranking approaches in two important ways. First, existing works measure unfairness at the level of subject groups while our measures capture unfairness at the level of individual subjects, and as such subsume group unfairness. Second, as no single ranking can achieve individual attention fairness, we propose a novel mechanism that achieves amortized fairness, where attention accumulated across a series of rankings is proportional to accumulated relevance. We formulate the challenge of achieving amortized individual fairness subject to constraints on ranking quality as an online optimization problem and show that it can be solved as an integer linear program. Our experimental evaluation reveals that unfair attention distribution in rankings can be substantial, and demonstrates that our method can improve individual fairness while retaining high ranking quality.Comment: Accepted to SIGIR 201

Look before you Hop: Conversational Question Answering over Knowledge Graphs Using Judicious Context Expansion

Fact-centric information needs are rarely one-shot; users typically ask follow-up questions to explore a topic. In such a conversational setting, the user's inputs are often incomplete, with entities or predicates left out, and ungrammatical phrases. This poses a huge challenge to question answering (QA) systems that typically rely on cues in full-fledged interrogative sentences. As a solution, we develop CONVEX: an unsupervised method that can answer incomplete questions over a knowledge graph (KG) by maintaining conversation context using entities and predicates seen so far and automatically inferring missing or ambiguous pieces for follow-up questions. The core of our method is a graph exploration algorithm that judiciously expands a frontier to find candidate answers for the current question. To evaluate CONVEX, we release ConvQuestions, a crowdsourced benchmark with 11,200 distinct conversations from five different domains. We show that CONVEX: (i) adds conversational support to any stand-alone QA system, and (ii) outperforms state-of-the-art baselines and question completion strategies

TEQUILA: Temporal Question Answering over Knowledge Bases

Question answering over knowledge bases (KB-QA) poses challenges in handling complex questions that need to be decomposed into sub-questions. An important case, addressed here, is that of temporal questions, where cues for temporal relations need to be discovered and handled. We present TEQUILA, an enabler method for temporal QA that can run on top of any KB-QA engine. TEQUILA has four stages. It detects if a question has temporal intent. It decomposes and rewrites the question into non-temporal sub-questions and temporal constraints. Answers to sub-questions are then retrieved from the underlying KB-QA engine. Finally, TEQUILA uses constraint reasoning on temporal intervals to compute final answers to the full question. Comparisons against state-of-the-art baselines show the viability of our method

UnCommonSense: Informative Negative Knowledge about Everyday Concepts

Commonsense knowledge about everyday concepts is an important asset for AIapplications, such as question answering and chatbots. Recently, we have seenan increasing interest in the construction of structured commonsense knowledgebases (CSKBs). An important part of human commonsense is about properties thatdo not apply to concepts, yet existing CSKBs only store positive statements.Moreover, since CSKBs operate under the open-world assumption, absentstatements are considered to have unknown truth rather than being invalid. Thispaper presents the UNCOMMONSENSE framework for materializing informativenegative commonsense statements. Given a target concept, comparable conceptsare identified in the CSKB, for which a local closed-world assumption ispostulated. This way, positive statements about comparable concepts that areabsent for the target concept become seeds for negative statement candidates.The large set of candidates is then scrutinized, pruned and ranked byinformativeness. Intrinsic and extrinsic evaluations show that our methodsignificantly outperforms the state-of-the-art. A large dataset of informativenegations is released as a resource for future research.<br

UnCommonSense: Informative Negative Knowledge about Everyday Concepts

Commonsense knowledge about everyday concepts is an important asset for AIapplications, such as question answering and chatbots. Recently, we have seenan increasing interest in the construction of structured commonsense knowledgebases (CSKBs). An important part of human commonsense is about properties thatdo not apply to concepts, yet existing CSKBs only store positive statements.Moreover, since CSKBs operate under the open-world assumption, absentstatements are considered to have unknown truth rather than being invalid. Thispaper presents the UNCOMMONSENSE framework for materializing informativenegative commonsense statements. Given a target concept, comparable conceptsare identified in the CSKB, for which a local closed-world assumption ispostulated. This way, positive statements about comparable concepts that areabsent for the target concept become seeds for negative statement candidates.The large set of candidates is then scrutinized, pruned and ranked byinformativeness. Intrinsic and extrinsic evaluations show that our methodsignificantly outperforms the state-of-the-art. A large dataset of informativenegations is released as a resource for future research.<br

{YAGO}2: A Spatially and Temporally Enhanced Knowledge Base from {Wikipedia}

We present YAGO2, an extension of the YAGO knowledge base, in which entities, facts, and events are anchored in both time and space. YAGO2 is built automatically from Wikipedia, GeoNames, and WordNet. It contains 80 million facts about 9.8 million entities. Human evaluation confirmed an accuracy of 95\% of the facts in YAGO2. In this paper, we present the extraction methodology, the integration of the spatio-temporal dimension, and our knowledge representation SPOTL, an extension of the original SPO-triple model to time and space

Evaluating the Knowledge Base Completion Potential of GPT

Structured knowledge bases (KBs) are an asset for search engines and other applications, but are inevitably incomplete. Language models (LMs) have been proposed for unsupervised knowledge base completion (KBC), yet, their ability to do this at scale and with high accuracy remains an open question. Prior experimental studies mostly fall short because they only evaluate on popular subjects, or sample already existing facts from KBs. In this work, we perform a careful evaluation of GPT's potential to complete the largest public KB: Wikidata. We find that, despite their size and capabilities, models like GPT-3, ChatGPT and GPT-4 do not achieve fully convincing results on this task. Nonetheless, they provide solid improvements over earlier approaches with smaller LMs. In particular, we show that, with proper thresholding, GPT-3 enables to extend Wikidata by 27M facts at 90% precision.</p

{UNIQORN}: {U}nified Question Answering over {RDF} Knowledge Graphs and Natural Language Text

Question answering over knowledge graphs and other RDF data has been greatly advanced, with a number of good systems providing crisp answers for natural language questions or telegraphic queries. Some of these systems incorporate textual sources as additional evidence for the answering process, but cannot compute answers that are present in text alone. Conversely, systems from the IR and NLP communities have addressed QA over text, but barely utilize semantic data and knowledge. This paper presents the first QA system that can seamlessly operate over RDF datasets and text corpora, or both together, in a unified framework. Our method, called UNIQORN, builds a context graph on the fly, by retrieving question-relevant triples from the RDF data and/or the text corpus, where the latter case is handled by automatic information extraction. The resulting graph is typically rich but highly noisy. UNIQORN copes with this input by advanced graph algorithms for Group Steiner Trees, that identify the best answer candidates in the context graph. Experimental results on several benchmarks of complex questions with multiple entities and relations, show that UNIQORN, an unsupervised method with only five parameters, produces results comparable to the state-of-the-art on KGs, text corpora, and heterogeneous sources. The graph-based methodology provides user-interpretable evidence for the complete answering process