A Study on Microtask Design for Data Grouping without Complete Information

Thesis (Master of Science in Informatics)--University of Tsukuba, no. 36006, 2016.3.25201

Bookshelf problem: a human-in-the-loop approach for data grouping without complete information

The problem of dividing a given set of data items into groups in the situation that the given input is not sufficient to solve it has a wide range of applications. However, the problem cannot be solved by computers alone. This paper defines the Bookshelf problem to deal with such a problem and discusses how to solve the problem with the help of humans. Intuitively, the Bookshelf problem is as follows. Given a set of books with tags and a book cabinet with N shelves, we need to construct N groups of books s.t. all books in each group share at least one common tag. However, the given tags and their connections to books may not be sufficient to make groups, and we have to find the missing tags and connections. This paper proposes a systematic human-in-the-loop method that uses two types of microtasks to solve the problem, and experimentally shows that human intelligence is effective to avoid the worst-case search

Evaluation Measures for Relevance and Credibility in Ranked Lists

Recent discussions on alternative facts, fake news, and post truth politics have motivated research on creating technologies that allow people not only to access information, but also to assess the credibility of the information presented to them by information retrieval systems. Whereas technology is in place for filtering information according to relevance and/or credibility, no single measure currently exists for evaluating the accuracy or precision (and more generally effectiveness) of both the relevance and the credibility of retrieved results. One obvious way of doing so is to measure relevance and credibility effectiveness separately, and then consolidate the two measures into one. There at least two problems with such an approach: (I) it is not certain that the same criteria are applied to the evaluation of both relevance and credibility (and applying different criteria introduces bias to the evaluation); (II) many more and richer measures exist for assessing relevance effectiveness than for assessing credibility effectiveness (hence risking further bias). Motivated by the above, we present two novel types of evaluation measures that are designed to measure the effectiveness of both relevance and credibility in ranked lists of retrieval results. Experimental evaluation on a small human-annotated dataset (that we make freely available to the research community) shows that our measures are expressive and intuitive in their interpretation

PALPAS - PAsswordLess PAssword Synchronization

Tools that synchronize passwords over several user devices typically store the encrypted passwords in a central online database. For encryption, a low-entropy, password-based key is used. Such a database may be subject to unauthorized access which can lead to the disclosure of all passwords by an offline brute-force attack. In this paper, we present PALPAS, a secure and user-friendly tool that synchronizes passwords between user devices without storing information about them centrally. The idea of PALPAS is to generate a password from a high entropy secret shared by all devices and a random salt value for each service. Only the salt values are stored on a server but not the secret. The salt enables the user devices to generate the same password but is statistically independent of the password. In order for PALPAS to generate passwords according to different password policies, we also present a mechanism that automatically retrieves and processes the password requirements of services. PALPAS users need to only memorize a single password and the setup of PALPAS on a further device demands only a one-time transfer of few static data.Comment: An extended abstract of this work appears in the proceedings of ARES 201