Evaluating Singleplayer and Multiplayer in Human Computation Games

Human computation games (HCGs) can provide novel solutions to intractable computational problems, help enable scientific breakthroughs, and provide datasets for artificial intelligence. However, our knowledge about how to design and deploy HCGs that appeal to players and solve problems effectively is incomplete. We present an investigatory HCG based on Super Mario Bros. We used this game in a human subjects study to investigate how different social conditions---singleplayer and multiplayer---and scoring mechanics---collaborative and competitive---affect players' subjective experiences, accuracy at the task, and the completion rate. In doing so, we demonstrate a novel design approach for HCGs, and discuss the benefits and tradeoffs of these mechanics in HCG design.Comment: 10 pages, 4 figures, 2 table

FMA: A Dataset For Music Analysis

We introduce the Free Music Archive (FMA), an open and easily accessible dataset suitable for evaluating several tasks in MIR, a field concerned with browsing, searching, and organizing large music collections. The community's growing interest in feature and end-to-end learning is however restrained by the limited availability of large audio datasets. The FMA aims to overcome this hurdle by providing 917 GiB and 343 days of Creative Commons-licensed audio from 106,574 tracks from 16,341 artists and 14,854 albums, arranged in a hierarchical taxonomy of 161 genres. It provides full-length and high-quality audio, pre-computed features, together with track- and user-level metadata, tags, and free-form text such as biographies. We here describe the dataset and how it was created, propose a train/validation/test split and three subsets, discuss some suitable MIR tasks, and evaluate some baselines for genre recognition. Code, data, and usage examples are available at https://github.com/mdeff/fmaComment: ISMIR 2017 camera-read

Social Tagging: Exploring the Image, the Tags, and the Game

An increasing amount of images are being uploaded, shared, and retrieved on the Web. These large image collections need to be properly stored, organized and easily retrieved. Tags have a key role in image retrieval but it is difficult for those who upload the images to also undertake the quality tag assignment for potential future retrieval by others. Relying on professional keyword assignment is not a practical option for large image collections due to resource constraints. Although a number of content-based image retrieval systems have been launched, they have not demonstrated sufficient utility on large-scale image sources on the web, and are usually used as a supplement to existing text-based image retrieval systems. An alternative to professional image indexing can be social tagging -- with two major types being photo-sharing networks and image labeling games. Here we analyze these applications to evaluate their usefulness from the semantic point of view. We also investigate whether social tagging behaviour can be managed. The findings of the study have shown that social tagging can generate a sizeable number of tags that can be classified as interpretive for an image, and that tagging behaviour has a manageable and adjustable nature depending on tagging guidelines

A Framework for Exploring and Evaluating Mechanics in Human Computation Games

Human computation games (HCGs) are a crowdsourcing approach to solving computationally-intractable tasks using games. In this paper, we describe the need for generalizable HCG design knowledge that accommodates the needs of both players and tasks. We propose a formal representation of the mechanics in HCGs, providing a structural breakdown to visualize, compare, and explore the space of HCG mechanics. We present a methodology based on small-scale design experiments using fixed tasks while varying game elements to observe effects on both the player experience and the human computation task completion. Finally we discuss applications of our framework using comparisons of prior HCGs and recent design experiments. Ultimately, we wish to enable easier exploration and development of HCGs, helping these games provide meaningful player experiences while solving difficult problems.Comment: 11 pages, 5 figure

Retrieval and Annotation of Music Using Latent Semantic Models

PhDThis thesis investigates the use of latent semantic models for annotation and retrieval from collections of musical audio tracks. In particular latent semantic analysis (LSA) and aspect models (or probabilistic latent semantic analysis, pLSA) are used to index words in descriptions of music drawn from hundreds of thousands of social tags. A new discrete audio feature representation is introduced to encode musical characteristics of automatically-identified regions of interest within each track, using a vocabulary of audio muswords. Finally a joint aspect model is developed that can learn from both tagged and untagged tracks by indexing both conventional words and muswords. This model is used as the basis of a music search system that supports query by example and by keyword, and of a simple probabilistic machine annotation system. The models are evaluated by their performance in a variety of realistic retrieval and annotation tasks, motivated by applications including playlist generation, internet radio streaming, music recommendation and catalogue searchEngineering and Physical Sciences Research Counci

"Being in a Knowledge Space": information behaviour of cult media fan communities

This article describes the first two parts of a three-stage study investigating the information behaviour of fans and fan communities, focusing on fans of cult media. A literature analysis shows that information practices are an inherent and major part of fan activities, and that fans are practitioners of new forms of information consumption and production, showing sophisticated activities of information organisation and dissemination. A subsequent Delphi study, taking the novel form of a 'serious leisure' Delphi, in which the participants are not experts in the usual sense, identifies three aspects of fan information behaviour of particular interest beyond the fan context: information gatekeeping; classifying and tagging; and entrepreneurship and economic activity