An investigation of techniques that aim to improve the quality of labels provided by the crowd

The 2013 MediaEval Crowdsourcing task looked at the problem of working with noisy crowdsourced annotations of image data. The aim of the task was to investigate possible techniques for estimating the true labels of an image by using the set of noisy crowdsourced labels, and possibly any content and metadata from the image itself. For the runs in this paper, we’ve applied a shotgun approach and tried a number of existing techniques, which include generative probabilistic models and further crowdsourcing

Human Computation and Convergence

Humans are the most effective integrators and producers of information, directly and through the use of information-processing inventions. As these inventions become increasingly sophisticated, the substantive role of humans in processing information will tend toward capabilities that derive from our most complex cognitive processes, e.g., abstraction, creativity, and applied world knowledge. Through the advancement of human computation - methods that leverage the respective strengths of humans and machines in distributed information-processing systems - formerly discrete processes will combine synergistically into increasingly integrated and complex information processing systems. These new, collective systems will exhibit an unprecedented degree of predictive accuracy in modeling physical and techno-social processes, and may ultimately coalesce into a single unified predictive organism, with the capacity to address societies most wicked problems and achieve planetary homeostasis.Comment: Pre-publication draft of chapter. 24 pages, 3 figures; added references to page 1 and 3, and corrected typ

When humans and machines collaborate: Cross-lingual Label Editing in Wikidata

The quality and maintainability of a knowledge graph are determined by the process in which it is created. There are different approaches to such processes; extraction or conversion of available data in the web (automated extraction of knowledge such as DBpedia from Wikipedia), community-created knowledge graphs, often by a group of experts, and hybrid approaches where humans maintain the knowledge graph alongside bots. We focus in this work on the hybrid approach of human edited knowledge graphs supported by automated tools. In particular, we analyse the editing of natural language data, i.e. labels. Labels are the entry point for humans to understand the information, and therefore need to be carefully maintained. We take a step toward the understanding of collaborative editing of humans and automated tools across languages in a knowledge graph. We use Wikidata as it has a large and active community of humans and bots working together covering over 300 languages. In this work, we analyse the different editor groups and how they interact with the different language data to understand the provenance of the current label data

Dataset search: a survey

Generating value from data requires the ability to find, access and make sense of datasets. There are many efforts underway to encourage data sharing and reuse, from scientific publishers asking authors to submit data alongside manuscripts to data marketplaces, open data portals and data communities. Google recently beta-released a search service for datasets, which allows users to discover data stored in various online repositories via keyword queries. These developments foreshadow an emerging research field around dataset search or retrieval that broadly encompasses frameworks, methods and tools that help match a user data need against a collection of datasets. Here, we survey the state of the art of research and commercial systems and discuss what makes dataset search a field in its own right, with unique challenges and open questions. We look at approaches and implementations from related areas dataset search is drawing upon, including information retrieval, databases, entity-centric and tabular search in order to identify possible paths to tackle these questions as well as immediate next steps that will take the field forward.</p

Data Work in a Knowledge-Broker Organization: How Cross-Organizational Data Maintenance shapes Human Data Interactions.

The term Human-Data Interaction (HDI) conceptualizes the growing importance of understanding how people need and desire to use and interact with data. Previous HDI cases have mainly focused on the interface between personal health data and the healthcare sector. This paper argues that it is relevant to consider HDI at an organisational level and examines how HDI can look in such a context, where data and data maintenance are core assets and activities. We report on initial findings of a study of a knowledge-broker organisation, where we follow how data are produced, shared, and maintained in a cross-organisational context. We discuss similarities and differences of HDI aroundpersonal health data and cross-organisational data maintenance. We propose to extend the notion of HDI to include the complexity of cross-organisational data work

Towards an Ontology for Public Procurement Based on the Open Contracting Data Standard

The release of a growing amount of open procurement data led to various initiatives for harmonising the data being provided. Among others, the Open Contracting Data Standard (OCDS) is highly relevant due to its high practical value and increasing traction. OCDS defines a common data model for publishing structured data throughout most of the stages of a contracting process. OCDS is document-oriented and focuses on packaging and delivering relevant data in an iterative and event-driven manner through a series of releases. Ontologies, beyond providing uniform access to heterogeneous procurement data, could enable integration with related data sets such as with supplier data for advanced analytics and insight extraction. Therefore, we developed an ontology, the “OCDS ontology”, by using OCDS’ main domain perspective and vocabulary, since it is an essential source of domain knowledge. In this paper, we provide an overview of the developed ontology.acceptedVersio

To Each Their Own: Exploring Highly Personalised Audiovisual Media Accessibility Interventions with People with Aphasia

Digital audiovisual media (e.g., TV, streamed video) is an essential aspect of our modern lives, yet it lacks accessibility – people living with disabilities can experience significant barriers. While accessibility interventions can improve the access to audiovisual media, people living with complex communication needs have been under-represented in research and are potentially left behind. Future visions of accessible digital audiovisual media posit highly personalised content that meets complex accessibility needs. We explore the impact of such a future by conducting bespoke co-design sessions with people with aphasia – a language impairment common post-stroke – creating four highly personal accessibility interventions that leverage audiovisual media personalisation. We then trialled these prototypes with 11 users with aphasia; examining the effects on shared social experiences, creative intent, interaction complexity, and feasibility for content producers. We conclude by critically reflecting on future implementations, raising open questions and suggesting future research directions