Iterative Bayesian Learning for Crowdsourced Regression

Crowdsourcing platforms emerged as popular venues for purchasing human intelligence at low cost for large volume of tasks. As many low-paid workers are prone to give noisy answers, a common practice is to add redundancy by assigning multiple workers to each task and then simply average out these answers. However, to fully harness the wisdom of the crowd, one needs to learn the heterogeneous quality of each worker. We resolve this fundamental challenge in crowdsourced regression tasks, i.e., the answer takes continuous labels, where identifying good or bad workers becomes much more non-trivial compared to a classification setting of discrete labels. In particular, we introduce a Bayesian iterative scheme and show that it provably achieves the optimal mean squared error. Our evaluations on synthetic and real-world datasets support our theoretical results and show the superiority of the proposed scheme

Crowdsourcing in Computer Vision

Computer vision systems require large amounts of manually annotated data to properly learn challenging visual concepts. Crowdsourcing platforms offer an inexpensive method to capture human knowledge and understanding, for a vast number of visual perception tasks. In this survey, we describe the types of annotations computer vision researchers have collected using crowdsourcing, and how they have ensured that this data is of high quality while annotation effort is minimized. We begin by discussing data collection on both classic (e.g., object recognition) and recent (e.g., visual story-telling) vision tasks. We then summarize key design decisions for creating effective data collection interfaces and workflows, and present strategies for intelligently selecting the most important data instances to annotate. Finally, we conclude with some thoughts on the future of crowdsourcing in computer vision.Comment: A 69-page meta review of the field, Foundations and Trends in Computer Graphics and Vision, 201

Knowledge Graph semantic enhancement of input data for improving AI

Intelligent systems designed using machine learning algorithms require a large number of labeled data. Background knowledge provides complementary, real world factual information that can augment the limited labeled data to train a machine learning algorithm. The term Knowledge Graph (KG) is in vogue as for many practical applications, it is convenient and useful to organize this background knowledge in the form of a graph. Recent academic research and implemented industrial intelligent systems have shown promising performance for machine learning algorithms that combine training data with a knowledge graph. In this article, we discuss the use of relevant KGs to enhance input data for two applications that use machine learning -- recommendation and community detection. The KG improves both accuracy and explainability

Patterns of Oral Microbiota Diversity in Adults and Children: A Crowdsourced Population Study.

Oral microbiome dysbiosis has been associated with various local and systemic human diseases such as dental caries, periodontal disease, obesity, and cardiovascular disease. Bacterial composition may be affected by age, oral health, diet, and geography, although information about the natural variation found in the general public is still lacking. In this study, citizen-scientists used a crowdsourcing model to obtain oral bacterial composition data from guests at the Denver Museum of Nature & Science to determine if previously suspected oral microbiome associations with an individual's demographics, lifestyle, and/or genetics are robust and generalizable enough to be detected within a general population. Consistent with past research, we found bacterial composition to be more diverse in youth microbiomes when compared to adults. Adult oral microbiomes were predominantly impacted by oral health habits, while youth microbiomes were impacted by biological sex and weight status. The oral pathogen Treponema was detected more commonly in adults without recent dentist visits and in obese youth. Additionally, oral microbiomes from participants of the same family were more similar to each other than to oral microbiomes from non-related individuals. These results suggest that previously reported oral microbiome associations are observable in a human population containing the natural variation commonly found in the general public. Furthermore, these results support the use of crowdsourced data as a valid methodology to obtain community-based microbiome data