Crowdsourcing opens the door to solving a wide variety of problems that previ-ously were unfeasible in the field of machine learning, allowing us to obtain rela-tively low cost labeled data in a small amount of time. However, due to theuncertain quality of labelers, the data to deal with are sometimes unreliable, forcingpractitioners to collect information redundantly, which poses new challenges in thefield. Despite these difficulties, many applications of machine learning usingcrowdsourced data have recently been published that achieved state of the artresults in relevant problems. We have analyzed these applications following a sys-tematic methodology, classifying them into different fields of study, highlightingseveral of their characteristics and showing the recent interest in the use of crowd-sourcing for machine learning. We also identify several exciting research linesbased on the problems that remain unsolved to foster future research in this field
