Parkes, David C.

Zou, James Yang

Harvard University - DASH 

 Get Another Worker? Active Crowdlearning With SequentialArrivals  (Article begins on next page)The Harvard community has made this article openly available.Please share how this access benefits you. Your story matters.Citation Zou, James, and David C. Parkes. 2012. Get another worker?Active crowdlearning with sequential arrivals. Proceedings of theWorkshop on Machine Learning in Human Computation andCrowdsourcing (in conjunction with ICML'12), June 30, 2012,Edinburgh, Scotland.Accessed February 19, 2015 1:11:17 PM ESTCitable Link http://nrs.harvard.edu/urn-3:HUL.InstRepos:11956912Terms of Use This article was downloaded from Harvard University's DASHrepository, and is made available under the terms and conditionsapplicable to Open Access Policy Articles, as set forth athttp://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#OAPGet another worker? Active crowdlearning with sequentialarrivals.James Zou jzou@fas.harvard.eduDavid Parkes parkes@eecs.harvard.eduPrior work. Much of the literature on crowdlearn-ing focus on the static case where we have a batchof noisy labels (Raykar, 2010) (Karger, 2011) andneed to infer the true labels. The emerging onlinecrowdlearning papers either assumes 1) we can re-peatedly access any worker (Yan, 2011), or 2) eachworker only labels one item. The decision task is howmany labels to obtain for each item (Sheng, 2008).The contributions of this work are:• We introduce a new model where the workersarrive sequentially and the decision problem ofwhich items to assign to a worker must be madeonline.• We use loopy belief propagation to estimatethe posterior distribution of the true labels andworker types given the current set reported la-bels. The current literature use maximum likeli-hood point estimate of the true label and workertype (usually through EM type approach). Inour online setting, it is advantageous to knowthe variances (uncertainty) as well as the modeof the distribution at every time step.• Using the approximate posterior distributionover true labels and worker types, we design andevaluate a family of active learning algorithms.Model set up. We have a ﬁxed set of items X ={x1;:::;xm} that need to be labeled. The labels arebinary {0;1}, and initially we do not have any knownlabels (no gold standard). Workers are employedto label these data. Let yij denote the label thatworker wi assigns to item xj. Let {yj} denote the(unknown) true labels.A simple worker model often used in crowdlearningliterature (Karger, 2011) (Sheng, 2008) is:• each worker is an iid sample from a binary dis-tribution of good and bad workers.• P(worker = good) = ;P(worker = bad) = 1 − .• P(yij = yj|wi good) =  > 0:5P(yij = yj|wi bad) = 0:5.We assume that workers arrive sequentially{w1;w2;:::}. Each wi is an iid sample from the bi-nary distribution above. At each time step, we mayassign any item xj to the current worker wi to label.Alternatively, we can dismiss wi and assign any itemxj to the new worker wi+1.A worker is given only one item to label at a time,and returns the label instantaneously. We assumethat if we assign the same item to a worker repeat-edly, he will return the same label. A worker may beassigned an arbitrary number of items in total. Afterhe is dismissed, he is no longer available for futuretasks (gone for good). This captures many naturalsettings including: 1) where workers are anonymousand can not be identiﬁed once the engagement is dis-continued; 2) where workers are busy and we onlyget one shot to learn from each one.There is a ﬁxed budget to pay for total of T labelingtasks. It costs the same for any worker to label anyitem. After collecting all the labels, the designersubmits a subset of items along with predicted la-bels to the oracle: {(xj;yj)|xj ∈ ˜ X ⊆ X}. From thesubmissions, he receives reward q1 for every correctlylabeled data, and penalty −q2 for each incorrect pre-diction. The total reward is R = q1 ∗ n1 − q2 ∗ n2,where n1 = |{xj|yj = yj}| and n2 = |{xj|yj ̸= yj}|.The objective is to maximize R.Result I: posterior inference over labels andworker types. Let Wi and Yj denote the randomvariables over the true worker type and item label,respectively. Let D = {(xj;yij)} be the data col-lected. The posterior is given byP({Wi};{Yj}|D) =1ZP(D|{Wi};{Yj})P({Wi};{Yj})where Z is the normalization factor.This can be represented by a Markov random ﬁeld ona bipartite graph with one node for every worker Wiand one for every item Yj. The adjacency matrix ofthe graph, A, is also the assignment matrix: Aij = 1if wi has labeled xj and Aij = 0 otherwise. Thefactor associated to the edge between Wi and Yj isP(yij|Wi;Yj).We run loopy belief propagation on this graph(Murphy, 1999). Message from node Wi to Yj en-capsulates worker wi’s belief on the true label of xj.And message from Yj to Wi encodes item xj’s con-tribution to the posterior belief of the true type ofwi. Before we obtain any labels, the initial marginalsare P(Yj = 0) = 0:5 and P(Wi = good) = . Thealgorithm quickly converges and the belief at eachnode approximates the posterior marginals P(Wi)and P(Yj). By comparing with exact marginals ontractable examples, we verify that the Loopy BP al-gorithm performs eﬀective inference for our model.So far the model assumes that we have  and . Insome applications, rather than knowing  and , wemay have some prior distributions over these param-eters. This more general model can also be encodedin a Markov Random Field, and we use a modiﬁedloopy BP to approximate its posterior marginals.Result II: active learning. The Bayesian optimalprediction is yj = argmaxl2f0;1g P(Yj = l), whereP(Yj) is the posterior approximated from loopy BP.The expected reward isR({yi}) =∑i[q1P(Yi = yi) − q2P(Yi ̸= yi)]: (1)To maximize R, the designer should only submit pre-diction (xj;yj) if q1P(Yj = yj) > q2P(Yj ̸= yj).Example 1. Suppose we have a budget to collect 20labels from a large set of items (M ≫ 20). One naivestrategy is to ask the ﬁrst worker that comes along tolabel 20 diﬀerent items. The probability that a label iscorrect is 0:5+(−0:5). Suppose the penalty is rel-atively large:q2q1 >0:5+( 0:5)0:5 ( 0:5). Then the Bayesianoptimal decision is to not submit any predictions atthe end. This strategy greedily explores item infor-mation to learn the most bits about {P(Yj)}. It doesnot learn anything about the worker type, nor doesit learn enough about P(Yj) for any particular itemxj in order to make a conﬁdent prediction.Example 2. Following the same set up, an al-ternative is to have the ﬁrst worker label items 1-7, second worker label items 1-7, and the thirdworker label 1-6. At the end, we can infer accu-rately the type of each worker (sharpe marginals forP(W1);P(W2);P(W3)), but have high variance initem posteriors.Example 3. In the third strategy, we recruit tenworkers to label item 1 and ten other workers to labelitem 2. We can infer the two correct item labels withhigh probability, but we only get at most 2q1 reward.We identify three dimensions in the learning prob-lem as illustrated by the examples: item explo-ration, worker exploration, and reward exploitation.Item and worker exploration greedily learn the itemand worker marginals. Reward exploitation greedilymaximizes the expected reward. The general ap-proach we adopt for online learning is to select ateach step the action that, in expectation, maximizesa weighted combination of these three objectives.Let Mt < M be the number of items with at leastone label by time t. Then the action space at t hassize at most 2(Mt + 1): assign one of the Mt pointsto either the current worker or to the new worker, orgive an unlabeled data to the current or new worker.It doesn’t matter which of the M − Mt unlabeleditems to pick since they are symmetric a priori.Item exploration. Let D(t) be the set of labelscollected up to time t and {P(Yj;t)} be the itemmarginals at t. Consider an action, a, that as-signs xj to wi. There are two outcomes, yij = 0or yij = 1. Run loopy BP on D(t)∪(xi;yij = 0)and D(t)∪(xi;yij = 1) to obtain the new marginals{P(Yj;a;0)} and {P(Yj;a;1)} respectively. De-ﬁne △Y (a;0) =∑j ∥ P(Yj;a;0) − P(Yj;t) ∥ and△Y (a;1) =∑j ∥ P(Yj;a;1) − P(Yj;t) ∥. Thenorm is L2. Set △Y (a) = △Y (a;0)P(yij = 0) +△Y (a;1)P(yij = 1). △Y (a) measures the expectedgain in item information from action a.Worker exploration. For each action a and pos-sible outcome, we similarly compute the marginaldistributions: {P(Wi;a;0)} and {P(Wi;a;0)}. De-ﬁne △W(a;0) =∑i ∥ P(Wi;a;0) − P(Wi;t) ∥ and△W(a;1) =∑i ∥ P(Wi;a;1)−P(Wi;t) ∥. The gainin worker information is △W(a) = △W(a;0)P(yij =0) + △W(a;1)P(yij = 1).Reward exploitation. Let R(t) be the expectedreward based on D(t) as in Eqn.1. For each ac-tion a and outcome, compute the expected rewardR(a;0) and R(a;1). Deﬁne R(a) = R(a;0)P(yij =0) + R(a;1)P(yij = 1). Then the gain in reward is△R(a) = R(a) − R(t).The algorithm chooses the action a that maximizes△R(a) + 1△Y (a) + 2△W(a). The parameters 1and 2 determine the relative weight of the objec-tives, and characterize our algorithmic space. Wegive analytic characterizations of the algorithm inthe asymptotic limits of 1 and 2. We empiricallyinvestigate the optimal choices of the parameters asa function of ,  while allowing it to also dependon time: 1(;;t) and 2(;;t). We also bench-mark the performance of our active learning algo-rithm against several non-adaptive strategies for as-signing items to workers.2ReferencesRaykar, V., Yu, S., Zhao, L., Valadez, G., Florin, C.,Bogoni, L., and Moy, L. Learning from crowds.Journal of Machine Learning Research, 11: 1297-1322, 2010.Karger,D., Oh,S., and Shah,D. Iterative learningfor reliable crowd-sourcing systems. Proceedingsof NIPS, 2011.Yan,Y., Rosales, R., Fung, G., and Dy J. ActiveLearning from Crowds. ICML, 2011.Sheng,V., Provost,F., and Ipeirotis, P. Get AnotherLabel? Improving Data Quality and Data Miningusing Multiple Noisy Labelers. KDD, 2008.Murphy, K., Weiss, Y., Jordan, M. Loopy BeliefPropagation for Approximate Inference: An Em-pirical Study. UAI, 1999.3

Get Another Worker? Active Crowdlearning With Sequential Arrivals

http://dash.harvard.edu/bitstream/handle/1/11956912/Parkes_GetAnother.pdf?sequence=1

Get Another Worker? Active Crowdlearning With Sequential Arrivals

Abstract

Similar works

Full text

Available Versions

Harvard University - DASH