Reliable Crowdsourcing for Multi-Class Labeling using Coding Theory

Crowdsourcing systems often have crowd workers that perform unreliable work on the task they are assigned. In this paper, we propose the use of error-control codes and decoding algorithms to design crowdsourcing systems for reliable classification despite unreliable crowd workers. Coding-theory based techniques also allow us to pose easy-to-answer binary questions to the crowd workers. We consider three different crowdsourcing models: systems with independent crowd workers, systems with peer-dependent reward schemes, and systems where workers have common sources of information. For each of these models, we analyze classification performance with the proposed coding-based scheme. We develop an ordering principle for the quality of crowds and describe how system performance changes with the quality of the crowd. We also show that pairing among workers and diversification of the questions help in improving system performance. We demonstrate the effectiveness of the proposed coding-based scheme using both simulated data and real datasets from Amazon Mechanical Turk, a crowdsourcing microtask platform. Results suggest that use of good codes may improve the performance of the crowdsourcing task over typical majority-voting approaches.Comment: 20 pages, 11 figures, under revision, IEEE Journal of Selected Topics in Signal Processin

Multi-object Classification via Crowdsourcing with a Reject Option

Consider designing an effective crowdsourcing system for an $M$-ary classification task. Crowd workers complete simple binary microtasks whose results are aggregated to give the final result. We consider the novel scenario where workers have a reject option so they may skip microtasks when they are unable or choose not to respond. For example, in mismatched speech transcription, workers who do not know the language may not be able to respond to microtasks focused on phonological dimensions outside their categorical perception. We present an aggregation approach using a weighted majority voting rule, where each worker's response is assigned an optimized weight to maximize the crowd's classification performance. We evaluate system performance in both exact and asymptotic forms. Further, we consider the setting where there may be a set of greedy workers that complete microtasks even when they are unable to perform it reliably. We consider an oblivious and an expurgation strategy to deal with greedy workers, developing an algorithm to adaptively switch between the two based on the estimated fraction of greedy workers in the anonymous crowd. Simulation results show improved performance compared with conventional majority voting.Comment: two column, 15 pages, 8 figures, submitted to IEEE Trans. Signal Proces

Poverty Eradication and Democracy in the Developing World

human development, democracy

Improving Performance Of English-Hindi Cross Language Information Retrieval Using Transliteration Of Query Terms

The main issue in Cross Language Information Retrieval (CLIR) is the poor performance of retrieval in terms of average precision when compared to monolingual retrieval performance. The main reasons behind poor performance of CLIR are mismatching of query terms, lexical ambiguity and un-translated query terms. The existing problems of CLIR are needed to be addressed in order to increase the performance of the CLIR system. In this paper, we are putting our effort to solve the given problem by proposed an algorithm for improving the performance of English-Hindi CLIR system. We used all possible combination of Hindi translated query using transliteration of English query terms and choosing the best query among them for retrieval of documents. The experiment is performed on FIRE 2010 (Forum of Information Retrieval Evaluation) datasets. The experimental result show that the proposed approach gives better performance of English-Hindi CLIR system and also helps in overcoming existing problems and outperforms the existing English-Hindi CLIR system in terms of average precision.Comment: International Journal on Natural Language Computing (IJNLC) Vol. 2, No.6, December 2013 http://airccse.org/journal/ijnlc/index.htm

Drag enhancement and drag reduction in viscoelastic flow

Creeping flow of polymeric fluid without inertia exhibits elastic instabilities and elastic turbulence accompanied by drag enhancement due to elastic stress produced by flow-stretched polymers. However, in inertia-dominated flow at high \mbox{Re} and low fluid elasticity $El$, a reduction in turbulent frictional drag is caused by an intricate competition between inertial and elastic stresses. Here, we explore the effect of inertia on the stability of viscoelastic flow in a broad range of control parameters $El$ and (\mbox{Re}, \mbox{Wi}). We present the stability diagram of observed flow regimes in \mbox{Wi}-\mbox{Re} coordinates and find that instabilities' onsets show unexpectedly non-monotonic dependence on $El$. Further, three distinct regions in the diagram are identified based on $El$. Strikingly, for high elasticity fluids we discover a complete relaminarization of flow at Reynolds number of the order of unity, different from a well-known turbulent drag reduction. These counterintuitive effects may be explained by a finite polymer extensibility and a suppression of vorticity at high \mbox{Wi}. Our results call for further theoretical and numerical development to uncover the role of inertial effect on elastic turbulence in a viscoelastic flow.Comment: 8 pages, 6 figure