Does Confidence Reporting from the Crowd Benefit Crowdsourcing Performance?

We explore the design of an effective crowdsourcing system for an $M$-ary classification task. Crowd workers complete simple binary microtasks whose results are aggregated to give the final classification decision. We consider the scenario where the workers have a reject option so that they are allowed to skip microtasks when they are unable to or choose not to respond to binary microtasks. Additionally, the workers report quantized confidence levels when they are able to submit definitive answers. We present an aggregation approach using a weighted majority voting rule, where each worker's response is assigned an optimized weight to maximize crowd's classification performance. We obtain a couterintuitive result that the classification performance does not benefit from workers reporting quantized confidence. Therefore, the crowdsourcing system designer should employ the reject option without requiring confidence reporting.Comment: 6 pages, 4 figures, SocialSens 2017. arXiv admin note: text overlap with arXiv:1602.0057

Developing a functional Natural Language Processing system for the Twi language with limited data

Undergraduate thesis submitted to the Department of Computer Science, Ashesi University, in partial fulfillment of Bachelor of Science degree in Computer Science, April 2019Language is a basic characteristic of human beings. It does not only play the vital role in the transmission of information, but it is also pivotal in the establishment of social connection and emotional bond. For centuries, machines had never possessed the ability to exchange ideas with mankind in an intelligent or intuitive way. However, in recent years, due to breakthroughs in the field of Artificial Intelligence and the rise of computational power, machines have made significant and quite impressive gains in the goal of understanding human language and interacting with it. The branch of Artificial Intelligence which is responsible for enabling machines to understand human language is known as Natural Language Processing. Natural Language Processing involves the utilization of statistical and mathematical models to create algorithms that can train machines to learn and understand human language. The major problem with the algorithms that are created in Natural Language Processing is that they require huge amounts of data to train. Unfortunately, this implies that Natural Language Systems cannot be created for languages that do not have large amounts of readily available data. These kinds of languages are called “low resource languages” and most Ghanaian languages, including the Twi language, fall into this category. This research would explore how a functional Natural Language System may be created for the Twi Ghanaian local language with limited language data.Ashesi Universit