Active learning in annotating micro-blogs dealing with e-reputation

Elections unleash strong political views on Twitter, but what do people really think about politics? Opinion and trend mining on micro blogs dealing with politics has recently attracted researchers in several fields including Information Retrieval and Machine Learning (ML). Since the performance of ML and Natural Language Processing (NLP) approaches are limited by the amount and quality of data available, one promising alternative for some tasks is the automatic propagation of expert annotations. This paper intends to develop a so-called active learning process for automatically annotating French language tweets that deal with the image (i.e., representation, web reputation) of politicians. Our main focus is on the methodology followed to build an original annotated dataset expressing opinion from two French politicians over time. We therefore review state of the art NLP-based ML algorithms to automatically annotate tweets using a manual initiation step as bootstrap. This paper focuses on key issues about active learning while building a large annotated data set from noise. This will be introduced by human annotators, abundance of data and the label distribution across data and entities. In turn, we show that Twitter characteristics such as the author's name or hashtags can be considered as the bearing point to not only improve automatic systems for Opinion Mining (OM) and Topic Classification but also to reduce noise in human annotations. However, a later thorough analysis shows that reducing noise might induce the loss of crucial information.Comment: Journal of Interdisciplinary Methodologies and Issues in Science - Vol 3 - Contextualisation digitale - 201

Mechanical Turk-based Experiment vs Laboratory-based Experiment: A Case Study on the Comparison of Semantic Transparency Rating Data

Have you thought of why you get tired or why you get hungry? Something in your body keeps track of time. It is almost like you have a clock that tells you all those things. And indeed, in the suparachiasmatic region of our hypothalamus reside cells which each act like an oscillator, and together form a coherent circadian rhythm to help our body keep track of time. In fact, such circadian clocks are not limited to mammals but can be found in many organisms including single-cell, reptiles and birds. The study of such rhythms constitutes a field of biology, chronobiology, and forms the background for my research and this thesis. Pioneers of chronobiology, Pittendrigh and Aschoff, studied biological clocks from an input-output view, across a range of organisms by observing and analyzing their overt activity in response to stimulus such as light. Their study was made without recourse to knowledge of the biological underpinnings of the circadian pacemaker. The advent of the new biology has now made it possible to "break open the box" and identify biological feedback systems comprised of gene transcription and protein translation as the core mechanism of a biological clock. My research has focused on a simple transcription-translation clock model which nevertheless possesses many of the features of a circadian pacemaker including its entrainability by light. This model consists of two nonlinear coupled and delayed differential equations. Light pulses can reset the phase of this clock, whereas constant light of different intensity can speed it up or slow it down. This latter property is a signature property of circadian clocks and is referred to in chronobiology as "Aschoff's rule". The discussion in this thesis focus on develop a connection and also a understanding of how constant light effect this clock model

Can the Crowd be Controlled?: A Case Study on Crowd Sourcing and Automatic Validation of Completed Tasks based on User Modeling

Abstract Annotation is an essential step in the development cycle of many Natural Language Processing (NLP) systems. Lately, crowdsourcing has been employed to facilitate large scale annotation at a reduced cost. Unfortunately, verifying the quality of the submitted annotations is a daunting task. Existing approaches address this problem either through sampling or redundancy. However, these approaches do have a cost associated with it. Based on the observation that a crowdsourcing worker returns to do a task that he has done previously, a novel framework for automatic validation of crowd-sourced task is proposed in this paper. A case study based on sentiment analysis is presented to elucidate the framework and its feasibility. The result suggests that validation of the crowd-sourced task can be automated to a certain extent. Keywords: Crowdsourcing, Evaluation, User-modelling Annotation is an unavoidable task for developing NLP systems. Large scale annotation projects such as 1. We present a framework for automatic verifying a crowd sourced task. This can save time and effort spend for validating the submitted task. Moreover, using this framework, a set of reliable worker force can selected a priori for a future task of similar nature. 2. Our results suggest that making the task easier can expedite the task completion rate when compared to increasing the monetary incentive associated with task

Management of Crowdsourcing in Language Teaching and Learning: The State-of-the-art and Future Directions

[EN] The new environment of technological development has led to a transformation of teaching and learning methodologies, especially in the field of language teaching. In this field, the so-called Crowdsourcing, applied in other fields, is an essential element to be considered. This paper tries to identify the applications of Crowdsourcing for teaching and learning. The aim is to observe the state of the art and its trends in order to improve efficiency in learning processes. The paper analyses 192 documents on crowdsourcing and language teaching and learning, extracted from the Web of Science in the period 2012 to 2023. The article carries out a literature review and performs a bibliometric and visualisation analysis essentially on journals, authors and keywords. The results show the importance of crowdsourcing in language teaching and learning, and the particularities of aspects such as Crowdteaching and Crowdlearning associated with new methodologies and technological developments. New trends indicate the relevance of including aspects such as, apart from language and linguistic considerations technological developments such as machine learning, natural language processing, sentiment analysis, or classification models. The results offer guidance to researchers and teachers to plan their research and to improve language teaching and learning processes.Garrigós Simón, FJ.; Narangajavana-Kaosiri, Y. (2023). Management of Crowdsourcing in Language Teaching and Learning: The State-of-the-art and Future Directions. Language Teaching Research Quarterly. 38:128-151. https://doi.org/10.32038/ltrq.2023.38.071281513

Online Gaming for Crowd-sourcing Phrase-equivalents

We propose the use of a game with a purpose (GWAP) to facilitate crowd-sourcing of phrase-equivalents, as an alternative to expert or paid crowd-sourcing. Doodling is an online multiplayer game, in which one player (drawer), draws pictures on a shared board to get the other players (guessers) to guess the meaning behind an assigned phrase. In this paper we describe the system and results from several experiments intended to improve the quality of information generated by the play. In addition, we describe the mechanism by which we take candidate phrases generated during the games and filter out true phrase equivalents. We expect that, at scale, this game will be more cost-efficient than paid mechanisms for a similar task, and demonstrate this by comparing the productivity of an hour of game play to an equivalent crowd-sourced Amazon Mechanical Turk task to produce phrase-equivalents over one week

Crowdsourcing na música - análise de caso do projeto song reader do cantor Beck Hansen │ Crowdsourcing in music - Case study of singer Beck Hansen's Song Reader project

RESUMO Este estudo busca analisar como o projeto Song Reader do cantor Beck Hansen utilizou o Crowdsourcing e como os participantes sentiram-se ao fazer parte do mesmo. Para isso, foram realizadas entrevistas com usuários do Youtube que postaram no site suas versões das canções das músicas do projeto; também, foram analisados dois reviews de sites de notícias. O foco da análise buscou identificar quais os perfis das pessoas que participaram do projeto, entender que tipos de participação/interação foram geradas com o projeto, analisar quais as motivações que levaram as pessoas a participar do projeto e, por fim, entender como essas pessoas avaliaram a experiência. Ao fim do estudo, pode-se deixar mais claro que, no ramo da música, gerar experiência com o público pode ser fundamental. Afinal, as pessoas anseiam, almejam e sentem algo ao ouvir as canções, e é disso que a experiência trata: sobre como fazer com que as pessoas sintam e se envolvam. Para isso, é preciso conhecer o público, e foi justamente o que o projeto Song Reader demonstrou neste trabalho, atendendo e gerando repercussão em seus diferentes públicos: os amadores, os artistas e a mídia. Palavras-chave: Crowdsourcing; Música; Song Reader; Beck Hansen. ABSTRACT This study aims to analyze how the Song Reader project of singer Beck Hansen used Crowdsourcing and how participants felt to be part of it. Interviews were conducted with YouTube users having posted on their site their versions of the songs in the project; two reviews of news sites were also analyzed. The focus of the analysis was to identify the profiles of the people who participated in the project, to understand what types of participation/interaction were generated with the project, to analyze the motivations that led people to participate in the project and finally to understand how these people evaluated the experience. At the end of the study, we can make it clear that in the music business, generating experience with the public can be fundamental. After all, people crave and feel something when listening to the songs, and that defines the experience: how to make people feel and become involved. For this, one must know the audience, and it was just what the Song Reader project demonstrated in this work, caring and generating impact on its stakeholders: amateurs, artists and media. Keywords: Crowdsourcing; Music; Song Reader; Beck Hansen