Using Games with a Purpose and Bootstrapping to Create Domain-Specific Sentiment Lexicons

Sentiment detection analyzes the positive or negative polarity of text. The field has received considerable attention in recent years, since it plays an important role in providing means to assess user opinions regarding an organisation's products, services, or actions. Approaches towards sentiment detection include machine learning techniques as well as computationally less expensive methods. The latter rely on the use of language-specific sentiment lexicons, which are lists of sentiment terms with their corresponding sentiment value. The effort involved in creating, customizing, and extending sentiment lexicons is considerable, particularly if less common languages and domains are targeted without access to appropriate language resources. This paper proposes a semi-automatic approach for the creation of sentiment lexicons which assigns sentiment values to sentiment terms via crowdsourcing. Furthermore, it introduces a bootstrapping process operating on unlabeled domain documents to extend the created lexicons, and to customize them according to the particular use case. This process considers sentiment terms as well as sentiment indicators occurring in the discourse surrounding a particular topic. Such indicators are associated with a positive or negative context in a particular domain, but might have a neutral connotation in other domains. A formal evaluation shows that bootstrapping considerably improves the method's recall. Automatically created lexicons yield a performance comparable to professionally created language resources such as the General Inquirer

A Novel Human Computation Game for Critique Aggregation

We present a human computation game based on the popular board game - Dixit. We ask the players not only for annotations, but for a direct critique of the result of an automated system.We present the results of the initial run of the game, in which the answers of 15 players were used to profile the mistakes of an aspect-based opinion mining system. We show that the gameplay allowed us to identify the major faults of the extracted opinions. The players' actions thus helped improve the opinion extraction algorithm

Echoes of Persuasion: The Effect of Euphony in Persuasive Communication

While the effect of various lexical, syntactic, semantic and stylistic features have been addressed in persuasive language from a computational point of view, the persuasive effect of phonetics has received little attention. By modeling a notion of euphony and analyzing four datasets comprising persuasive and non-persuasive sentences in different domains (political speeches, movie quotes, slogans and tweets), we explore the impact of sounds on different forms of persuasiveness. We conduct a series of analyses and prediction experiments within and across datasets. Our results highlight the positive role of phonetic devices on persuasion

Leveraging the Wisdom of the Crowds for the Acquisition of Multilingual Language Resources

Games with a purpose are an increasingly popular mechanism for leveraging the wisdom of the crowds to address tasks which are trivial for humans but still not solvable by computer algorithms in a satisfying manner. As a novel mechanism for structuring human-computer interactions, a key challenge when creating them is motivating users to participate while generating useful and unbiased results. This paper focuses on important design choices and success factors of effective games with a purpose. Our findings are based on lessons learned while developing and deploying Sentiment Quiz, a crowdsourcing application for creating sentiment lexicons (an essential component of most sentiment detection algorithms). We describe the goals and structure of the game, the underlying application framework, the sentiment lexicons gathered through crowdsourcing, as well as a novel approach to automatically extend the lexicons by means of a bootstrapping process. Such an automated extension further increases the efficiency of the acquisition process by limiting the number of terms that need to be gathered from the game participants

Design of Eco-Feedback Technology to Motivate Sustainable Behavior: Cultural Aspects in a Brazilian Context

The perceived lack of connection between global environmental problems and an individual’s immediate context is among the main reasons why people prove resistant to changing their decisions and actions towards a more sustainable way of life. By bridging this gap and better relating individual behavior to its local and global consequences, properly designed eco-feedback technology may evoke intrinsic and extrinsic motivations and may help to translate awareness into collective action. The concept of culture encompasses the way people relate to the environment and to technology. It influences the perception of control mechanism and guides individual and collective behavior. Considering cultural aspects when designing eco-feedback technology, thus, may improve its persuasive force. This paper presents a conceptual analysis of relevant cultural aspects of the Brazilian society that impact eco-feedback technology adoption and appropriation, and discusses new forms of communication and collaboration that support these processes

Social Tagging: Exploring the Image, the Tags, and the Game

An increasing amount of images are being uploaded, shared, and retrieved on the Web. These large image collections need to be properly stored, organized and easily retrieved. Tags have a key role in image retrieval but it is difficult for those who upload the images to also undertake the quality tag assignment for potential future retrieval by others. Relying on professional keyword assignment is not a practical option for large image collections due to resource constraints. Although a number of content-based image retrieval systems have been launched, they have not demonstrated sufficient utility on large-scale image sources on the web, and are usually used as a supplement to existing text-based image retrieval systems. An alternative to professional image indexing can be social tagging -- with two major types being photo-sharing networks and image labeling games. Here we analyze these applications to evaluate their usefulness from the semantic point of view. We also investigate whether social tagging behaviour can be managed. The findings of the study have shown that social tagging can generate a sizeable number of tags that can be classified as interpretive for an image, and that tagging behaviour has a manageable and adjustable nature depending on tagging guidelines

Aspectos motivacionais no design de tecnologia para mudanças sociais

Orientador: Maria Cecília Calani BaranauskasTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Conectando pessoas e presente em todos os aspectos da vida, quando projetadas para este fim, as tecnologias têm potencial de influenciar a forma com que pessoas em um grupo social percebem e se relacionam com as coisas no ambiente. Este estudo de doutorado em Interação Humano-Computador (IHC) investiga como elementos motivacionais da Psicologia podem ser aplicados para informar o design, explo- rando esse potencial da tecnologia em promover mudanças sociais. O estudo é instanciado no domínio de consumo de energia elétrica, lidando com o desafio contemporâneo de cons- cientizar a sociedade dos limites naturais do planeta no que diz respeito ao uso de recursos naturais. Informar o design com aspectos motivacionais é uma abordagem recente em IHC. Quando encontrada na literatura, comumente tem foco em aspectos individuais e intrín- secos da motivação. Contudo, como argumentado nessa pesquisa, o contexto sociocultural evidencia a importância de considerar também os fatores externos que motivam as pessoas a se engajarem com uma tecnologia e com uma determinada questão social. Por considerar tanto fontes intrínsecas quanto extrínsecas de motivação, a Teoria da Autodeterminação é então considerada o principal referencial teórico da Psicologia nessa investigação, e a Semiótica Organizacional é a base metodológica para analisar os elemen- tos socioculturais que influenciam a motivação extrínseca. A análise situada dos dados socioculturais por uma perspectiva motivacional levou ao design da Tecnologia Socialmente Informada para Eco-Feedback de Energia (sigla SEET, em inglês), uma arquitetura que tem por objetivo estabelecer um novo padrão de com- portamento, ou uma nova maneira de perceber o consumo de energia coletivamente. O SEET é composto por um sistema interativo que promove colaboração, e pela Árvore da Energia, um dispositivo de feedback tangível para locais onde há encontro de pessoas. O SEET é avaliado em dois cenários complementares: uma Escola de Ensino Funda- mental no Brasil, onde os dados socioculturais foram coletados, analisados e aplicados para informar o design; e no contexto de um departamento de uma universidade no Reino Unido. Aspectos motivacionais da arquitetura do SEET são então analisadas, assim como o impacto dessa tecnologia ao desencadear as esperadas mudanças sociaisAbstract: By connecting people and being present in almost all aspects of life, when properly de- signed for that, technology can potentially influence the way people in a social group perceive and relate with things in their environment. This PhD study in the Human-Computer Interaction (HCI) field investigates how motivational elements from Psychology can be applied to inform the design aiming at exploring this potential of technology for promoting a social change. The study is in- stantiated in the energy consumption domain, coping with the contemporary challenge of raising awareness among the society of the planet¿s natural resources usage and limits. Informing the design with motivational aspects is a recent approach in HCI. When found in literature, it is mostly focused on individual and intrinsic aspects of motivation. However, as argued in this research, the sociocultural context evidences the importance of considering also the external factors that motivate people to be engaged with technology and the social issue. By taking into account both intrinsic and extrinsic sources of motivation, the Self- Determination Theory is then considered the main theoretical background from Psychol- ogy in this investigation, and the Organisational Semiotics the methodological basis to analyse sociocultural elements that influence extrinsic motivation. The situated analysis of sociocultural data with motivational lenses has led to the de- sign of a Socially-informed Energy Eco-feedback Technology (SEET), an architecture that aims at establishing a "new pattern of behaviour", or a new way of perceiving collective energy consumption. The SEET is composed by an interactive system that promotes collaboration and The Energy Tree, a tangible and public feedback device for gathering places. The SEET is evaluated in two complementary scenarios: an elementary school in Brazil, where the sociocultural data was collected, analysed and applied to inform design; and in the context of an university department in the United Kingdom. Motivational as- pects of the SEET architecture are then analysed, as well as the impact of this technology to trigger the desired social changeDoutoradoCiência da ComputaçãoDoutora em Ciência da Computaçã

Crowdsourcing for linguistic field research and e-learning

Crowdsourcing denotes the transfer of work commonly carried out by single humans to a large group of people. Nowadays, crowdsourcing is employed for many purposes, like people contributing their knowledge to Wikipedia, researchers predicting diseases from data on Twitter, or players solving protein folding problems in games. Still, there are areas for which the application of crowdsourcing has not yet been investigated thoroughly. This thesis examines crowdsourcing for two such areas: for empirical research in sciences oriented on humans -focusing on linguistic field research- and for e-learning. Sciences oriented on humans -like linguistics, sociology, or art history- depend on empirical research. For example, in traditional linguistic field research researchers ask questions and fill in forms. Such methods are time-consuming, costly, and not free of biases. This thesis proposes the application of crowdsourcing techniques to overcome these disadvantages and to support empirical research in getting more efficient. Therefore, the concept of a generic market for trading with symbolic goods and speculating on their characteristics in a playful manner, called Agora is introduced. Agora aims to be an "operating system" for social media applications gathering data. Furthermore, the Web-based crowdsourcing platform metropolitalia has been established for hosting two social media applications based upon Agora: Mercato Linguistico and Poker Parole. These applications have been conceived as part of this thesis for gathering complementary data and meta-data on Italian language varieties. Mercato Linguistico incites players to express their own knowledge or beliefs, Poker Parole incites players to make conjectures on the contributions of others. Thereby the primary meta-data collected with Mercato Linguistico are enriched with secondary, reflexive meta-data from Poker Parole, which are needed for studies on the perception of languages. An evaluation of the data gathered on metropolitalia exhibits the viability of the market-based approach of Agora and highlights its strengths. E-learning is concerned with the use of digital technology for learning, nowadays especially via the Internet. This thesis investigates how e-learning applications can support students with association-based learning and lecturers with teaching. For that, a game-like e-learning tool named Termina is proposed in this thesis. From the data collected with Termina association maps are constructed. An association map is a simplified version of a concept map, in which concepts are represented as rectangles and relationships between concepts as links. They constitute an abstract comprehension of a topic. Students profit from the association maps' availability, learn from other participating students, and can track their own learning progress. Lecturers gain insights into the knowledge and into potential misunderstandings of their students. An evaluation of Termina and the collected data along a university course exhibits Termina's usefulness for both students and lecturers. The main contributions of this thesis are (1) a literature review over collective intelligence, crowdsourcing, and related fields, (2) a model of a generic market for gathering data for empirical research efficiently, (3) two applications based on this model and results of an evaluation of the data gathered with them, (4) the game-like e-learning tool Termina together with insights from its evaluation, and (5) a generic software architecture for all aforementioned applications.Crowdsourcing bezeichnet die Auslagerung von Arbeit an eine Gruppe von Menschen zur Lösung eines Problems. Heutzutage wird Crowdsourcing für viele Zwecke verwendet, zum Beispiel tragen Leute ihr Wissen zu Wikipedia bei, Wissenschaftler sagen Krankheiten anhand von Twitter-Daten vorher oder Spieler lösen Proteinfaltungsprobleme in Spielen. Es gibt dennoch Gebiete, für die der Einsatz von Crowdsourcing noch nicht gründlich untersucht wurde. Diese Arbeit untersucht Crowdsourcing für zwei solche Gebiete: für empirische Forschung in auf den Menschen bezogenen Wissenschaften mit Fokus auf linguistischer Feldforschung sowie für E-Learning. Auf den Menschen bezogene Wissenschaften wie Linguistik, Soziologie oder Kunstgeschichte beruhen auf empirischer Forschung. In traditioneller linguistischer Feldforschung zum Beispiel stellen Wissenschaftler Fragen und füllen Fragebögen aus. Solche Methoden sind zeitaufwändig, teuer und nicht unbefangen. Diese Arbeit schlägt vor, Crowdsourcing-Techniken anzuwenden, um diese Nachteile zu überwinden und um empirische Forschung effizienter zu gestalten. Dazu wird das Konzept eines generischen Marktes namens Agora für den Handel mit symbolischen Gütern und für die Spekulation über deren Charakteristika eingeführt. Agora ist ein generisches "Betriebssystem" für Social Media Anwendungen. Außerdem wurde die Internet-basierte Crowdsourcing-Plattform metropolitalia eingerichtet, um zwei dieser Social Media Anwendungen, die auf Agora basieren, bereitzustellen: Mercato Linguistico und Poker Parole. Diese Anwendungen wurden als Teil dieser Arbeit entwickelt, um komplementäre Daten und Metadaten über italienische Sprachvarietäten zu sammeln. Mercato Linguistico regt Spieler dazu an, ihr eigenes Wissen und ihre Überzeugungen auszudrücken. Poker Parole regt Spieler dazu an, Vermutungen über die Beiträge anderer Spieler anzustellen. Damit werden die mit Mercato Linguistico gesammelten primären Metadaten mit reflexiven sekundären Metadaten aus Poker Parole, die für Studien über die Wahrnehmung von Sprachen notwendig sind, bereichert. Eine Auswertung der auf metropolitalia gesammelten Daten zeigt die Zweckmäßigkeit des marktbasierten Ansatzes von Agora und unterstreicht dessen Stärken. E-Learning befasst sich mit der Verwendung von digitalen Technologien für das Lernen, heutzutage vor allem über das Internet. Diese Arbeit untersucht, wie E-Learning-Anwendungen Studenten bei assoziationsbasiertem Lernen und Dozenten bei der Lehre unterstützen können. Dafür wird eine Spiel-ähnliche Anwendung namens Termina in dieser Arbeit eingeführt. Mit den über Termina gesammelten Daten werden Association-Maps konstruiert. Eine Association-Map ist eine vereinfachte Variante einer Concept-Map, in der Begriffe als Rechtecke und Beziehungen zwischen Begriffen als Verbindungslinien dargestellt werden. Sie stellen eine abstrakte Zusammenfassung eines Themas dar. Studenten profitieren von der Verfügbarkeit der Association-Maps, lernen von anderen Studenten und können ihren eigenen Lernprozess verfolgen. Dozenten bekommen Einblicke in den Wissensstand und in eventuelle Missverständnisse ihrer Studenten. Eine Evaluation von Termina und der damit gesammelten Daten während eines Universitätskurses bestätigt, dass Termina sowohl für Studenten als auch für Dozenten hilfreich ist. Die Kernbeiträge dieser Arbeit sind (1) eine Literaturrecherche über kollektive Intelligenz, Crowdsourcing und verwandte Gebiete, (2) ein Modell eines generischen Marktes zur effizienten Sammlung von Daten für empirische Forschung, (3) zwei auf diesem Modell basierende Anwendungen und Ergebnisse deren Evaluation, (4) die Spiel-ähnliche E-Learning-Anwendung Termina zusammen mit Einblicken aus dessen Evaluation und (5) eine generische Softwarearchitektur für alle vorgenannten Anwendungen