A Human Computation Approach to English Translation of Internet Lingo

Lingo is an emerging language on the Internet. To understand the meaning of lingo can help analyze the web content and understand various cultures in the online communities. However, providing a standardized definition remains difficult due to continuous changes made to its nature. We proposed Tranzzl!n9o, a crossword puzzle game for engaging crowds to translate Internet lingo. In our game, players provide explanations for lingo in parallel and iteratively verify the explanations from other players. We conducted experiments with 45 qualified workers to evaluate our design on Amazon Mechanical Turk. There are 138 explanations generated from 20 puzzles by 45 qualified players. Results show that we achieved 77.06% precision and 85.71% recall for collecting explanations of lingo. With at least twice agreements, we achieved 90.57% precision and 48.98% recall. Moreover, crowed-sourced explanations are very informative, not only explaining lingo itself but also containing lingo usage. Follow-up questionnaires show that over 60% of players like our game and would like to play it again. Considering weekly players, 75% of them said so. By keeping our lingo dictionary updated, we hope to support out-of-vocabulary issues in language processing and an annotated corpus of lingo for machine learning, and help Internet users better-understand lingo.Contents 口試委員會審定書 iii 誌謝 v Abstract vii 1 Introduction 1 1.1 Motivation.................................. 1 1.2 TranslatingInternetLingo ......................... 2 1.3 ProblemDefinition ............................. 3 2 Related Work 7 2.1 InternetLingoDictionaryLookup ..................... 7 2.2 MachineComputationinTextNormalization . . . . . . . . . . . . . . . 8 2.2.1 SpellingCorrectionApproach ................... 8 2.2.2 MachineTranslationApproach................... 8 2.2.3 AutomaticSpeechRecognitionApproach . . . . . . . . . . . . . 9 2.3 Human 2.3.1 Jinx for Generating Word Sense Disambiguation Dataset . . . . . 10 2.3.2 GuesstimentforSentimentAnnotation. . . . . . . . . . . . . . . 10 3 Internet Lingo Translating System 11 3.1 SystemDesignIncentive .......................... 11 3.2 InternetLingoExtraction.......................... 13 ComputationinWordGames.................... 9 ix 3.2.1 LanguageDetection ........................ 13 3.2.2 Rule-basedFiltering ........................ 14 3.3 DictionarySetupandQuestionPool .................... 15 3.4 HumanComputation:WordGame..................... 15 3.4.1 PuzzleGenerator .......................... 16 3.4.2 GameWorkflow .......................... 17 3.5 HumanComputation:DesignFeatures................... 19 3.5.1 PlayerQualification ........................ 19 3.5.2 Two-stageAgreement ....................... 21 4 Experiments and Evaluation 23 4.1 SystemDeployment............................. 23 4.2 DataProfile................................. 24 4.2.1 TweetsDataStatistics ....................... 24 4.2.2 CrosswordPuzzleSetup ...................... 25 4.3 Experiments................................. 26 4.3.1 LessonLearnedfromPilotStudy ................. 26 4.3.2 DiscussionofCollectedExplanations . . . . . . . . . . . . . . . 27 4.4 Evaluation.................................. 29 4.4.1 GroundTruthCollection...................... 29 4.4.2 EvaluationMetrics ......................... 30 4.4.3 Evaluation:LingofyTask...................... 31 4.4.4 Evaluation:UnlingofyTask .................... 32 4.4.5 Evaluation:EngaginginTranzz!n9o. . . . . . . . . . . . . . . . 34 5 Conclusions and Future work 39 5.1 Conclusions................................. 39 5.2 Limitations ................................. 40 5.3 FutureWork................................. 40 Bibliography 41