Automated Hate Speech Detection and the Problem of Offensive Language

A key challenge for automatic hate-speech detection on social media is the separation of hate speech from other instances of offensive language. Lexical detection methods tend to have low precision because they classify all messages containing particular terms as hate speech and previous work using supervised learning has failed to distinguish between the two categories. We used a crowd-sourced hate speech lexicon to collect tweets containing hate speech keywords. We use crowd-sourcing to label a sample of these tweets into three categories: those containing hate speech, only offensive language, and those with neither. We train a multi-class classifier to distinguish between these different categories. Close analysis of the predictions and the errors shows when we can reliably separate hate speech from other offensive language and when this differentiation is more difficult. We find that racist and homophobic tweets are more likely to be classified as hate speech but that sexist tweets are generally classified as offensive. Tweets without explicit hate keywords are also more difficult to classify.Comment: To appear in the Proceedings of ICWSM 2017. Please cite that versio

Q&A: The N-Word

Question: What is the best way to approach someone who uses blatantly disrespectful language such as the n word and argues that if it is said in the presence of only white people it is not offensive? [excerpt

Language Matters: A Guide To Everyday Questions About Language

Is Ebonics really a dialect or simply bad English? Do women and men speak differently? Will computers ever really learn human language? Does offensive language harm children? These are only a few of the issues surrounding language that crop up every day. Most of us have very definite opinions on these questions one way or another. Yet as linguists Donna Jo Napoli and Vera Lee-Schoenfeld point out in this short and thoroughly readable volume, many of our most deeply held ideas about the nature of language and its role in our lives are either misconceived or influenced by myths and stereotypes Language Matters provides a highly informative tour of the world of language, examining these and other vexing and controversial language-related questions. Throughout, Napoli and Lee-Schoenfeld encourage and lead the reader to use common sense and everyday experience rather than preconceived notions or technical linguistic expertise. Both their questions and their conclusions are surprising, sometimes provocative, and always entertaining. This thoroughly revised second edition updates the book with a new coauthor and includes new chapters on language and power, language extinction, and what it is linguists actually do. Language Matters is sure to engage both general readers and students of language and linguistics at any level