SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice

To counter online abuse and misinformation, social media platforms have been establishing content moderation guidelines and employing various moderation policies. The goal of this paper is to study these community guidelines and moderation practices, as well as the relevant research publications to identify the research gaps, differences in moderation techniques, and challenges that should be tackled by the social media platforms and the research community at large. In this regard, we study and analyze in the US jurisdiction the fourteen most popular social media content moderation guidelines and practices, and consolidate them. We then introduce three taxonomies drawn from this analysis as well as covering over one hundred interdisciplinary research papers about moderation strategies. We identified the differences between the content moderation employed in mainstream social media platforms compared to fringe platforms. We also highlight the implications of Section 230, the need for transparency and opacity in content moderation, why platforms should shift from a one-size-fits-all model to a more inclusive model, and lastly, we highlight why there is a need for a collaborative human-AI system

Impact of Stricter Content Moderation on Parler's Users' Discourse

Social media platforms employ various content moderation techniques to remove harmful, offensive, and hate speech content. The moderation level varies across platforms; even over time, it can evolve in a platform. For example, Parler, a fringe social media platform popular among conservative users, was known to have the least restrictive moderation policies, claiming to have open discussion spaces for their users. However, after linking the 2021 US Capitol Riots and the activity of some groups on Parler, such as QAnon and Proud Boys, on January 12, 2021, Parler was removed from the Apple and Google App Store and suspended from Amazon Cloud hosting service. Parler would have to modify their moderation policies to return to these online stores. After a month of downtime, Parler was back online with a new set of user guidelines, which reflected stricter content moderation, especially regarding the \emph{hate speech} policy. In this paper, we studied the moderation changes performed by Parler and their effect on the toxicity of its content. We collected a large longitudinal Parler dataset with 17M parleys from 432K active users from February 2021 to January 2022, after its return to the Internet and App Store. To the best of our knowledge, this is the first study investigating the effectiveness of content moderation techniques using data-driven approaches and also the first Parler dataset after its brief hiatus. Our quasi-experimental time series analysis indicates that after the change in Parler's moderation, the severe forms of toxicity (above a threshold of 0.5) immediately decreased and sustained. In contrast, the trend did not change for less severe threats and insults (a threshold between 0.5 - 0.7). Finally, we found an increase in the factuality of the news sites being shared, as well as a decrease in the number of conspiracy or pseudoscience sources being shared

DIGITAL LITERACY AS A FOUNDATION FOR RELIGIOUS MODERATION LEARNING AT SALATIGA'S AL-HIJRAH TINGKIR ISLAMIC BOARDING SCHOOL

The current millennial generation, includes santri, who have digital literacy skills, but unfortunately it has not been matched by ability to use digital media to obtain information and self development. The purpose of the research was to determine the extent to which students’ digital literacy skills, how to use digital literacy as a basis for religious moderation learning, and the obstacles faced during the training on the use of digital literacy at the Al-Hijrah Islamic Boarding School Tingkir Salatiga. This research is a qualitative and participatory action research. Data collection techniques used are participant observation, interviews, and documentation. The data analysis technique uses Mile and Hubermen in the form of reduction, presentation, and conclusions. The research procedure includes planning, implementation, observation, and reflection. The result of the studi show that digital literacy can be used as a basis for learning religious moderation. The students at the Al-Hijrah Islamic Boarding School can get digital content in the form of religious moderation material from website, social media, and others. The students also take advantage of digital literacy to create religious moderation content

The Perception of Millennial Generation on Religious Moderation through Social Media in the Digital Era

The phenomenon of learning religion through social media is rife among the millennial generation. This is an opportunity for the internalization of religious moderation in the millennial environment. This study aims to reveal the millennial generation's perception of religious moderation through social media. This study uses a qualitative research design with case study methods, while data collection techniques include interviews, observation, and documentation integrally with data analysis techniques through the theory of Milles and Huberman. This study shows that the millennial generation's perception of the implementation of religious moderation can be internalized through social media intermediaries with video, reading articles, and intensive persuasion strategies from social media content owners. The research findings illustrate that the phenomenon of the implementation of religious moderation in the millennial generation can be improved through the distribution of religious materials that are relevant to modern issues without reducing the essence of religious education. This study recommends further research to examine the effectiveness of understanding religious moderation in the younger generation in dealing with pluralism in Indonesia

Moderation of religion in the Fatwa of Majelis Ulama Indonesia about the Ethics of da’wah in the Digital Age

Purpose - The rampant da’wah content in social media that triggers social and national disintegration is a problem and a big task for the government and Muslims in Indonesia to prevent it. This study seeks to identify the values of moderation in the fatwa of the Majelis Ulama Indonesia of East Java Province Number—06 of 2022 concerning da’wah Ethics in the Digital Era.Method - This qualitative study uses a philosophical normative approach. The study's primary data used the fatwa document of the Majelis Ulama Indonesia, East Java Province Number—06 of 2022 concerning da’wah Ethics in the Digital Era. Indicators of religious moderation conceptualized by the Ministry of Religious Affairs of the Republic of Indonesia as the primary analysis theory. The documentation techniques used in collecting data in this study and the data analysis techniques used were data reduction, presentation, and verification.Results - This study concludes that there is a dimension of religious moderation in the fatwa of Majelis Ulama Indonesia of East Java Province Number—06 of 2022 concerning da’wah Ethics in the Digital Era, among others, the value of anti-radicalism in the prohibition of the delivery of provocative da’wah content, the value of tolerance in the recommendation of delivering pluralist da’wah content, the value of national commitment in the recommendation of delivering da’wah content that leads to compliance with the country's constitution, and the accommodating value of local wisdom in the recommendation of delivering da’wah content that maintains public conduciveness. Implications – This study shows that mainstreaming religious moderation in Indonesia can be internalized in religious social fatwas.Originality - This study shows that the four indicators of religious moderation in Indonesia (tolerance, anti-radicalism, national commitment, and accommodating to local wisdom) can be internalized in the digital media space's code of ethics for da’wah activities.***Tujuan - Maraknya konten dakwah di media sosial yang memicu disintegrasi sosial dan bangsa merupakan masalah dan tugas besar bagi pemerintah dan umat Islam di Indonesia untuk mencegahnya. Kajian ini berupaya mengidentifikasi nilai-nilai moderasi dalam fatwa Majelis Ulama Indonesia Provinsi Jawa Timur Nomor—06 Tahun 2022 tentang Etika Dakwah di Era Digital.Metode – Penelitian kualitatif ini menggunakan pendekatan filosofis normatif. Data primer penelitian ini menggunakan dokumen fatwa Majelis Ulama Indonesia Provinsi Jawa Timur Nomor—06 Tahun 2022 tentang Etika Dakwah di Era Digital. Indikator moderasi beragama dikonseptualisasikan oleh Kementerian Agama Republik Indonesia sebagai teori analisis utama. Teknik dokumentasi yang digunakan dalam mengumpulkan data dalam penelitian ini dan teknik analisis data yang digunakan adalah reduksi data, penyajian, dan verifikasi.Hasil - Penelitian ini menyimpulkan bahwa terdapat dimensi moderasi beragama dalam fatwa Majelis Ulama Indonesia Provinsi Jawa Timur Nomor—06 Tahun 2022 tentang Etika Dakwah di Era Digital antara lain nilai anti radikalisme dalam larangan penyampaian konten dakwah yang provokatif, nilai toleransi dalam rekomendasi penyampaian konten dakwah pluralis, nilai komitmen nasional dalam rekomendasi penyampaian konten dakwah yang mengarah pada kepatuhan terhadap konstitusi negara, dan akomodatif nilai kearifan lokal dalam rekomendasi penyampaian konten dakwah yang menjaga kondusifitas publik.Implikasi – Kajian ini menunjukkan bahwa pengarusutamaan moderasi beragama di Indonesia dapat terinternalisasi dalam fatwa-fatwa sosial keagamaan.Orisinalitas - Studi ini menunjukkan bahwa empat indikator moderasi beragama di Indonesia (toleransi, anti radikalisme, komitmen kebangsaan, dan akomodatif terhadap kearifan lokal) dapat diinternalisasikan dalam kode etik ruang media digital untuk kegiatan dakwah

Detection of Hate-Speech Tweets Based on Deep Learning: A Review

Cybercrime, cyberbullying, and hate speech have all increased in conjunction with the use of the internet and social media. The scope of hate speech knows no bounds or organizational or individual boundaries. This disorder affects many people in diverse ways. It can be harsh, offensive, or discriminating depending on the target's gender, race, political opinions, religious intolerance, nationality, human color, disability, ethnicity, sexual orientation, or status as an immigrant. Authorities and academics are investigating new methods for identifying hate speech on social media platforms like Facebook and Twitter. This study adds to the ongoing discussion about creating safer digital spaces while balancing limiting hate speech and protecting freedom of speech.   Partnerships between researchers, platform developers, and communities are crucial in creating efficient and ethical content moderation systems on Twitter and other social media sites. For this reason, multiple methodologies, models, and algorithms are employed. This study presents a thorough analysis of hate speech in numerous research publications. Each article has been thoroughly examined, including evaluating the algorithms or methodologies used, databases, classification techniques, and the findings achieved.   In addition, comprehensive discussions were held on all the examined papers, explicitly focusing on consuming deep learning techniques to detect hate speech

Parler and the Road to the Capitol Attack: Investigating Alt-Tech Ties to January 6

The January 6, 2021 mob assault on the U.S. Capitol exposed deep fissures between Americans and shook the very foundations of the country. The violence that day and the tech industry's response to the tsunami of polarizing content triggered a major public debate over how social media and tech companies manage their platforms and services and the impact of content moderation policies on polarization, extremism, and political violence in the United States. That debate is also now playing out in Congress where the House Select Committee to Investigate the January 6th Attack on the United States Capitol is now underway. One big question is: How did niche social media sites geared toward far-right audiences, like Parler, contribute to polarization around the 2020 presidential election and to what extent did Parler and other platforms factor into the January 6 attack? The first in a series of investigations into the impact of the alt-tech movement on U.S. national security, this report provides an initial snapshot of observations culled from an ongoing analysis of open source data related to the Capitol attack.Based, in part, on an early assessment of a cache of an estimated 183 million Parler posts publicly archived after Parler was temporarily deplatformed, the analysis in this report offers unique insights into online and offline early warning signs of the potential for election-related violence in the year-long run up to the Capitol attack. On the streets and online, the networked effects of poor platform governance across the internet during the 2020 presidential election were notable on mainstream and fringe social media sites. Nevertheless, the combined impact of Parler's loose content moderation scheme as well as data-management practices and platform features—either by design or neglect, or both—may have made the social media startup especially vulnerable to strategic influence campaigns that relied heavily on inauthentic behavior like automated content amplification and deceptive techniques like astroturfing

Behind the screen: the hidden digital labor of commercial content moderation

Commercial content moderation (CCM) is the practice of screening of user-generated content (UGC) posted to Internet sites, social media platforms and other online outlets that encourage and rely upon such material to generate visits to and participation from users. Despite being essential to the media production cycle for these commercial websites and social media platforms as a major source of brand protection, gatekeeping and tastemaking, commercial content moderation is largely unknown outside its own industry and those that rely on it. This research endeavors to unveil the practice of commercial content moderation and to further contextualize it alongside contemporary trends of globalization, outsourcing and other economic and geospatial reconfigurations facilitated by the increasingly networked nature of the world. CCM tasks vacillate from the mind-numbingly repetitive and mundane to exposure to images and material that can be violent, disturbing and, at worst, psychologically damaging, and it requires these tasks of workers that are frequently relatively low-status and low-wage. The workers are typically further isolated because the work they do is in secret, considered an issue of brand protection by their employers. Using analysis of in-depth qualitative interviews with CCM workers in a variety of sectors coupled with techniques of Critical Discourse Analysis of industry-generated materials (e.g., websites; work solicitations), this research connects the practices of CCM to digital media economics, digital media practices and their sociopolitical, economic and ethical implications. It reports on and describes the experiences of content moderators in a number of different contexts and situations. It provides a taxonomy for understanding the worksites and practices of CCM and maps it, on theoretical grounds, to other types of digital and non-digital work, aligning it in the greater context of the ecology of social media to the end of recognizing, acknowledging and improving the conditions under which the CCM workers labor

Media Management of the Nahdlatul Ulama Da'wah Institute in Promoting Religious Moderation in the City of Padang Sidempuan

The focus of this article is on the socialization of religious moderation in the City of Padang Sidempuan. The city of Padang Sidempuan has a diverse community, both ethnic, cultural and religious. For peace in the area, it is necessary to cultivate religious moderation in the midst of society which is carried out by the Nahdatul Ulama Da'wah Institute. In this socialization, communication and management are needed because in this process there are a series of concepts that must be prepared in a message before the message is channeled to the communicant. The method used is a qualitative method with data collection techniques using interviews and observation then using descriptive analysis. With the finding that in Padang Sidempuan, the Nahdlatul Ulama Da'wah Institute uses social media as their communication tool in socializing the important role of moderation in people's religious life. The Nahdlatul Ulama Da'wah Institute also thinks about a plan, organization, implementation and supervision that is carried out with the aim of influencing the people of Padang Sidempuan. The Nahdlatul Ulama Da'wah Institute uses social media such as Facebook, Instagram, Whatsapp to disseminate their content which leads to forms of counseling through social media with the aim of stimulating the public for the socialization of religious moderation

Marginalizing the Mainstream: How Social Media Privilege Political Information

The following reports on research undertaken concerning the “misinformation problem” on social media during the run-up to the U.S. presidential elections in 2020. Employing techniques borrowed from data journalism, it develops a form of cross-platform analysis that is attuned to both commensurability as well as platform specificity. It analyses the most engaged-with or top-ranked political content on seven online platforms: TikTok, 4chan, Reddit, Twitter, Facebook, Instagram and Google Web Search. Discussing the extent to which social media platforms marginalize mainstream media and mainstream the fringe, the analyses found that TikTok parodies mainstream media, 4chan and Reddit dismiss it and direct users to alternative influencer networks and extreme YouTube content. Twitter prefers the hyperpartisan over it. Facebook’s “fake news” problem also concerns declining amounts of mainstream media referenced. Instagram has influencers (rather than, say, experts) dominating user engagement. By comparison, Google Web Search buoys the liberal mainstream (and sinks conservative sites), but generally gives special interest sources, as they were termed in the study, the privilege to provide information rather than official sources. The piece concludes with a discussion of source and “platform criticism”, concerning how online platforms are seeking to filter the content that is posted or found there through increasing editorial intervention. These “editorial epistemologies”, applied especially around COVID-19 keywords, are part of an expansion of so-called content moderation to what I call “serious queries”, or keywords that return official information. Other epistemological strategies for editorially moderating the misinformation problem are also treated