Interactive Search and Exploration in Online Discussion Forums Using Multimodal Embeddings

In this paper we present a novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users. It allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting. Our approach is based on novel multimodal representations of users, words and concepts, which we simultaneously learn by deploying a general-purpose neural embedding model. We show these representations to be useful not only for categorizing users, but also for automatically generating user and community profiles. Inspired by traditional summarization approaches, we create the profiles by selecting diverse and representative content from all available modalities, i.e. the text, image and user modality. The usefulness of the approach is evaluated using artificial actors, which simulate user behavior in a relevance feedback scenario. Multiple experiments were conducted in order to evaluate the quality of our multimodal representations, to compare different embedding strategies, and to determine the importance of different modalities. We demonstrate the capabilities of the proposed approach on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter, which are particularly interesting due to the high semantic level of the discussions they feature

Natural Language Understanding and Multimodal Discourse Analysis for Interpreting Extremist Communications and the Re-Use of These Materials Online

This paper reports on a study that is part of a project which aims to develop a multimodal analytical approach for big data analytics, initially in the context of violent extremism. The findings reported here tested the application of natural language processing models to the text of a sample of articles from the online magazines Dabiq and Rumiyah, produced by the Islamic extremist organisation ISIS. For comparison, text of articles found by reverse image search software which re-used the lead images from the original articles in text which either reported on or opposed extremist activities was also analysed. The aim was to explore what insights the natural language processing models could provide to distinguish between texts produced as propaganda to incite violent extremism and texts which either reported on or opposed violent extremism. The results showed that some valuable insights can be gained from such an approach and that these results could be improved through integrating automated analyses with a theoretical approach with analysed language and images in their immediate and social contexts. Such an approach will inform the interpretation of results and will be used in training software so that stronger results can be achieved in the future

Countering Extremists on Social Media:Challenges for Strategic Communication and Content Moderation

Extremist exploitation of social media platforms is an important regulatory question for civil society, government, and the private sector. Extremists exploit social media for a range of reasons-from spreading hateful narratives and propaganda to financing, recruitment, and sharing operational information. Policy responses to this question fit under two headings, strategic communication and content moderation. At the center of both of these policy responses is a calculation about how best to limit audience exposure to extremist narratives and maintain the marginality of extremist views, while being conscious of rights to free expression and the appropriateness of restrictions on speech. This special issue on "Countering Extremists on Social Media: Challenges for Strategic Communication and Content Moderation" focuses on one form of strategic communication, countering violent extremism. In this editorial we discuss the background and effectiveness of this approach, and introduce five articles which develop multiple strands of research into responses and solutions to extremist exploitation of social media. We conclude by suggesting an agenda for future research on how multistakeholder initiatives to challenge extremist exploitation of social media are conceived, designed, and implemented, and the challenges these initiatives need to surmount