Cluster-based Deep Ensemble Learning for Emotion Classification in Internet Memes

Memes have gained popularity as a means to share visual ideas through the Internet and social media by mixing text, images and videos, often for humorous purposes. Research enabling automated analysis of memes has gained attention in recent years, including among others the task of classifying the emotion expressed in memes. In this paper, we propose a novel model, cluster-based deep ensemble learning (CDEL), for emotion classification in memes. CDEL is a hybrid model that leverages the benefits of a deep learning model in combination with a clustering algorithm, which enhances the model with additional information after clustering memes with similar facial features. We evaluate the performance of CDEL on a benchmark dataset for emotion classification, proving its effectiveness by outperforming a wide range of baseline models and achieving state-of-the-art performance. Further evaluation through ablated models demonstrates the effectiveness of the different components of CDEL

Evolving linguistic divergence on polarizing social media

Language change is influenced by many factors, but often starts from synchronic variation, where multiple linguistic patterns or forms coexist, or where different speech communities use language in increasingly different ways. Besides regional or economic reasons, communities may form and segregate based on political alignment. The latter, referred to as political polarization, is of growing societal concern across the world. Here we map and quantify linguistic divergence across the partisan left-right divide in the United States, using social media data. We develop a general methodology to delineate (social) media users by their political preference, based on which (potentially biased) news media accounts they do and do not follow on a given platform. Our data consists of 1.5M short posts by 10k users (about 20M words) from the social media platform Twitter (now “X”). Delineating this sample involved mining the platform for the lists of followers (n = 422M) of 72 large news media accounts. We quantify divergence in topics of conversation and word frequencies, messaging sentiment, and lexical semantics of words and emoji. We find signs of linguistic divergence across all these aspects, especially in topics and themes of conversation, in line with previous research. While US American English remains largely intelligible within its large speech community, our findings point at areas where miscommunication may eventually arise given ongoing polarization and therefore potential linguistic divergence. Our flexible methodology — combining data mining, lexicostatistics, machine learning, large language models and a systematic human annotation approach — is largely language and platform agnostic. In other words, while we focus here on US political divides and US English, the same approach is applicable to other countries, languages, and social media platforms

Evolving linguistic divergence on polarizing social media

Language change is influenced by many factors, but often starts from synchronic variation, where multiple linguistic patterns or forms coexist, or where different speech communities use language in increasingly different ways. Besides regional or economic reasons, communities may form and segregate based on political alignment. The latter, referred to as political polarization, is of growing societal concern across the world. Here we map and quantify linguistic divergence across the partisan left-right divide in the United States, using social media data. We develop a general methodology to delineate (social) media users by their political preference, based on which (potentially biased) news media accounts they do and do not follow on a given platform. Our data consists of 1.5M short posts by 10k users (about 20M words) from the social media platform Twitter (now "X"). Delineating this sample involved mining the platform for the lists of followers (n=422M) of 72 large news media accounts. We quantify divergence in topics of conversation and word frequencies, messaging sentiment, and lexical semantics of words and emoji. We find signs of linguistic divergence across all these aspects, especially in topics and themes of conversation, in line with previous research. While US American English remains largely intelligible within its large speech community, our findings point at areas where miscommunication may eventually arise given ongoing polarization and therefore potential linguistic divergence. Our methodology - combining data mining, lexicostatistics, machine learning, large language models and a systematic human annotation approach - is largely language and platform agnostic. In other words, while we focus here on US political divides and US English, the same approach is applicable to other countries, languages, and social media platforms

Employees on social media: A multi-spokespeople model of CSR communication

Increasing societal and stakeholder expectations, along with easy access to information through social media, means corporations are asked for more information. The traditional approach to CSR communication, with corporations controlling what and how much to share with stakeholders has been restructured by social media, with stakeholders taking control. As legitimacy on social media is created through the positive and negative judgements of stakeholders, corporations must plan how to meet stakeholder demands for information effectively and legitimately, and this includes choosing appropriate spokespeople. Corporations in India have now turned towards their employees as CSR spokespeople. By encouraging employee activity on social media, these corporations are attempting to meet stakeholder demands and generate legitimacy through spokespeople whom stakeholders perceive as equals. This article examines that strategy and discusses its viability of using employees as spokespeople for CSR communication and engagement with stakeholder

WiFi-Based Human Activity Recognition Using Attention-Based BiLSTM

Recently, significant efforts have been made to explore human activity recognition (HAR) techniques that use information gathered by existing indoor wireless infrastructures through WiFi signals without demanding the monitored subject to carry a dedicated device. The key intuition is that different activities introduce different multi-paths in WiFi signals and generate different patterns in the time series of channel state information (CSI). In this paper, we propose and evaluate a full pipeline for a CSI-based human activity recognition framework for 12 activities in three different spatial environments using two deep learning models: ABiLSTM and CNN-ABiLSTM. Evaluation experiments have demonstrated that the proposed models outperform state-of-the-art models. Also, the experiments show that the proposed models can be applied to other environments with different configurations, albeit with some caveats. The proposed ABiLSTM model achieves an overall accuracy of 94.03%, 91.96%, and 92.59% across the 3 target environments. While the proposed CNN-ABiLSTM model reaches an accuracy of 98.54%, 94.25% and 95.09% across those same environments

4th. International Conference on Advanced Research Methods and Analytics (CARMA 2022)

Research methods in economics and social sciences are evolving with the increasing availability of Internet and Big Data sources of information. As these sources, methods, and applications become more interdisciplinary, the 4th International Conference on Advanced Research Methods and Analytics (CARMA) is a forum for researchers and practitioners to exchange ideas and advances on how emerging research methods and sources are applied to different fields of social sciences as well as to discuss current and future challenges. Due to the covid pandemic, CARMA 2022 is planned as a virtual and face-to-face conference, simultaneouslyDoménech I De Soria, J.; Vicente Cuervo, MR. (2022). 4th. International Conference on Advanced Research Methods and Analytics (CARMA 2022). Editorial Universitat Politècnica de València. https://doi.org/10.4995/CARMA2022.2022.1595

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Monitoring and Expressing Opinions on Social Networking Sites – Empirical Investigations based on the Spiral of Silence Theory

Social networking technologies such as Facebook are increasingly used for the exchange of information and opinions on politically and civically relevant issues. Drawing on the spiral of silence theory, this dissertation investigated the psychological mechanisms leading to the formation of opinion climates on social networking platforms. Specifically, this work focused on (a) whether and how users monitor other people’s opinions through these technologies and (b) under which circumstances they are willing to contribute to these opinion climates by voicing their personal viewpoint on these platforms. These two processes were addressed by five empirical studies. Study 1 examined the effects of different opinion cues (available on Facebook) on people’s inferences about public opinion. Results of a two-session experiment showed that individuals’ fear of isolation sharpened their attention toward user-generated comments, which, in turn, affected recipients’ public opinion perceptions. The latter influenced subjects’ opinions and their willingness to participate in social media discussions. Study 2 explored the situational manifestations of people’s fear of isolation and environmental variables as influence factors of people’s outspokenness. Results from qualitative interviews revealed a variety of sanctions people expect from others when voicing a minority opinion and a series of factors such as the size or the relationship to the audience which could exert an impact on one’s willingness to express their opinion. Study 3 further investigated the expectations of sanctions and their explanatory value regarding people’s communication behavior in different situations. Findings from an experiment demonstrated that the expectation of being personally attacked can explain why people are more inclined to express a minority opinion in offline rather than in online communication settings. Study 4 tested whether the publicness of social networking platforms in terms of the size and relational diversity of the audience affect people’s outspokenness. Results from a cross-cultural experiment showed that in Germany, a higher level of publicness of a controversial discussion on Facebook reduced people’s likelihood to express their viewpoint. This pattern, however, was not found in Singapore. Study 5 analyzed whether the relationship to the audience determines people’s likelihood to express their opinion on Facebook. Findings from an experiment revealed no effects of relational closeness to the audience on outspokenness. Instead, people’s certainty about the prevailing opinion climate among the audience increased their willingness to voice their opinion on Facebook. This collection of studies extends the previous state of knowledge by testing the validity of the spiral of silence theory but also pointing to potential boundaries thereof in the context of increasingly popular communication environments.Soziale Netzwerktechnologien wie Facebook werden immer mehr zum Informations- und Meinungsaustausch hinsichtlich politisch und gesellschaftlich relevanter Themen genutzt. Vor diesem Hintergrund untersucht die vorliegende Dissertation auf Basis der Theorie der Schweigespirale die psychologischen Mechanismen, die zur Bildung von Meinungsklimata auf sozialen Netzwerkplattformen führen. Dabei betrachtet diese Arbeit, (a) ob und wie Nutzer/innen anhand dieser Technologien die Meinungen anderer Menschen wahrnehmen und (b) unter welchen Umständen sie bereit sind, zu diesem Meinungsklima beizutragen und ihre Meinung auf diesen Plattformen zu äußern. Diese zwei Prozesse wurden mit Hilfe von fünf empirischen Studien analysiert. Studie 1 untersuchte die Wirkungen von verschiedenen Meinungs-Hinweisreizen (auf Facebook) auf die individuelle Wahrnehmung der öffentlichen Meinung. Ein zweiwelliges Experiment zeigte, dass die dispositionelle Isolationsfurcht die Aufmerksamkeit auf nutzergenerierte Kommentare erhöhte, welche wiederum die Wahrnehmungen des öffentlichen Meinungsklimas beeinflussten. Diese wirkten sich auf die persönliche Meinung der Rezipient/inn/en sowie deren Bereitschaft, sich an dieser thematischen Diskussion auf Facebook zu beteiligen, aus. Studie 2 fokussierte die situativen Erscheinungsformen der Isolationsfurcht und Umgebungsvariablen als Einflussfaktoren der Redebereitschaft. Anhand der Ergebnisse einer qualitativen Interview-Studie ließen sich diverse Sanktionen identifizieren, die Menschen von ihrer Umgebung erwarten, wenn sie eine Minderheitsmeinung kundtun würden, sowie eine Reihe von Faktoren, etwa die Größe oder die Beziehung zum Publikum, die Einfluss auf die Bereitschaft zur Meinungsäußerung nehmen könnten. Studie 3 analysierte weiterhin die Erklärungskraft der erwarteten Sanktionen auf das Kommunikationsverhalten in unterschiedlichen Situationen. Ein Experiment zeigte, dass die Erwartung, von anderen persönlich attackiert zu werden, einen Grund darstellt, warum Menschen eher dazu bereit sind, eine Minderheitsmeinung in der Offline- als in der Online-Kommunikation zu äußern. Studie 4 testete, ob die Öffentlichkeit auf sozialen Netzwerkseiten im Sinne der Größe und Diversität des Publikums die Redebereitschaft von den Nutzer/inne/n beeinflusst. Die Ergebnisse eines kulturvergleichenden Experiments legten offen, dass der höhere Öffentlichkeitsgrad einer kontroversen Diskussion auf Facebook – in Deutschland – die Wahrscheinlichkeit senkt, dass Menschen bei dieser Diskussion ihren Standpunkt zum Thema äußern. Dieses Muster konnte in Singapur nicht festgestellt werden. Studie 5 untersuchte, ob die Beziehung zum Publikum die Redebereitschaft zu einem kontroversen Thema beeinflussen kann. Ein Experiment fand jedoch keinen Effekt der Beziehungsnähe zum Publikum auf die Bereitwilligkeit von Nutzer/innen, ihre Meinung zum Thema auf Facebook kundzutun. Stattdessen erwies sich die verspürte Sicherheit über das wahrgenommene Meinungsklima unter dem entsprechenden Publikum als entscheidend: Je höher diese war, desto eher waren Menschen bereit, ihre Meinung auf Facebook kundzutun. Diese Studienreihe erweitert den aktuellen Forschungsstand, indem sie die Gültigkeit der Theorie der Schweigespirale, aber auch deren Grenzen in zunehmend beliebten Kommunikationsumgebungen aufzeigt

Computational Transformation of the Public Sphere : Theories and Case Studies

This book is an edited collection of MA research paper on the digital revolution of the public and governance. It covers cyber governance in Finland, and the securitization of cyber security in Finland. It investigates the cases of Brexit, the 2016 US presidental election of Donald Trump, the 2017 presidential election of Volodymyr Zelensky, and Brexit. It examines the environmental concerns of climate change and greenwashing, and the impact of digital communication giving rise to the #MeToo and Incel movements. It considers how digitilization can serve to emancipate women through ride-sharing, and how it leads to the question of robot rights. It considers fake news and algorithmic governance with respect to case studies of the Chinese social credit system, the US FICO credit score, along with Facebook, Twitter, Cambridge Analytica and the European effort to regulate and protect data usage.Non peer reviewe