Beyond Words: Analyzing Social Media with Text and Images

People express their opinions and experiences through text and images in social media platforms. Analyzing social media content has several applications in natural language processing such as sentiment analysis, hate speech detection, fact checking and sarcasm detection. Combining text and images from social media posts is challenging due to weak visual-text relationships. For instance, a post with the text: Feeling on top of the world after acing my final exams! and a picture of a group of friends at the beach. The image and the text are weakly related as the image does not directly align with the academic context, potentially leading to confusion or misinterpretation of the intended message. Thus, effectively modeling text and images from social media posts is crucial for advancing natural language understanding. This thesis proposes a number of new challenging multimodal classification tasks: point-of-interest (POI) type prediction, political advertisements analysis, and influencer content analysis. First, we introduce POI type prediction which consists of inferring the type of location from which a social media message was posted such as a park or a restaurant. This task is relevant to study a place's identity and has applications such as POI visualization and recommendation. Second, we analyze political advertisements by introducing two new datasets containing political ads labeled by the sponsor's ideology (conservative, liberal), and the sponsor type (political party, third party); and we experiment with multimodal models for advertisement classification. Analyzing political ads is important for researching the characteristics of online campaigns (e.g. voter targeting, non-party campaigns and misinformation) on a large scale. Next, we perform an extensive analysis of influencer content including multimodal approaches for identifying commercial posts, i.e., content that is monetized. Automatically detecting influencer commercial posts is of utmost importance for addressing issues related to transparency and regulatory compliance, such as misleading advertising. Finally, this thesis also presents novel methods for tackling the challenges of modeling text and visual content in social media. We propose two auxiliary losses, Image-Text Contrastive which encourages the model to capture the underlying dependencies in multimodal posts; and Image-Text Matching to enable visual and language alignment

Forensic science in combat of human trafficking

Although Forensic Science has become a crucial part of the investigation of many types of crime, the low number of scientific publications on the usage of Forensic Science to eliminate Human Trafficking or to speed up crime investigation, has given rise to the idea of conducting research on the role of Forensic Science in the investigation of Human Trafficking cases. The following literature review aims at judging the current importance of Forensic Science in solving and preventing Human Trafficking cases, at gathering ideas for the introduction of novel techniques and at identifying gaps of research within this field. For this purpose, a wider view, also addressing socio-economic topics, was applied

Gaining Insight into Determinants of Physical Activity using Bayesian Network Learning

Contains fulltext : 228326pre.pdf (preprint version ) (Open Access) Contains fulltext : 228326pub.pdf (publisher's version ) (Open Access)BNAIC/BeneLearn 202

From Machine to Musical Instrument: The Life and Workings of the Metrostyle Pianola

Media histories of music often frame technological innovation in the early twentieth century within a general zeal for automated musical reproduction. The engineering efforts of the Aeolian Company and its Pianola counter such narratives by fostering active music-making rather than passive listening. As a pneumatically powered attachment to a piano, the Pianola was initially limited to reproducing strictly mechanical renditions of music from perforated paper rolls. But the invention of the Metrostyle in 1903, a hand lever to achieve tempo-specific effects, significantly refined the musical capacities of the instrument. It allowed for inscribing onto the music rolls authoritative performance instructions that could be enacted by the player. Revisiting the various places that the Metrostyle Pianola inhabited, from the manufacturing site to the concert hall and the bourgeois living room, I illuminate the different sociocultural relationships and musical experiences that it mediates. By relegating certain tasks of conventional piano-playing to the mechanical workings inside the instrument, the Pianola was marketed as facilitating simplified music-making in ever wider parts of society. The Metrostyle annotations served as a pedagogical device for instructing novice players in principles of nuanced and tasteful interpretation. My analysis exposes the reciprocal relationships between the instrument and its human players, from attempts to adapt the physical interface to human physiologies, to the ways in which the instrument, in turn, imposes certain mechanistic affordances on its players.European Research Council Starting Gran

Passion for the Real Through Snuff Film in Bret Easton Ellis's American Psycho

Alain Badiou defines the twentieth century in terms of "the passion for the real". The aim of this paper is to discuss how the term is materialized through the use of snuff film in Bret Easton Ellis's American Psycho, and how scoff film functions as a narrative apparatus in the novel

31th International Conference on Information Modelling and Knowledge Bases

Information modelling is becoming more and more important topic for researchers, designers, and users of information systems.The amount and complexity of information itself, the number of abstractionlevels of information, and the size of databases and knowledge bases arecontinuously growing. Conceptual modelling is one of the sub-areas ofinformation modelling. The aim of this conference is to bring together experts from different areas of computer science and other disciplines, who have a common interest in understanding and solving problems on information modelling and knowledge bases, as well as applying the results of research to practice. We also aim to recognize and study new areas on modelling and knowledge bases to which more attention should be paid. Therefore philosophy and logic, cognitive science, knowledge management, linguistics and management science are relevant areas, too. In the conference, there will be three categories of presentations, i.e. full papers, short papers and position papers

Spectacularly Binocular: Exploiting Binocular Luster Effects for HCI Applications

Ph.DDOCTOR OF PHILOSOPH