Interactive Search and Exploration in Online Discussion Forums Using Multimodal Embeddings

In this paper we present a novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users. It allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting. Our approach is based on novel multimodal representations of users, words and concepts, which we simultaneously learn by deploying a general-purpose neural embedding model. We show these representations to be useful not only for categorizing users, but also for automatically generating user and community profiles. Inspired by traditional summarization approaches, we create the profiles by selecting diverse and representative content from all available modalities, i.e. the text, image and user modality. The usefulness of the approach is evaluated using artificial actors, which simulate user behavior in a relevance feedback scenario. Multiple experiments were conducted in order to evaluate the quality of our multimodal representations, to compare different embedding strategies, and to determine the importance of different modalities. We demonstrate the capabilities of the proposed approach on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter, which are particularly interesting due to the high semantic level of the discussions they feature

Interaktive Kennzeichnung großer multimedialer Nachrichten-Korpora

Heutzutage werden in immer mehr Bereichen große Mengen an Informationen gesammelt und veröffentlicht. Nachrichtenagenturen weltweit veröffentlichen Nachrichtensendungen zu jeglichen Ereignissen. Das Auftreten vieler Ereignisse und Themen erstreckt sich dabei über einen längeren Zeitraum und somit bietet sich die Möglichkeit an, durch eine entsprechende Visualisierung den Lebenszyklus und die Dynamik der Themen zu Untersuchen. Allerdings ist hierfür eine Unterteilung der Nachrichtensendungen in die einzelnen Berichte und einer Kategorisierung dieser von Vorteil. Der Prozess diese zu kategorisieren ist jedoch sehr langsam und mühsam. Diese Arbeit befasst sich nun mit einem Visuellen Ansatz dem Benutzer eine schnelle und effektivere Lösung zur Kategorisierung und Kennzeichnung von solchen Nachrichten-Korpora bereitzustellen

A framework for hierarchical time-oriented data visualisation

The paradigm of exploratory data analysis advocates the use of multiple perspectives to formulate hypotheses on the data. This thesis presents a framework to support it through the use of interactive hierarchical visualisations for the exploration of temporal data. The research that leads to the framework involves investigating what are the conventional interactive techniques for temporal data, how they can be combined with hierarchical methods and which are the conceptual transformations that enable navigating between multiple perspectives. The aim of the research is to facilitate the design of interactive visualisations based on the use of granularities or units of time, which hide or reveal processes at various scales and is a key aspect of temporal data. Characteristics of granularities are suitable for hierarchical visualisations as evidenced in the literature. However, current conceptual models and frameworks lack means to incorporate characteristics of granularities as an integral part of visualisation design. The research addresses this by combining features of hierarchical and time-oriented visualisations and enabling systematic re-configuration of visualisations. Current techniques for visualising temporal data are analysed and specified at previously unsupported levels by breaking down visual encodings into decomposed layers, which can be arranged and recombined through hierarchical composition methods. Afterwards, the transformations of the properties of temporal data are defined by drawing from the interactions found in the literature and formalising them as a set of conceptual operators. The complete framework is introduced by combining the different components that form it and enable specifying visual encodings, hierarchical compositions and the temporal transformations. A case study then demonstrates how the framework can be used and its benefits for evaluating analysis strategies in visual exploration

Comparing personal image collections with PICTuReVis

Digital image collections contain a wealth of information, which for instance can be used to trace illegal activities and investigate criminal networks. We present a method that enables analysts to reveal relations among people, based on the patterns in their collections. Similar temporal and spatial patterns can be found using a parameterized algorithm, visualization is used to choose the right parameters and to inspect the patterns found. The visualization shows relations between image properties: the person it belongs to, the concepts in the image, its time stamp and location. We demonstrate the method with image collections of 10, 000 people containing 460, 000 images in total.\u3cbr/\u3