Casual Information Visualization on Exploring Spatiotemporal Data

The goal of this thesis is to study how the diverse data on the Web which are familiar to everyone can be visualized, and with a special consideration on their spatial and temporal information. We introduce novel approaches and visualization techniques dealing with different types of data contents: interactively browsing large amount of tags linking with geospace and time, navigating and locating spatiotemporal photos or videos in collections, and especially, providing visual supports for the exploration of diverse Web contents on arbitrary webpages in terms of augmented Web browsing

Visual Event Cueing in Linked Spatiotemporal Data

abstract: The media disperses a large amount of information daily pertaining to political events social movements, and societal conflicts. Media pertaining to these topics, no matter the format of publication used, are framed a particular way. Framing is used not for just guiding audiences to desired beliefs, but also to fuel societal change or legitimize/delegitimize social movements. For this reason, tools that can help to clarify when changes in social discourse occur and identify their causes are of great use. This thesis presents a visual analytics framework that allows for the exploration and visualization of changes that occur in social climate with respect to space and time. Focusing on the links between data from the Armed Conflict Location and Event Data Project (ACLED) and a streaming RSS news data set, users can be cued into interesting events enabling them to form and explore hypothesis. This visual analytics framework also focuses on improving intervention detection, allowing users to hypothesize about correlations between events and happiness levels, and supports collaborative analysis.Dissertation/ThesisMasters Thesis Computer Science 201

Using treemaps for variable selection in spatio-temporal visualisation

We demonstrate and reflect upon the use of enhanced treemaps that incorporate spatial and temporal ordering for exploring a large multivariate spatio-temporal data set. The resulting data-dense views summarise and simultaneously present hundreds of space-, time-, and variable-constrained subsets of a large multivariate data set in a structure that facilitates their meaningful comparison and supports visual analysis. Interactive techniques allow localised patterns to be explored and subsets of interest selected and compared with the spatial aggregate. Spatial variation is considered through interactive raster maps and high-resolution local road maps. The techniques are developed in the context of 42.2 million records of vehicular activity in a 98 km(2) area of central London and informally evaluated through a design used in the exploratory visualisation of this data set. The main advantages of our technique are the means to simultaneously display hundreds of summaries of the data and to interactively browse hundreds of variable combinations with ordering and symbolism that are consistent and appropriate for space- and time- based variables. These capabilities are difficult to achieve in the case of spatio-temporal data with categorical attributes using existing geovisualisation methods. We acknowledge limitations in the treemap representation but enhance the cognitive plausibility of this popular layout through our two-dimensional ordering algorithm and interactions. Patterns that are expected (e.g. more traffic in central London), interesting (e.g. the spatial and temporal distribution of particular vehicle types) and anomalous (e.g. low speeds on particular road sections) are detected at various scales and locations using the approach. In many cases, anomalies identify biases that may have implications for future use of the data set for analyses and applications. Ordered treemaps appear to have potential as interactive interfaces for variable selection in spatio-temporal visualisation. Information Visualization (2008) 7, 210-224. doi: 10.1057/palgrave.ivs.950018

Visual Text Analysis in Digital Humanities

In 2005, Franco Moretti introduced Distant Reading to analyse entire literary text collections. This was a rather revolutionary idea compared to the traditional Close Reading, which focuses on the thorough interpretation of an individual work. Both reading techniques are the prior means of Visual Text Analysis. We present an overview of the research conducted since 2005 on supporting text analysis tasks with close and distant reading visualizations in the digital humanities. Therefore, we classify the observed papers according to a taxonomy of text analysis tasks, categorize applied close and distant reading techniques to support the investigation of these tasks and illustrate approaches that combine both reading techniques in order to provide a multi-faceted view of the textual data. In addition, we take a look at the used text sources and at the typical data transformation steps required for the proposed visualizations. Finally, we summarize collaboration experiences when developing visualizations for close and distant reading, and we give an outlook on future challenges in that research area

Entity-Based Insight Discovery in Visual Data Exploration

Visual data exploration (VDE) allows the human to get insight into the data via interaction with visual depictions of that data. Despite the state-of-the-art visualization design models and evaluation methods proposed to support VDE, the community still lacks an understanding of interaction design in visualization and how users extract insight through interacting with the data. This research aims to address these two challenges. For interaction design, a literature review reveals that a lack of actionability hinders the application of existing visualization design methods. To address this challenge, this research proposes an approach abstracting data to entities and designing entity-based interactions to achieve the higher-level interaction goals. Three case studies, i.e., interacting with information facets to support fluid exploratory search, interacting with drug-target relations for insight discovery and sharing, and supporting insight externalization through references to visualization components, demonstrate the applicability of this approach in practice. The three cases detail how the approach could address the design requirements derived from related work to fulfill the various task goals following the nested model of visualization design and the resulting designs’ transferability to other datasets. Reflecting on the case studies, we provide design guidelines to help improve the entity-based interaction design. To understand the insight generation process of VDE, we present two user studies asking users to explore a visualization tool and externalize insights by inputting notes. We logged user interactions and characterized collected insights for correlation and prediction analysis. Correlation analysis of the first study showed that exploration actions tended to relate to unexpected insights; the drill-down interaction pattern could lead to insights with higher domain values. Besides asking users to input notes as insights, the second study enabled users to refer to relevant entities (visualization components and prior notes) to assist their narration. Results showed evidence that entity references provided better predictions than interactions on insight characteristics (category, overview versus detail, and using prior knowledge). We discuss study limitations and results’ implications on knowledge-assisted visualization, such as supporting insight recommendations.Visuaalinen datan tutkiminen antaa ihmiselle mahdollisuuden löytää uutta tietämystä datasta vuorovaikutuksessa tästä datasta tehtyjen visuaalisten kuvausten kanssa. Vaikka visuaalista datan tutkimista tukemaan on ehdotettu erilaisia visualisoinnin suunnittelumalleja ja arviointimenetelmiä, alan yhteisöltä puuttuu silti ymmärrystä siitä, kuinka visualisointiin liittyvää vuorovaikutusta pitäisi suunnitella ja kuinka käyttäjät voivat suodattaa uutta tietämystä olemalla vuorovaikutuksessa datan kanssa. Tässä työssä pyritään vastaamaan näihin kahteen haasteeseen. Kirjallisuuden mukaan olemassa olevien visualisoinnin suunnittelumenetelmien soveltamista vuorovaikutuksen suunnitteluun estää niiden toimivuuden puute. Tähän haasteeseen vastaamiseksi tässä työssä ehdotetaan korkeamman tason vuorovaikutustavoitteiden saavuttamiseksi lähestymistapaa, jossa data abstrahoidaan kokonaisuuksiksi eli entiteeteiksi ja jossa vuorovaikutus suunnitellaan sitten näihin entiteetteihin pohjautuen. Tämän lähestymistavan soveltuvuutta käytäntöön esitellään kolmen eri tapaustutkimuksen kautta. Nämä kolme tapaustutkimusta liittyvät erilaisiin tietoluokkiin liittyvän vuorovaikutuksen hyödyntämiseen sujuvassa tutkivassa tiedonhaussa, lääkkeiden ja niiden vaikutuskohteiden välisiin suhteisiin kohdistuvan vuorovaikutuksen hyödyntämiseen tietämyksen etsimisessä ja jakamisessa sekä visuaalisiin komponentteihin liittyvien viitteiden hyödyntämiseen tietämyksen ulkoistamisen tukemisessa. Tapaustutkimukset osoittavat, kuinka lähestymistavassa voidaan hyödyntää aiemmasta tutkimuksesta johdettuja suunnitteluvaatimuksia ja täyttää erilaiset tehtävätavoitteet noudattamalla visualisoinnin suunnittelun sisäkkäismallia ja tuloksena syntyneiden suunnitelmien siirrettävyyttä muihin datajoukkoihin. Näiden tapaustutkimusten pohjalta esitämme suunnitteluohjeita, jotka auttavat parantamaan entiteettipohjaista vuorovaikutuksen suunnittelua. Jotta voisimme ymmärtää tietämyksen luontiprosessia visuaalisen datan tutkimisessa, esittelemme kaksi käyttäjätutkimusta, joissa käyttäjiä pyydettiin käyttämään annettua visualisointityökalua ja tekemään muistiinpanoja löytämästään tietämyksestä. Käyttäjien toiminnot talletettiin, ja heidän keräämäänsä tietämystä kuvailtiin korrelaatio- ja ennusteanalyysiä varten. Ensimmäisen tutkimuksen korrelaatioanalyysi osoitti, että käyttäjien tutkimistoiminnot liittyivät useimmiten odottamattoman tietämyksen löytämiseen; porautuva vuorovaikutustapa saattoi johtaa korkeamman tason tietämyksen löytämiseen. Sen lisäksi, että käyttäjiä pyydettiin tekemään muistiinpanoja löydetystä tietämyksestä, toisessa tutkimuksessa käyttäjät pystyivät myös viittaamaan asiaankuuluviin entiteetteihin (visualisointikomponentteihin ja aiempiin muistiinpanoihin) ja näin helpottamaan toiminnastaan kertomista. Tulokset osoittivat, että entiteettiviittaukset johtivat parempiin ennustuksiin kuin vuorovaikutus, joka liittyi pelkästään tietämyksen ominaisuuksiin (luokka, yleiskuva vs. yksityiskohdat sekä aiemman tietämyksen käyttö). Työssä pohditaan myös tutkimusten rajoituksia sekä tutkimustulosten vaikutusta tietämykseen pohjautuvaan visualisointiin, kuten esimerkiksi tietämyssuositusten tukemiseen

Close and Distant Reading Visualizations for the Comparative Analysis of Digital Humanities Data

Traditionally, humanities scholars carrying out research on a specific or on multiple literary work(s) are interested in the analysis of related texts or text passages. But the digital age has opened possibilities for scholars to enhance their traditional workflows. Enabled by digitization projects, humanities scholars can nowadays reach a large number of digitized texts through web portals such as Google Books or Internet Archive. Digital editions exist also for ancient texts; notable examples are PHI Latin Texts and the Perseus Digital Library. This shift from reading a single book “on paper” to the possibility of browsing many digital texts is one of the origins and principal pillars of the digital humanities domain, which helps developing solutions to handle vast amounts of cultural heritage data – text being the main data type. In contrast to the traditional methods, the digital humanities allow to pose new research questions on cultural heritage datasets. Some of these questions can be answered with existent algorithms and tools provided by the computer science domain, but for other humanities questions scholars need to formulate new methods in collaboration with computer scientists. Developed in the late 1980s, the digital humanities primarily focused on designing standards to represent cultural heritage data such as the Text Encoding Initiative (TEI) for texts, and to aggregate, digitize and deliver data. In the last years, visualization techniques have gained more and more importance when it comes to analyzing data. For example, Saito introduced her 2010 digital humanities conference paper with: “In recent years, people have tended to be overwhelmed by a vast amount of information in various contexts. Therefore, arguments about ’Information Visualization’ as a method to make information easy to comprehend are more than understandable.” A major impulse for this trend was given by Franco Moretti. In 2005, he published the book “Graphs, Maps, Trees”, in which he proposes so-called distant reading approaches for textual data that steer the traditional way of approaching literature towards a completely new direction. Instead of reading texts in the traditional way – so-called close reading –, he invites to count, to graph and to map them. In other words, to visualize them. This dissertation presents novel close and distant reading visualization techniques for hitherto unsolved problems. Appropriate visualization techniques have been applied to support basic tasks, e.g., visualizing geospatial metadata to analyze the geographical distribution of cultural heritage data items or using tag clouds to illustrate textual statistics of a historical corpus. In contrast, this dissertation focuses on developing information visualization and visual analytics methods that support investigating research questions that require the comparative analysis of various digital humanities datasets. We first take a look at the state-of-the-art of existing close and distant reading visualizations that have been developed to support humanities scholars working with literary texts. We thereby provide a taxonomy of visualization methods applied to show various aspects of the underlying digital humanities data. We point out open challenges and we present our visualizations designed to support humanities scholars in comparatively analyzing historical datasets. In short, we present (1) GeoTemCo for the comparative visualization of geospatial-temporal data, (2) the two tag cloud designs TagPies and TagSpheres that comparatively visualize faceted textual summaries, (3) TextReuseGrid and TextReuseBrowser to explore re-used text passages among the texts of a corpus, (4) TRAViz for the visualization of textual variation between multiple text editions, and (5) the visual analytics system MusikerProfiling to detect similar musicians to a given musician of interest. Finally, we summarize our and the collaboration experiences of other visualization researchers to emphasize the ingredients required for a successful project in the digital humanities, and we take a look at future challenges in that research field

Multidimensional projections for the visual exploration of multimedia data

Multidimensional data analysis is considerably important when dealing with such large and complex datasets. Among the possibilities when analyzing such kind of data, applying visualization techniques can help the user find and understand patters, trends and establish new goals. This thesis aims to present several visualization methods to interactively explore multidimensional datasets aimed from specialized to casual users, by making use of both static and dynamic representations created by multidimensional projections

Revisiting Urban Dynamics through Social Urban Data:

The study of dynamic spatial and social phenomena in cities has evolved rapidly in the recent years, yielding new insights into urban dynamics. This evolution is strongly related to the emergence of new sources of data for cities (e.g. sensors, mobile phones, online social media etc.), which have potential to capture dimensions of social and geographic systems that are difficult to detect in traditional urban data (e.g. census data). However, as the available sources increase in number, the produced datasets increase in diversity. Besides heterogeneity, emerging social urban data are also characterized by multidimensionality. The latter means that the information they contain may simultaneously address spatial, social, temporal, and topical attributes of people and places. Therefore, integration and geospatial (statistical) analysis of multidimensional data remain a challenge. The question which, then, arises is how to integrate heterogeneous and multidimensional social urban data into the analysis of human activity dynamics in cities? To address the above challenge, this thesis proposes the design of a framework of novel methods and tools for the integration, visualization, and exploratory analysis of large-scale and heterogeneous social urban data to facilitate the understanding of urban dynamics. The research focuses particularly on the spatiotemporal dynamics of human activity in cities, as inferred from different sources of social urban data. The main objective is to provide new means to enable the incorporation of heterogeneous social urban data into city analytics, and to explore the influence of emerging data sources on the understanding of cities and their dynamics.  In mitigating the various heterogeneities, a methodology for the transformation of heterogeneous data for cities into multidimensional linked urban data is, therefore, designed. The methodology follows an ontology-based data integration approach and accommodates a variety of semantic (web) and linked data technologies. A use case of data interlinkage is used as a demonstrator of the proposed methodology. The use case employs nine real-world large-scale spatiotemporal data sets from three public transportation organizations, covering the entire public transport network of the city of Athens, Greece.  To further encourage the consumption of linked urban data by planners and policy-makers, a set of webbased tools for the visual representation of ontologies and linked data is designed and developed. The tools – comprising the OSMoSys framework – provide graphical user interfaces for the visual representation, browsing, and interactive exploration of both ontologies and linked urban data.   After introducing methods and tools for data integration, visual exploration of linked urban data, and derivation of various attributes of people and places from different social urban data, it is examined how they can all be combined into a single platform. To achieve this, a novel web-based system (coined SocialGlass) for the visualization and exploratory analysis of human activity dynamics is designed. The system combines data from various geo-enabled social media (i.e. Twitter, Instagram, Sina Weibo) and LBSNs (i.e. Foursquare), sensor networks (i.e. GPS trackers, Wi-Fi cameras), and conventional socioeconomic urban records, but also has the potential to employ custom datasets from other sources. A real-world case study is used as a demonstrator of the capacities of the proposed web-based system in the study of urban dynamics. The case study explores the potential impact of a city-scale event (i.e. the Amsterdam Light festival 2015) on the activity and movement patterns of different social categories (i.e. residents, non-residents, foreign tourists), as compared to their daily and hourly routines in the periods  before and after the event. The aim of the case study is twofold. First, to assess the potential and limitations of the proposed system and, second, to investigate how different sources of social urban data could influence the understanding of urban dynamics. The contribution of this doctoral thesis is the design and development of a framework of novel methods and tools that enables the fusion of heterogeneous multidimensional data for cities. The framework could foster planners, researchers, and policy makers to capitalize on the new possibilities given by emerging social urban data. Having a deep understanding of the spatiotemporal dynamics of cities and, especially of the activity and movement behavior of people, is expected to play a crucial role in addressing the challenges of rapid urbanization. Overall, the framework proposed by this research has potential to open avenues of quantitative explorations of urban dynamics, contributing to the development of a new science of cities

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance