Analytic provenance for sensemaking: a research agenda

Sensemaking is a process of find meaning from information, and often involves activities such as information foraging and hypothesis generation. It can be valuable to maintain a history of the data and reasoning involved, commonly known as provenance information. Provenance information can be a resource for “reflection-in-action” during analysis, supporting collaboration between analysts, and help trace data quality and uncertainty through analysis process. Currently, there is limited work of utilizing analytic provenance, which captures the interactive data exploration and human reasoning process, to support sensemaking. In this article, we present and extend the research challenges discussed in a IEEE VIS 2014 workshop in order to provide an agenda for sensemaking analytic provenance

Analytic provenance for sensemaking: A research agenda

Sensemaking is a process of finding meaning from information that often involves activities such as information foraging and hypothesis generation. It can be valuable to maintain a history of the data and reasoning involved. This history, commonly known as provenance information, can be a resource for 'reflection-in-action'' during analysis, supporting collaboration between analysts, and can help trace data quality and uncertainty through the analysis process. Currently, there is limited work on utilizing analytic provenance, which captures the interactive data exploration and human reasoning process, to support sensemaking. This article presents and extends the research challenges discussed in a IEEE VIS 2014 workshop on this topic to provide an agenda for sensemaking analytic provenance

Interaction Design for Mixed-Focus Collaboration in Cross-Device Environments

The proliferation of interactive technologies has resulted in a multitude of form factors for computer devices, such as tablets and phones, and large tabletop and wall displays. Investigating how these devices may be used together as Cross-Device Environments (XDEs) to facilitate collaboration is an active area of research in Human Computer Interaction (HCI) and Computer-Supported Cooperative Work (CSCW). The research community has explored the role of personal and shared devices in supporting group work and has introduced a number of cross-device interaction techniques to enable interaction among devices in an XDE. However, there is little understanding of how the interface design of those techniques may change the way people conduct collaboration, which, in turn, could influence the outcome of the activity. This thesis studies the impact of cross-device interaction techniques on collaborative processes. In particular, I investigated how interface design of cross-device interaction techniques may impact communication and coordination during group work. First, I studied the impact of two specific cross-device interaction techniques on collaboration in an XDE comprised of tablets and a tabletop. The findings confirmed that the choice of interaction techniques mattered when it came to facilitating both independent and joint work periods during group work. The study contributes knowledge towards problematizing the impact of cross-device interaction techniques on collaboration in HCI research. This early work gave rise to deeper questions regarding coordination in cross-device transfer and leveraging that to support the flexibility of work periods in collaborative activities. Consequently, I explored a range of interface design choices that varied the degree of synchronicity in coordinating data transfer across two devices. Additionally, I studied the impact of those interface designs on collaborative processes. My findings resulted in design considerations as well as adapting a synchronicity framework to articulate the impact of cross-device transfer techniques on collaboration. While performing the two research projects, I identified a need for a tool to articulate the impact of specific user interface elements on collaboration. Through a series of case studies, I developed a visual framework that researchers can use as a formative and summative method to understand if a given interaction technique hinders or supports collaboration in the specific task context. I discuss the contributions of my work to the field of HCI, design implications beyond the environments studied, and future research directions to build on and extend my findings

Sensemaking Handoffs: Why? How? and When?

Sensemaking tasks are challenging and typically involve collecting, organizing and understanding information. Sensemaking often involves a handoff where a subsequent recipient picks up work done by a provider. Sensemaking handoffs are very challenging because handoffs introduce discontinuity in sensemaking. This dissertation attempts to explore various factors involved in sensemaking handoffs. This work drew on existing literature on sensemaking to propose five sensemaking task attributes: representation novelty required, encoding difficulty, broader applicability, representation search space and subtask interdependence. These attributes capture what makes sensemaking difficult and also help in choosing tasks to study sensemaking as well as modifying laboratory tasks so that they involve more sensemaking. Synthesizing existing literature on collaboration, the dissertation identified important elements in a sensemaking handoff: intent to collaborate, common ground, shared space, awareness, additional communication and handoff artifacts. These make up an ecology that helps deal with challenges of sensemaking expressed by the attributes of sensemaking tasks. A study of sensemaking handoffs in computer-support helpdesks found that sensemaking handoffs could be successful, especially when various collaboration elements complement handoff materials. The study also raised questions about the quality and utility of handoff material from incomplete sensemaking, and about the timing of handoffs. Three lab-studies conducted in the dissertation provided insights regarding the role of artifacts in sensemaking handoffs. The first study confirmed that handoff can be as effective as simultaneous collaboration. The second lab-study suggests that the quality of the handed-off material was important. Poor quality material seemed to be used at different times and in different ways from good quality material. The third lab-study found that available structure in the form of websites as well as handoff artifacts can have an effect on sensemaking. When external structure was available people adapted and used it early on. People appropriated structure sooner from the handoff artifacts when structure was not easily available externally, as compared to when structure was easily available externally. Artifact maturity was also found to have an effect; artifacts from late stages that were placeholders for structures in a task were used more often and were rated higher by the recipients.Ph.D.InformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/75865/1/nsharma_1.pd

The Influence of Visual Provenance Representations on Strategies in a Collaborative Hand-off Data Analysis Scenario

Conducting data analysis tasks rarely occur in isolation. Especially in intelligence analysis scenarios where different experts contribute knowledge to a shared understanding, members must communicate how insights develop to establish common ground among collaborators. The use of provenance to communicate analytic sensemaking carries promise by describing the interactions and summarizing the steps taken to reach insights. Yet, no universal guidelines exist for communicating provenance in different settings. Our work focuses on the presentation of provenance information and the resulting conclusions reached and strategies used by new analysts. In an open-ended, 30-minute, textual exploration scenario, we qualitatively compare how adding different types of provenance information (specifically data coverage and interaction history) affects analysts' confidence in conclusions developed, propensity to repeat work, filtering of data, identification of relevant information, and typical investigation strategies. We see that data coverage (i.e., what was interacted with) provides provenance information without limiting individual investigation freedom. On the other hand, while interaction history (i.e., when something was interacted with) does not significantly encourage more mimicry, it does take more time to comfortably understand, as represented by less confident conclusions and less relevant information-gathering behaviors. Our results contribute empirical data towards understanding how provenance summarizations can influence analysis behaviors.Comment: to be published in IEEE Vis 202

Supporting Story Synthesis: Bridging the Gap between Visual Analytics and Storytelling

Visual analytics usually deals with complex data and uses sophisticated algorithmic, visual, and interactive techniques. Findings of the analysis often need to be communicated to an audience that lacks visual analytics expertise. This requires analysis outcomes to be presented in simpler ways than that are typically used in visual analytics systems. However, not only analytical visualizations may be too complex for target audience but also the information that needs to be presented. Hence, there exists a gap on the path from obtaining analysis findings to communicating them, which involves two aspects: information and display complexity. We propose a general framework where data analysis and result presentation are linked by story synthesis, in which the analyst creates and organizes story contents. Differently, from the previous research, where analytic findings are represented by stored display states, we treat findings as data constructs. In story synthesis, findings are selected, assembled, and arranged in views using meaningful layouts that take into account the structure of information and inherent properties of its components. We propose a workflow for applying the proposed framework in designing visual analytics systems and demonstrate the generality of the approach by applying it to two domains, social media, and movement analysis

Entity-Based Insight Discovery in Visual Data Exploration

Visual data exploration (VDE) allows the human to get insight into the data via interaction with visual depictions of that data. Despite the state-of-the-art visualization design models and evaluation methods proposed to support VDE, the community still lacks an understanding of interaction design in visualization and how users extract insight through interacting with the data. This research aims to address these two challenges. For interaction design, a literature review reveals that a lack of actionability hinders the application of existing visualization design methods. To address this challenge, this research proposes an approach abstracting data to entities and designing entity-based interactions to achieve the higher-level interaction goals. Three case studies, i.e., interacting with information facets to support fluid exploratory search, interacting with drug-target relations for insight discovery and sharing, and supporting insight externalization through references to visualization components, demonstrate the applicability of this approach in practice. The three cases detail how the approach could address the design requirements derived from related work to fulfill the various task goals following the nested model of visualization design and the resulting designs’ transferability to other datasets. Reflecting on the case studies, we provide design guidelines to help improve the entity-based interaction design. To understand the insight generation process of VDE, we present two user studies asking users to explore a visualization tool and externalize insights by inputting notes. We logged user interactions and characterized collected insights for correlation and prediction analysis. Correlation analysis of the first study showed that exploration actions tended to relate to unexpected insights; the drill-down interaction pattern could lead to insights with higher domain values. Besides asking users to input notes as insights, the second study enabled users to refer to relevant entities (visualization components and prior notes) to assist their narration. Results showed evidence that entity references provided better predictions than interactions on insight characteristics (category, overview versus detail, and using prior knowledge). We discuss study limitations and results’ implications on knowledge-assisted visualization, such as supporting insight recommendations.Visuaalinen datan tutkiminen antaa ihmiselle mahdollisuuden löytää uutta tietämystä datasta vuorovaikutuksessa tästä datasta tehtyjen visuaalisten kuvausten kanssa. Vaikka visuaalista datan tutkimista tukemaan on ehdotettu erilaisia visualisoinnin suunnittelumalleja ja arviointimenetelmiä, alan yhteisöltä puuttuu silti ymmärrystä siitä, kuinka visualisointiin liittyvää vuorovaikutusta pitäisi suunnitella ja kuinka käyttäjät voivat suodattaa uutta tietämystä olemalla vuorovaikutuksessa datan kanssa. Tässä työssä pyritään vastaamaan näihin kahteen haasteeseen. Kirjallisuuden mukaan olemassa olevien visualisoinnin suunnittelumenetelmien soveltamista vuorovaikutuksen suunnitteluun estää niiden toimivuuden puute. Tähän haasteeseen vastaamiseksi tässä työssä ehdotetaan korkeamman tason vuorovaikutustavoitteiden saavuttamiseksi lähestymistapaa, jossa data abstrahoidaan kokonaisuuksiksi eli entiteeteiksi ja jossa vuorovaikutus suunnitellaan sitten näihin entiteetteihin pohjautuen. Tämän lähestymistavan soveltuvuutta käytäntöön esitellään kolmen eri tapaustutkimuksen kautta. Nämä kolme tapaustutkimusta liittyvät erilaisiin tietoluokkiin liittyvän vuorovaikutuksen hyödyntämiseen sujuvassa tutkivassa tiedonhaussa, lääkkeiden ja niiden vaikutuskohteiden välisiin suhteisiin kohdistuvan vuorovaikutuksen hyödyntämiseen tietämyksen etsimisessä ja jakamisessa sekä visuaalisiin komponentteihin liittyvien viitteiden hyödyntämiseen tietämyksen ulkoistamisen tukemisessa. Tapaustutkimukset osoittavat, kuinka lähestymistavassa voidaan hyödyntää aiemmasta tutkimuksesta johdettuja suunnitteluvaatimuksia ja täyttää erilaiset tehtävätavoitteet noudattamalla visualisoinnin suunnittelun sisäkkäismallia ja tuloksena syntyneiden suunnitelmien siirrettävyyttä muihin datajoukkoihin. Näiden tapaustutkimusten pohjalta esitämme suunnitteluohjeita, jotka auttavat parantamaan entiteettipohjaista vuorovaikutuksen suunnittelua. Jotta voisimme ymmärtää tietämyksen luontiprosessia visuaalisen datan tutkimisessa, esittelemme kaksi käyttäjätutkimusta, joissa käyttäjiä pyydettiin käyttämään annettua visualisointityökalua ja tekemään muistiinpanoja löytämästään tietämyksestä. Käyttäjien toiminnot talletettiin, ja heidän keräämäänsä tietämystä kuvailtiin korrelaatio- ja ennusteanalyysiä varten. Ensimmäisen tutkimuksen korrelaatioanalyysi osoitti, että käyttäjien tutkimistoiminnot liittyivät useimmiten odottamattoman tietämyksen löytämiseen; porautuva vuorovaikutustapa saattoi johtaa korkeamman tason tietämyksen löytämiseen. Sen lisäksi, että käyttäjiä pyydettiin tekemään muistiinpanoja löydetystä tietämyksestä, toisessa tutkimuksessa käyttäjät pystyivät myös viittaamaan asiaankuuluviin entiteetteihin (visualisointikomponentteihin ja aiempiin muistiinpanoihin) ja näin helpottamaan toiminnastaan kertomista. Tulokset osoittivat, että entiteettiviittaukset johtivat parempiin ennustuksiin kuin vuorovaikutus, joka liittyi pelkästään tietämyksen ominaisuuksiin (luokka, yleiskuva vs. yksityiskohdat sekä aiemman tietämyksen käyttö). Työssä pohditaan myös tutkimusten rajoituksia sekä tutkimustulosten vaikutusta tietämykseen pohjautuvaan visualisointiin, kuten esimerkiksi tietämyssuositusten tukemiseen

Enabling Collaborative Visual Analysis across Heterogeneous Devices

We are surrounded by novel device technologies emerging at an unprecedented pace. These devices are heterogeneous in nature: in large and small sizes with many input and sensing mechanisms. When many such devices are used by multiple users with a shared goal, they form a heterogeneous device ecosystem. A device ecosystem has great potential in data science to act as a natural medium for multiple analysts to make sense of data using visualization. It is essential as today's big data problems require more than a single mind or a single machine to solve them. Towards this vision, I introduce the concept of collaborative, cross-device visual analytics (C2-VA) and outline a reference model to develop user interfaces for C2-VA. This dissertation covers interaction models, coordination techniques, and software platforms to enable full stack support for C2-VA. Firstly, we connected devices to form an ecosystem using software primitives introduced in the early frameworks from this dissertation. To work in a device ecosystem, we designed multi-user interaction for visual analysis in front of large displays by finding a balance between proxemics and mid-air gestures. Extending these techniques, we considered the roles of different devices–large and small–to present a conceptual framework for utilizing multiple devices for visual analytics. When applying this framework, findings from a user study showcase flexibility in the analytic workflow and potential for generation of complex insights in device ecosystems. Beyond this, we supported coordination between multiple users in a device ecosystem by depicting the presence, attention, and data coverage of each analyst within a group. Building on these parts of the C2-VA stack, the culmination of this dissertation is a platform called Vistrates. This platform introduces a component model for modular creation of user interfaces that work across multiple devices and users. A component is an analytical primitive–a data processing method, a visualization, or an interaction technique–that is reusable, composable, and extensible. Together, components can support a complex analytical activity. On top of the component model, the support for collaboration and device ecosystems comes for granted in Vistrates. Overall, this enables the exploration of new research ideas within C2-VA

Structured Sensemaking of Videographic Information within Dataphoric Space

Attempts to create a structured sensemaking model have proven difficult. Much of the research today has evolved into a cacophony of conceptual models. Many of these sensemaking models have been proposed but not tested. Using structural equations, a unified model of sensemaking was developed and tested. This structured sensemaking model contains five sensemaking constructs: chaos, anchoring, articulation, retrospection, and identity. This model was tested using data collected from 224 educationally focused YouTube videos. The confirmatory factor model developed for this research has a measured Comparative Fit Index of 0.979, a measured Standardized Root Mean Square Residual of 0.078, and a measured Akaike’s Information Criterion of 182.892. The associated structural model has a measured Comparative Fit Index of 0.991, a measured Standardized Root Mean Square Residual of 0.047, and a measured Akaike’s Information Criterion of 131.680. This theory of structured sensemaking supports a) the unification of five sensemaking constructs b) a structured sensemaking framework c) the integration of information theory and d) a reusable sensemaking method. This structured sensemaking framework is the first of its kind