Uncertainty Visualization using Hypothetical Outcome Plots

Project Work presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceData-driven decision-making is crucial for any business. With an increasing interest in Business Intelligence, Data Visualization is playing a major role in decision-making processes. For making a well-informed and accurate decision, it is important to understand uncertainty in the data visualizations. Uncertainty visualizations improve the way users understand the data, as well as the confidence in their conclusions. An important type of uncertainty visualizations is the Hypothetical Outcome Plots (HOPs), which allow the audience to gain an intuitive idea of uncertainty through animated sequences of random draws from a distribution, leading to a more accurate understanding and decision. This document intends to detail a proof-of-concept by carrying out a comparison of static visualization vs. HOPs in terms of efficiency and accuracy of results interpretation for Wayne Enterprises (fictional name) forecasting projects, in particularly the ones related with product launches and product loss of exclusivity. Wayne Enterprises is a world-leading supplier of advanced analytics, technological services and clinical investigation solutions for the life sciences industry. For that objective, it was built two prototypes using Python to support the proof-of-concept execution. A between-group experiment was carried out with 40 members of the German consulting team of Wayne Enterprises, where half answered a survey based on static visualizations and the other half based on HOPs. From this experiment, it is possible to conclude that HOPs can achieve similar results that static visualizations, with people taking the decision in less than half of the time when visualizing a HOP. Thus, it is possible to improve Wayne Enterprises decision-making process by accelerating it with Hypothetical Outcome Plots.A tomada de decisões baseada em dados é crucial para qualquer negócio. Com um interesse crescente em Business Intelligence, a Visualização de Dados está a desempenhar um papel importante nos processos de tomada de decisão. Para se tomar uma decisão bem informada e precisa, é importante compreender a incerteza nas visualizações de dados. As visualizações de incerteza melhoram a forma como os utilizadores compreendem os dados, bem como a confiança nas suas conclusões. Um tipo importante de visualizações de incerteza é o Hypothetical Outcome Plots (HOPs), que permite ao público obter uma ideia intuitiva da incerteza através de sequências animadas de desenhos aleatórios de uma distribuição, conduzindo a uma compreensão e decisão mais precisas. Este documento pretende detalhar uma prova de conceito através da realização de uma comparação entre visualizações estáticas e HOPs em termos de eficiência e exactidão de interpretação de resultados para projectos de forecast da Wayne Enterprises (nome fictício), em particular os relacionados com lançamentos de produtos e perda de exclusividade de produtos. A Wayne Enterprises é um líder mundial de análises avançadas, serviços tecnológicos e soluções de investigação clínica para a indústria das ciências da vida. Para esse objectivo, foram construídos dois protótipos utilizando Python para apoiar a execução da prova de conceito. Foi realizada uma experiência entre grupos com 40 membros da equipa de consultoria alemã da Wayne Enterprises, onde metade respondeu a um inquérito baseado em visualizações estáticas e a outra metade com base em HOPs. A partir desta experiência, é possível concluir que os HOPs podem alcançar resultados semelhantes aos das visualizações estáticas, com as pessoas a tomarem a decisão em menos de metade do tempo quando visualizam um HOP. Por conseguinte, é possível melhorar o processo de tomada de decisão da Wayne Enterprises, acelerando-o com Hypothetical Outcome Plots

Interactive animated visualizations of probabilistic models

Bayesian probabilistic models’ structure (determined by the mathematical relations of the model’s variables) and outputs (i.e., the posterior distributions inferred through Bayesian inference) are complex and difficult to grasp and interprete without specialized knowledge. Various visualizations of probabilistic models exist but it is very little known about whether and how they support users’ comprehension of the models. The aim of this thesis is to investigate whether adding interaction or animation to visual representations of probabilistic models help people better understand the structure of models and interprete the (causal and non-causal) relations of the variables. This research presents a generic pipeline to transform a probabilistic model expressed in a Probabilistic Programming Language (PPL) and associated inference results into a standardized format which can then be automatically translated into an interactive probabilistic models explorer (IPME). IPME provides at-a-glance communication of a model’s structure and uncertainty, and allows interactive exploration of the multi-dimensional prior or posterior MCMC sample space. A collapsible tree-like structure represents the structure of the model in IPME. Each variable is represented by a node that presents graphically the prior or posterior distribution of the variable. Slicing on indexing dimensions or forming conjunctive restrictions on variables by interacting with the distribution visualizations is supported. Each user interaction with the explorer triggers the reestimation and visualization of the model’s uncertainty. This closed-loop exchange of responses between the user and the explorer allows the user to gain a more intuitive comprehension of the model. IPME was designed to enhance informativeness, transparency and explainability and ultimately, the potential of increasing trust in models. This research investigates also whether adding interactive conditioning to classical scatter plot matrices that present samples from the prior distribution of probabilistic models helps users better understand the models, and if there are levels of structural detail and model designs for which it is beneficial. A user study was conducted. The analysis of the collected data showed that interactive conditioning is beneficial in cases of sophisticated model designs and the difference in response time between the interaction and static group becomes less important in higher levels of structural detail. Participants using interactive conditioning were more confident about their responses overall with the effect being stronger in tasks of lower level of detail. This research proposes a pipeline to generate simulated probabilistic data from interven tions applied on causal structures that are expressed in PPLs using probabilistic modeling and Bayesian inference. An automatic visualization tool for visualizing the simulated probabilistic data generated by this pipeline was developed. A user study to evaluate the proposed tool was conducted. How effectively and efficiently people identify the causal model of the presented data and make decisions on interventional experiments when the uncertainty in the simulated data of interventions was presented using static, animated, or interactive visualizations was investigated. The findings suggested that participants were able to identify the causal model of the presented data either given a single intervention or by exploring various interventions. Their performance in identifying sufficient interventions was poor. Participants did not rely on the sufficient interventions to identify the causal model in the case of multi-interventional tasks. They might have relied more on combining information from multiple interventions to draw their conclusions. There were three different visual exploration strategies of the information in the scatter plot matrices which participants followed; roughly 1/3 of them relied on both the scatter and KDE plots, another 1/3 of them relied more on the scatter plots, and the last 1/3 of them relied more on the KDE plots. Those who followed the last strategy had a better performance in identifying the causal model given a specific intervention. Most participants judged the design of the visualization positively with many having mentioned that “it was informative”