How information visualization systems change users’ understandings of complex data

User-centered evaluations of information systems often focus on the usability of the system rather its usefulness. This study examined how a using an interactive knowledge-domain visualization (KDV) system affected users’ understanding of a domain. Interactive KDVs allow users to create graphical representations of domains that depict important papers, authors, or terms. Interactive KDVs have several potential advantages over other presentation methods, such as making connections explicit, and the ability for users to see the overall structure of the domain.The project examined CiteSpace, an interactive KDV that uses article cocitation analysis and text analysis to create visualizations of important papers and terms in a domain. In this study, participants completed several tasks related to the field of artificial intelligence. Depending on the experimental condition, participants read a review article about the domain, interacted with a CiteSpace visualization containing equivalent information, or both. Participants who neither read the article nor used the KDV system served as a baseline. The participants completed three tasks in which they sorted papers and terms according to their importance and relatedness. The hypotheses predicted that participants who used the KDV, especially in conjunction with the review article, would show a more expert-like understanding of the domain compared to the baseline and to participants who used only the article.The study measured the quality of participants’ understanding by comparing their card sorting responses to benchmark responses obtained from domain experts. Participants who produced judgments of importance and relatedness that were similar to the benchmarks were considered as demonstrating a good understanding of the domain. The card sorting results were analyzed using several statistical techniques, including multidimensional scaling and cluster analysis.The results showed that while participants’ understanding of the domain was influenced by using the KDV, this influence was not in the direction of the benchmarks. The data suggest that a lack of agreement between the benchmarks and the depiction of the field presented in the KDV may have led to these findings. The study discusses several possible reasons for these results and recommends possible changes to KDVs that may increase their usefulness.Ph.D., Information Studies -- Drexel University, 200

Managing sociotechnical risks in infrastructure projects : Sociotechnical Systems (STS) perspectives on systems

As systems become larger, more complex and integrated, the cost of failure increases rapidly, leading to a need for effective risk management tools. However, conventional risk management tools such as the ones based on hazard analysis or accident causation analysis have a narrow focus on either human or technical actors and on single causal chains at one organisational level. This led researchers to introduce the concept of Sociotechnical Systems (STS), involving the interaction of human and non-human technical components. The present study was conducted with the aim of developing ways of applying STS principles and STS-based methods to improve the risk management in large infrastructure projects. Initially, the sets of STS principles for the system design, which had been developed so far, were identified and then integrated and synthesised to produce a list of 20 core STS principles for applying them further in the current study. A comprehensive literature review of the work done in this field since its inception in the 1950s was then conducted, producing a unified list of 103 STS-based methods. These methods were then evaluated for their validity and visibility (occurrence). To identify and analyse major risks in complex infrastructure projects from an STS perspective, an observational case study of a large-scale collaborative design project at Heriot-Watt University was conducted, including running the surveys and interviews with the project participants. The aim was to find out if the presence or absence of the 20 STS principles and 18 associated risk factors affected the performance of the teams. It was found that the team performance was strongly related to the presence or absence of STS principles that was supported by statistically meaningful results of a quantitative analysis. The same STS principles were then applied retrospectively to a second case study, which was the construction of the Edinburgh Tram Network, based on documentary sources and employing the AcciMap and Abstraction Hierarchy (AH) methods. It was concluded that failure to apply these principles and the resulting risks could play a major role in the failure to deliver the project on time and within budget. Finally, a five-phase framework was constructed for STS-based risk management framework of infrastructure projects, with the guideline principles aligning the existing risk management framework with STS theory