Artifact usefulness and usage in sensemaking handoffs

The complexities of sensemaking suggest that collaboration should be difficult, requiring a rich ecology of collaboration support. This can be a problem for handoff sensemaking, where one person must continue where another has left off, sometimes with only material artifacts as the basis of the handoff. A detailed analysis of essential attributes of sensemaking tasks, and elements identified in the computer supported collaborative work literature were combined to yield insight into handoff sensemaking and guide empirical work. A lab-study showed that handoffs relying only on artifacts from previous sensemaking could be successful. The lab studies also indicated timing and quality affects on the sensemaking handoffs, with different quality materials used differently, and early efforts possibly being particularly difficult to hand off. Design of support for sensemaking handoffs will have to take such effects into account.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/78323/1/1450460219_ftp.pd

Sensemaking Handoffs: Why? How? and When?

Sensemaking tasks are challenging and typically involve collecting, organizing and understanding information. Sensemaking often involves a handoff where a subsequent recipient picks up work done by a provider. Sensemaking handoffs are very challenging because handoffs introduce discontinuity in sensemaking. This dissertation attempts to explore various factors involved in sensemaking handoffs. This work drew on existing literature on sensemaking to propose five sensemaking task attributes: representation novelty required, encoding difficulty, broader applicability, representation search space and subtask interdependence. These attributes capture what makes sensemaking difficult and also help in choosing tasks to study sensemaking as well as modifying laboratory tasks so that they involve more sensemaking. Synthesizing existing literature on collaboration, the dissertation identified important elements in a sensemaking handoff: intent to collaborate, common ground, shared space, awareness, additional communication and handoff artifacts. These make up an ecology that helps deal with challenges of sensemaking expressed by the attributes of sensemaking tasks. A study of sensemaking handoffs in computer-support helpdesks found that sensemaking handoffs could be successful, especially when various collaboration elements complement handoff materials. The study also raised questions about the quality and utility of handoff material from incomplete sensemaking, and about the timing of handoffs. Three lab-studies conducted in the dissertation provided insights regarding the role of artifacts in sensemaking handoffs. The first study confirmed that handoff can be as effective as simultaneous collaboration. The second lab-study suggests that the quality of the handed-off material was important. Poor quality material seemed to be used at different times and in different ways from good quality material. The third lab-study found that available structure in the form of websites as well as handoff artifacts can have an effect on sensemaking. When external structure was available people adapted and used it early on. People appropriated structure sooner from the handoff artifacts when structure was not easily available externally, as compared to when structure was easily available externally. Artifact maturity was also found to have an effect; artifacts from late stages that were placeholders for structures in a task were used more often and were rated higher by the recipients.Ph.D.InformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/75865/1/nsharma_1.pd

Using Google Docs to Support Work Flow Management in Teams of Engineering Students

The purpose of the present study was to investigate how teams of engineering students integrated Google Docs to support their workflow management process. ABET criteria dictate that engineering students need to learn how to work together and practice effective ways of communication. Learning how to work well as a team is linked to the development of positive interdependence, which is at the core of the cooperative learning model and is based on social interdependence theory. A “sink or swim together” attitude in students is an important component of a successful teamwork experience (Smith, 1996). One of the important aspects of supporting interdependence in teams is to provide multiple opportunities for interaction in and outside the classroom

Supporting Situation Awareness and Decision Making in Weather Forecasting

Weather forecasting is full of uncertainty, and as in domains such as air traffic control or medical decision making, decision support systems can affect a forecaster’s ability to make accurate and timely judgments. Well-designed decision aids can help forecasters build situation awareness (SA), a construct regarded as a component of decision making. SA involves the ability to perceive elements within a system, comprehend their significance, and project their meaning into the future in order to make a decision. However, how SA is affected by uncertainty within a system has received little attention. This tension between managing uncertainty, situation assessment, and the impact that technology has on the two, is the focus of this dissertation. To address this tension, this dissertation is centered on the evaluation of a set of coupled models that integrate rainfall observations and hydrologic simulations, coined “the FLASH system” (Flooded Locations and Simulated Hydrographs project). Prediction of flash flooding is unique from forecasting other weather-related threats due to its multi-disciplinary nature. In the United States, some weather forecasters have limited hydrologic forecasting experience. Unlike FLASH, current flash flood forecasting tools are based upon rainfall rates, and with the recent expansion into coupled rainfall and hydrologic models, forecasters have to learn quickly how to incorporate these new data sources into their work. New models may help forecasters to increase their prediction skill, but no matter how far the technology advances, forecasters must be able to accept and integrate the new tools into their work in order to gain any benefit. A focus on human factors principles in the design stage can help to ensure that by the time the product is transitioned into operational use, the decision support system addresses users’ needs while minimizing task time, workload, and attention constraints. This dissertation discusses three qualitative and quantitative studies designed to explore the relationship between flash flood forecasting, decision aid design, and SA. The first study assessed the effects of visual data aggregation methods on perception and comprehension of a flash flood threat. Next, a mixed methods approach described how forecasters acquire SA and mitigate situational uncertainty during real-time forecasting operations. Lastly, the third study used eye tracking assessment to identify the effects of an automated forecasting decision support tool on SA and information scanning behavior. Findings revealed that uncertainty management in forecasting involves individual, team, and organizational processes. We make several recommendations for future decision support systems to promote SA and performance in the weather forecasting domain

Investigating the impact of building information modeling on collaboration in the architecture, engineering, construction and operations industry

The research work presented in this thesis investigates collaboration in the Architecture, Engineering, Construction and Operations (AECO) industry. More precisely, it investigates the characteristics of collaboration, their dynamics in the context of temporary and permanent organizations as well as the impact that the transition to innovative project delivery approaches, namely Building Information Modeling (BIM), is having on this collaboration. The research was practically motivated through our industrial partners’ desire to better understand the impact of BIM and other innovative project delivery approaches on project outcomes. The research was theoretically motivated by the lack of a clear definition of collaboration in the AECO industry and its seemingly amorphous nature. This scarcity of systematic and structured approaches to investigate collaboration and its outcomes in the literature confirmed this. The research was therefore both exploratory and prescriptive in nature. Its principal aim was to develop an artifact that allows consistent development, management and assessment of innovation enabled collaboration. As such, the research project was conducted using a design science research design. The research process continuously iterated between the development and building of the artifact and its local evaluation in context. In parallel, the artifact was concurrently evaluated to ensure its relevance and maintain the rigor of its development. A critical realist perspective was adopted to frame the epistemic and ontological foundation of knowledge being developed and also contradistinguish the predominantly pragmatic perspective traditionally adopted in design science research. The development and building of the artifact followed a systematic combining methodology. Showing similarities with grounded theory, this particular methodology accepts the a priori framing of knowledge to inform the investigation. It recognizes also that the knowledge held within this frame will evolve as the project progresses and as new insight is gained into the phenomena Under investigation. Mixed-methods of data collection were conducted on two main research sites to inform and support the research project. The first site was that of a large institutional designbuild project located in Edmonton, Alberta. Data collection on this site started in February 2013 and is still being carried out. The data collected on this site allowed an in-depth investigation of collaboration within a temporary project organization having fully implemented BIM. The second site was that of a specialty mechanical contracting small enterprise located in Vancouver, British Columbia. Data collection on this site started in April 2012 and ended in April 2015. The data collected on this site allowed a breadth of investigation into collaboration from an organizational perspective. Data was collected on four other sites to support relevance checks of the artifact being developed. The findings of the work are presented through the artifact, namely the constructs developed to characterize collaboration, a multi-layered model representing the relationships between the constructs and the method of operationalization of this model. The artifact serves to inform, manage and assess BIM-based collaboration in the context of this particular research work, though it could be extended to include other innovative project delivery approaches as future work. The artifact also evokes a substantive theory of collaboration in the AECO industry in the form of alignments amongst constructs developed in the model. Lastly, the artifact is operationalized to investigate the impact of BIM on collaboration in the AECO industry. The evolution of the different constructs and indicators, both measured and perceived, the alignments and misalignments uncovered as well as the outcomes of collaboration are evaluated through the artifact. Furthermore, the evolution of the constructs and the alignments uncovered through the artifact can serve as an indicator of performance within collaborative environments. Thus, the artifact developed in this research project solves the problem that was set out by the industrial partners. It also addresses the gap that was uncovered in the literature with respects to collaboration through innovation. Further work is required to fully evaluate the artifact, however, it is believed that the groundwork to move towards a more systematic and structured investigation into collaboration in the AECO industry has been laid

Designing AI Experiences: Boundary Representations, Collaborative Processes, and Data Tools

Artificial Intelligence (AI) has transformed our everyday interactions with technology through automation, intelligence augmentation, and human-machine partnership. Nevertheless, we regularly encounter undesirable and often frustrating experiences due to AI. A fundamental challenge is that existing software practices for coordinating system and experience designs fall short when creating AI for diverse human needs, i.e., ``human-centered AI'' or HAI. ``AI-first'' development workflows allow engineers to first develop the AI components, and then user experience (UX) designers create end-user experiences around the AI's capabilities. Consequently, engineers encounter end-user blindness when making critical decisions about AI training data needs, implementation logic, behavior, and evaluation. In the conventional ``UX-first'' process, UX designers lack the needed technical understanding of AI capabilities (technological blindness) that limits their ability to shape system design from the ground up. Human-AI design guidelines have been offered to help but neither describe nor prescribe ways to bridge the gaps in needed expertise in creating HAI. In this dissertation, I investigate collaboration approaches between designers and engineers to operationalize the vision for HAI as technology inspired by human intelligence that augments human abilities while addressing societal needs. In a series of studies combining technical HCI research with qualitative studies of AI production in practice, I contribute (1) an approach to software development that blurs rigid design-engineering boundaries, (2) a process model for co-designing AI experiences, and (3) new methods and tools to empower designers by making AI accessible to UX designers. Key findings from interviews with industry practitioners include the need for ``leaky'' abstractions shared between UX and AI designers. Because modular development and separation of concerns fail with HAI design, leaky abstractions afford collaboration across expertise boundaries and support human-centered design solutions through vertical prototyping and constant evaluation. Further, by observing how designers and engineers collaborate on HAI design in an in-lab study, I highlight the role of design `probes' with user data to establish common ground between AI system and UX design specifications, providing a critical tool for shaping HAI design. Finally, I offer two design methods and tool implementations --- Data-Assisted Affinity Diagramming and Model Informed Prototyping --- for incorporating end-user data into HAI design. HAI is necessarily a multidisciplinary endeavor, and human data (in multiple forms) is the backbone of AI systems. My dissertation contributions inform how stakeholders with differing expertise can collaboratively design AI experiences by reducing friction across expertise boundaries and maintaining agency within team roles. The data-driven methods and tools I created provide direct support for software teams to tackle the novel challenges of designing with data. Finally, this dissertation offers guidance for imagining future design tools for human-centered systems that are accessible to diverse stakeholders.PHDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169917/1/harihars_1.pd

A dynamic visual analytics framework for complex temporal environments

Introduction: Data streams are produced by sensors that sample an external system at a periodic interval. As the cost of developing sensors continues to fall, an increasing number of data stream acquisition systems have been deployed to take advantage of the volume and velocity of data streams. An overabundance of information in complex environments have been attributed to information overload, a state of exposure to overwhelming and excessive information. The use of visual analytics provides leverage over potential information overload challenges. Apart from automated online analysis, interactive visual tools provide significant leverage for human-driven trend analysis and pattern recognition. To facilitate analysis and knowledge discovery in the space of multidimensional big data, research is warranted for an online visual analytic framework that supports human-driven exploration and consumption of complex data streams. Method: A novel framework was developed called the temporal Tri-event parameter based Dynamic Visual Analytics (TDVA). The TDVA framework was instantiated in two case studies, namely, a case study involving a hypothesis generation scenario, and a second case study involving a cohort-based hypothesis testing scenario. Two evaluations were conducted for each case study involving expert participants. This framework is demonstrated in a neonatal intensive care unit case study. The hypothesis generation phase of the pipeline is conducted through a multidimensional and in-depth one subject study using PhysioEx, a novel visual analytic tool for physiologic data stream analysis. The cohort-based hypothesis testing component of the analytic pipeline is validated through CoRAD, a visual analytic tool for performing case-controlled studies. Results: The results of both evaluations show improved task performance, and subjective satisfaction with the use of PhysioEx and CoRAD. Results from the evaluation of PhysioEx reveals insight about current limitations for supporting single subject studies in complex environments, and areas for future research in that space. Results from CoRAD also support the need for additional research to explore complex multi-dimensional patterns across multiple observations. From an information systems approach, the efficacy and feasibility of the TDVA framework is demonstrated by the instantiation and evaluation of PhysioEx and CoRAD. Conclusion: This research, introduces the TDVA framework and provides results to validate the deployment of online dynamic visual analytics in complex environments. The TDVA framework was instantiated in two case studies derived from an environment where dynamic and complex data streams were available. The first instantiation enabled the end-user to rapidly extract information from complex data streams to conduct in-depth analysis. The second allowed the end-user to test emerging patterns across multiple observations. To both ends, this thesis provides knowledge that can be used to improve the visual analytic pipeline in dynamic and complex environments