Modeling forward base camps as complex adaptive sociotechnical systems

Work for this thesis focuses on managing complexity within complex adaptive sociotechnical systems by using model based systems engineering and virtual engineering tools. The hypothesis of the work is that integrated virtual models can be used to increase the understanding of these complex adaptive sociotechnical systems, resulting in a reduction in the perceived complexity. This was tested by the use of a two factor survey given to experts of a system (the customer and members of the model design team) and to a target user-group. This group received a demonstration and had hands on experience with a preliminary model of the same system. Results of the survey show that new system designers using an integrated virtual modeling tool view the system as less complex than experts involved with designing the same system without using a tool. Further data is required to support this conclusion, and a plan for gathering more data is described. The application of this method to an emergency response system is then discussed to show how it can be applied to other complex sociotechnical systems and guidelines for applying this methodology are proposed

An analytic framework to assess organizational resilience

Background: Resilience Engineering is a paradigm for safety management that focuses on coping with complexity to achieve success, even considering several conflicting goals. Modern socio-technical systems have to be resilient to comply with the variability of everyday activities, the tight-coupled and underspecified nature of work and the nonlinear interactions among agents. At organizational level, resilience can be described as a combination of four cornerstones: monitoring, responding, learning and anticipating. Methods: Starting from these four categories, this paper aims at defining a semi-quantitative analytic framework to measure organizational resilience in complex socio-technical systems, combining the Resilience Analysis Grid (RAG) and the Analytic Hierarchy Process (AHP). Results: This paper presents an approach for defining resilience abilities of an organization, creating a structured domain-dependent framework to define a resilience profile at different levels of abstraction, to identify weaknesses and strengths of the system and thus potential actions to increase system’s adaptive capacity. An illustrative example in an anaesthesia department clarifies the outcomes of the approach. Conclusions: The outcome of the RAG, i.e. a weighted set of probing questions, can be used in different domains, as a support tool in a wider Safety-II oriented managerial action to bring safety management into the core business of the organization

Design, Implementation, and Evaluation of a User Training Program for Integrating Health Information Technology into Clinical Processes

Health information technology (IT) implementation can be costly, and remains a challenging problem with mixed outcomes on patient safety and quality of care. Systems engineering and IT management experts have advocated the use of sociotechnical models to understand the impact of health IT on user and organizational factors. Sociotechnical models suggest the need for user-centered implementation approaches, such as user training and support, and focus on processes to mitigate the negative impact and facilitate optimal IT use during training. The training design and development should also follow systematic processes guided by instructional development models. It should take into account of users’ characteristics of learning, and employ scientific training theories to adopt validated methods that facilitate learning and health IT integration. My study aimed to develop and evaluate a scientific model-guided and systematically developed health IT user training program that explicitly mitigate IT negative impact and facilitate optimal use. I used an electronic health record (EHR) as the health IT, and used medication reconciliation as the clinical task. I developed a sociotechnical model to guide analysis of users’ clinical tasks and their IT interaction, and utilized this model to analyze technical aspects of an EHR, and explicitly integrate the EHR into the workflow of a medication reconciliation task. I designed and developed the training program following existing models, and designed cognitive mapping based interventions to facilitate learning and health IT integration. I implemented and evaluated the training program using a controlled experiment with nursing senior baccalaureate students. Evaluation of participants’ training performance showed that the developed training program was effective. The training program improved trainees’ system use competency by comparing trainees’ pre- and post- training performance, i.e., trainees were able to conduct clinical tasks using the EHR correctly and efficiently, and transfer the competency to use another EHR after training. The training also improved trainees’ clinical outcomes by comparing clinical outcomes between the two training conditions, i.e., trainees who learned cognitive mapping were more competent to identify medication discrepancies. This result implied the proposed methodology could be used as an approach to health IT training, and may be generalizable to other clinical tasks, environments, or role-types

Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture

Scholars and practitioners across domains are increasingly concerned with algorithmic transparency and opacity, interrogating the values and assumptions embedded in automated, black-boxed systems, particularly in user-generated content platforms. I report from an ethnography of infrastructure in Wikipedia to discuss an often understudied aspect of this topic: the local, contextual, learned expertise involved in participating in a highly automated social-technical environment. Today, the organizational culture of Wikipedia is deeply intertwined with various data-driven algorithmic systems, which Wikipedians rely on to help manage and govern the "anyone can edit" encyclopedia at a massive scale. These bots, scripts, tools, plugins, and dashboards make Wikipedia more efficient for those who know how to work with them, but like all organizational culture, newcomers must learn them if they want to fully participate. I illustrate how cultural and organizational expertise is enacted around algorithmic agents by discussing two autoethnographic vignettes, which relate my personal experience as a veteran in Wikipedia. I present thick descriptions of how governance and gatekeeping practices are articulated through and in alignment with these automated infrastructures. Over the past 15 years, Wikipedian veterans and administrators have made specific decisions to support administrative and editorial workflows with automation in particular ways and not others. I use these cases of Wikipedia's bot-supported bureaucracy to discuss several issues in the fields of critical algorithms studies, critical data studies, and fairness, accountability, and transparency in machine learning -- most principally arguing that scholarship and practice must go beyond trying to "open up the black box" of such systems and also examine sociocultural processes like newcomer socialization.Comment: 14 pages, typo fixed in v

CSCW and Enterprise 2.0 - Towards an Integrated Perspective

In CSCW we are researching support for collaboration in work groups for several decades now. Web 2.0 and Social Software entered this field from another starting point recently, and quickly expanding towards support for collaboration in enterprises (Enterprise 2.0). However, the interaction between both fields is minimal. In this paper I am trying to contribute to bridging the gap by identifying the core contributions of the two fields, and how they can be integrated or used to the benefit of both fields

STAT-HI: A Socio-Technical Assessment Tool for Health Informatics Implementations

This paper proposes a socio-technical assessment tool (STAT-HI) for health informatics implementations. We explore why even projects allegedly using sound methodologies repeatedly fail to give adequate attention to socio-technical issues, and we present an initial draft of a structured assessment tool for health informatics implementation that encapsulates socio-technical good practice. Further work is proposed to enrich and validate the proposed instrument. This proposal was presented for discussion at a meeting of the UK Faculty of Health Informatics in December 2009

In Search of a “Fair Explanation”: Helping Young People to Consider the Possibilities, Limitations, and Risks of Computer- and Data-Mediated Systems

Significant resources have been directed towards K-12 computing and data education over the past ten years, as part of what has come to be known as the CSforAll initiative. This initiative has focused on raising awareness of computing education among parents and students, developing situated learning progressions that resonate with many different interests and pursuits, training teachers, and addressing issues of underrepresentation in computing among females and racial minorities. In this dissertation, I argue that as the CSforAll initiative continues to expand, it is important for the education community to also reflect on the forms of knowledge that are believed to be essential, and the presumed benefits of computing and data education. Specifically, how might the goal of producing citizens with robust computing and data literacies change what is considered to be fundamental to a computing education; as well as the kinds of contexts in which computing and data science are situated?I use the term sociotechnical literacy to name this vision for computing education, which I define as a broad set of social and technical practices, strategies, ideas, and dispositions that can help people to reason about the computer-mediated systems that shape their everyday lives. As the term suggests, I argue that it is important for learners to engage with technical ideas as well as their social applications and implications. To examine what this might mean for teaching and learning, I describe two design experiments that I conducted with young people (ages 14 – 22). Each approach aimed to make the applications of computing primary (rather than treating applications as the backdrop from which the abstractions of computation are motivated), so that learners could examine some of the specific ways in which data and computing might be directed to particular goals, subject to real possibilities and constraints, and in relation to alternative forms of participation. I examine the possibilities and limitations of each approach. I also analyze some of the assumptions that framed the design experiments – which were naïve, but also reflective of a broader ethos that pervades CSforAll. I reflect on what these studies collectively reveal about the possibilities, limitations, and risks of data and computing, as situated in the lives of young people; as well as what this might mean for helping young people develop a robust sociotechnical literacy. There are very real limits to what can be accomplished with computing and data alone. There are also significant benefits and risks associated with the many sociotechnical systems that shape our lives. As such, I argue that rather than positioning computing education as a remedy to various social ills, we instead offer young people a fair explanation of what computing is and is not capable of, grounded within specific contexts involving real people. I conclude with what this fair explanation might include, and how it might be fostered

Human Performance Contributions to Safety in Commercial Aviation

In the commercial aviation domain, large volumes of data are collected and analyzed on the failures and errors that result in infrequent incidents and accidents, but in the absence of data on behaviors that contribute to routine successful outcomes, safety management and system design decisions are based on a small sample of non- representative safety data. Analysis of aviation accident data suggests that human error is implicated in up to 80% of accidents, which has been used to justify future visions for aviation in which the roles of human operators are greatly diminished or eliminated in the interest of creating a safer aviation system. However, failure to fully consider the human contributions to successful system performance in civil aviation represents a significant and largely unrecognized risk when making policy decisions about human roles and responsibilities. Opportunities exist to leverage the vast amount of data that has already been collected, or could be easily obtained, to increase our understanding of human contributions to things going right in commercial aviation. The principal focus of this assessment was to identify current gaps and explore methods for identifying human success data generated by the aviation system, from personnel and within the supporting infrastructure