Beyond problem identification: valuing methods in a ‘system usability practice’

Historically, usability evaluation methods (UEMs) have been evaluated on their capability for problem identification. However, the relevance of this approach has been questioned for applied usability work. To investigate alternative explanations of what is important for method use a grounded theory of usability practitioners was developed (9 interviews from the website domain and 13 in the safety-critical domain). The analysis proceeded in bottom-up and top-down stages. The bottom-up stages produced insight from the data in an exploratory and inductive manner. This highlighted the importance of contextual factors and the need for system descriptions: UEM adoption and adaptation cannot be fully understood devoid of context. The top-down stages used Distributed Cognition and Resilience Engineering conceptual frameworks as leverage for exploring the data in a deductive manner. These were chosen for their functional descriptions of systems. To illustrate the importance of context we describe three models: 1) where previous research has highlighted the downstream utility of UEMs we expand the metaphor to consider the landscape through which the stream flows, where the landscape represents the project’s context; 2) where information propagation and transformation in a project is influenced by social, information flow, artefact, physical and evolutionary factors; and 3) where the functional couplings between parts of the system of usability practice can be monitored and managed to positively resonate with each other, thereby improving the performance of the system overall. The concept of ‘Positive Resonance’ is introduced to describe how practitioners adapt to the context to maximise their impact under constrained resources. The functional couplings are described in a functional resonance model of HCI practice. This model is validated by interviewees and other practitioners outside of the study. This research shows that problem identification is limited for valuing UEMs. Instead, functional couplings of UEMs should be considered to improve system performance, which influence UEM adoption and adaptation in practice

Resilience markers for safer systems and organisations

If computer systems are to be designed to foster resilient performance it is important to be able to identify contributors to resilience. The emerging practice of Resilience Engineering has identified that people are still a primary source of resilience, and that the design of distributed systems should provide ways of helping people and organisations to cope with complexity. Although resilience has been identified as a desired property, researchers and practitioners do not have a clear understanding of what manifestations of resilience look like. This paper discusses some examples of strategies that people can adopt that improve the resilience of a system. Critically, analysis reveals that the generation of these strategies is only possible if the system facilitates them. As an example, this paper discusses practices, such as reflection, that are known to encourage resilient behavior in people. Reflection allows systems to better prepare for oncoming demands. We show that contributors to the practice of reflection manifest themselves at different levels of abstraction: from individual strategies to practices in, for example, control room environments. The analysis of interaction at these levels enables resilient properties of a system to be ‘seen’, so that systems can be designed to explicitly support them. We then present an analysis of resilience at an organisational level within the nuclear domain. This highlights some of the challenges facing the Resilience Engineering approach and the need for using a collective language to articulate knowledge of resilient practices across domains

Codifying distributed cognition: a case study of emergency medical dispatch

The theory of distributed cognition is recognised as being relevant to system analysis and design but it has lacked visibility for practice. In this paper I develop a codified method of analysis based on distributed cognition which provides both structure and guidance in the use of the theory. The method developed comprises a systematic exploration and description of three functional levels of a system, namely, the information flow model, physical model, and artefact model. These levels are analytically separate but integrate in modelling the propagation and transformation of information within a system. The approach to developing this method has been exploratory and iterative: developing the understanding of distributed cognition and contextual study literature, with practical application to the London Ambulance Service Central Ambulance Control room context. The application of the method to this context reveals a number of design issues and concerns lending support to its use in these situations. Furthermore, this paper introduces a conception of how distributed cognition can be used to deliberate about potential design scenarios, which is a use of distributed cognition that has been alluded to but has not been explained elsewhere. This paper makes progress in narrowing the gap between distributed cognition theory and practice by adding guidance through a structured codified methodology. The method provides an accessible, practical approach to analysing team based systems using distributed cognition

Exploring the importance of reflection in the control room

While currently difficult to measure or explicitly design for, evidence suggests that providing people with opportunities to reflect on experience must be recognized and valued during safety-critical work. We provide an insight into reflection as a mechanism that can help to maintain both individual and team goals. In the control room, reflection can be task-based, critical for the 'smooth' day-to-day operational performance of a socio-technical system, or can foster learning and organisational change by enabling new understandings gained from experience. In this position paper we argue that technology should be designed to support the reflective capacity of people. There are many interaction designs and artefacts that aim to support problem-solving, but very few that support self-reflection and group reflection. Traditional paradigms for safety-critical systems have focussed on ensuring the functional correctness of designs, minimising the time to complete tasks, etc. Work in the area of user experience design may be of increasing relevance when generating artefacts that aim to encourage reflection

Observation of O+ 4P-4D0 lines in proton aurora over Svalbard

Spectra of a proton aurora event show lines of O+ 4P-4D0 multiplet (4639–4696 Å) enhanced relative to the N2 +1N(0,2) compared to normal electron aurora. Conjugate satellite particle measurements are used as input to electron and proton transport models, to show that p/H precipitation is the dominant source of both the O+ and N2 +1N emissions. The emission cross-section of the multiplet in p collisions with O and O2 estimated from published work does not explain the observed O+ brightness, suggesting a higher emission cross-section for low energy p impact on O

Using FRAM beyond safety: A case study to explore how sociotechnical systems can flourish or stall

FRAM (Functional Resonance Analysis Method) is a relatively new method that has been proposed to explore how functional variability can escalate into unexpected, and often unwanted, events. It has been used for accident analyses and risk assessments in safety. We apply (and slightly modify) FRAM, to analyse how functions are configured to create systems that excel. Our case study focuses on how functions in human factors project work positively resonate to improve the delivery of value. From interviews with 22 practitioners we derived 29 functions and 6 subsystems showing how functions are coupled. Practitioners validated this model through respondent validation. Our case study evaluates the applicability and usability of FRAM. It shows how we adapted the method to make it more usable. It shows that FRAM can be used to examine positive and negative resonance in systems, to investigate how complex sociotechnical systems can flourish or stall.This work was supported by the UK Engineering and Physical Sciences Research Council under Grant [GR/S67494/01], [GR/S67500/01] and [EP/G059063/1

Exploring organisational competences in Human Factors and UX project work: Managing careers, project tactics and organisational strategy

Organisational competence in Human Factors and UX (user experience) has not been looked at before despite its relevance to project success. We define organisational competence as the collective competence of the individuals, bringing together their complementary abilities to deliver an outcome that is typically more than the sum of its parts. Twenty-two UX and Human Factors practitioners were interviewed about their project work in two contrasting domains: web design and safety-critical systems to explore organisational competences. Through doing a FRAM analysis 29 functions and six main areas of competences were identified: the central project process; the process of learning about the problem; maintaining and developing client relations; staff development; evolving practices; and the management of documentation for audit and quality control. These dynamic and situated competences form a web of interactions. Managing competences is essential for project success. Implications for managing careers, project tactics and organisational strategy are discussed