Measuring the meta and cognitive abilities of air defence operators

Introduction: This study aimed to understand more fully some factors that influence decision confidence and accuracy related to air defence. To investigate the metacognitive abilities of air defence operators a Within-Subjects Confidence-Accuracy (W-S C-A) measure was used. Specifically, therefore, this study investigated the impact of Decision Criticality (DC) and Task Stress (TS) on decision making, measures of confidence, accuracy, and the W-S C-A relation. Personality constructs, workload and situation awareness were also included. Method: Participants were allocated to either a high, moderate or low task stress condition. Each participant then took part in a computer generated air defence scenario where they were required to make various decisions and provide a confidence rating for each of those decisions. Confidence, accuracy and W-S C-A were calculated. Results & Discussion: DC impacted both on decision confidence and accuracy, with low DC increasing confidence in decisions and high DC increasing accuracy in decisions

Microgravity induces overconfidence in perceptual decision-making

Does gravity affect decision-making? This question comes into sharp focus as plans for interplanetary human space missions solidify. In the framework of Bayesian brain theories, gravity encapsulates a strong prior, anchoring agents to a reference frame via the vestibular system, informing their decisions and possibly their integration of uncertainty. What happens when such a strong prior is altered? We address this question using a self-motion estimation task in a space analog environment under conditions of altered gravity. Two participants were cast as remote drone operators orbiting Mars in a virtual reality environment on board a parabolic flight, where both hyper- and microgravity conditions were induced. From a first-person perspective, participants viewed a drone exiting a cave and had to first predict a collision and then provide a confidence estimate of their response. We evoked uncertainty in the task by manipulating the motion's trajectory angle. Post-decision subjective confidence reports were negatively predicted by stimulus uncertainty, as expected. Uncertainty alone did not impact overt behavioral responses (performance, choice) differentially across gravity conditions. However microgravity predicted higher subjective confidence, especially in interaction with stimulus uncertainty. These results suggest that variables relating to uncertainty affect decision-making distinctly in microgravity, highlighting the possible need for automatized, compensatory mechanisms when considering human factors in space research.Comment: 12 pages, 10 figure

Code Red: Mobile, a live/synthetic test bed for firefighter training

The State of Victoria, Australia is prone to disastrous bushfires. The Country Fire Authority of Victoria is the principal bushfire response organisation. Local brigades of mainly volunteers learn how to fight fires through classroom learning and field exercises. The CODE RED: MOBILE test bed features Live/Synthetic elements, where ‘Live’ firefighters participate in an exercise where a virtual or ‘synthetic’ bushfire is delivered to them in the field on an iPad3 in the 7scenes game framework (7scenes.com). A model of the Hanging Rock Reserve, with a bushfire undergoing a wind change, was made in the Sandbox2 game editor (crytek.com). This was a model of the real world location of the exercise. Firefighters were divided into two groups: one group received the information about the bushfire’s progress as dynamic-static movies, and the other group saw static screenshots from the same movies. Both were annotated with further information. This media supported with maps and textual information, was delivered at the real world location of three stages of the bushfire undergoing a wind change at the Hanging Rock Reserve. This experiment showed that either form of media was suitable for training firefighters in a mobile learning and decision making exercise using a mobile device. The participants carried GPS and their movements were tracked. Spatio-temporal analysis was used to detect problems with the design of the exercise, and to find participants with aberrant behaviour or difficulties with the exercise. Fractal analysis of the tracks uncovered five Domains of Spatial Scale. Three of the domains at different spatial scales revealed where participants were walking through open areas in relatively straight paths, in another domain they had to find ways through gateways and over bridges, and at another they were meeting the boundaries of the exercise and turning sharply back, or recrossing their tracks. This data is valuable for the analysis and design Live/Synthetic exercises: at various spatial scales particular exercises can be made more difficult or easier to complete depending on the fitness or expertise of the participants. The research proposes that emergency organisations would benefit from Live/Synthetic exercises using mobile devices, for joint or individual training of firefighters and Incident Management Teams

Expert knowledge elicitation in the firefighting domain and the implications for training novices

Background/Purpose: Experienced fireground commanders are often required to make important decisions in time-pressured and dynamic environments that are characterized by a wide range of task constraints. The nature of these environments is such that firefighters are sometimes faced with novel situations that seek to challenge their expertise and therefore necessitate making knowledge-based as opposed to rule-based decisions. The purpose of this study is to elicit the tacitly held knowledge which largely underpinned expert competence when managing non-routine fire incidents. Design/Methodology/Approach: The study utilized a formal knowledge elicitation tool known as the critical decision method (CDM). The CDM method was preferred to other cognitive task analysis (CTA) methods as it is specifically designed to probe the cognitive strategies of domain experts with reference to a single incident that was both challenging and memorable. Thirty experienced firefighters and one staff development officer were interviewed in-depth across different fire stations in the UK and Nigeria (UK=15, Nigeria=16). The interview transcripts were analyzed using the emergent themes analysis (ETA) approach. Findings: Findings from the study revealed 42 salient cues that were sought by experts at each decision point. A critical cue inventory (CCI) was developed and cues were categorized into five distinct types based on the type of information each cue generated to an incident commander. The study also developed a decision making model — information filtering and intuitive decision making model (IFID), which describes how the experienced firefighters were able to make difficult fireground decisions amidst multiple informational sources without having to deliberate on their courses of action. The study also compiled and indexed the elicited tacit knowledge into a competence assessment framework (CAF) with which the competence of future incident commanders could potentially be assessed. Practical Implications: Through the knowledge elicitation process, training needs were identified, and the practical implications for transferring the elicited experts’ knowledge to novice firefighters were also discussed. The four component instructional design model aided the conceptualization of the CDM outputs for training purposes. Originality/Value: Although it is widely believed that experts perform exceptionally well in their domains of practice, the difficulty still lies in finding how best to unmask expert (tacit) knowledge, particularly when it is intended for training purposes. Since tacit knowledge operates in the unconscious realm, articulating and describing it has been shown to be challenging even for experts themselves. This study is therefore timely since its outputs can facilitate the development of training curricula for novices, who then will not have to wait for real fires to occur before learning new skills. This statement holds true particularly in this era where the rate of real fires and therefore the opportunity to gain experience has been on a decline. The current study also presents and discusses insights based on the cultural differences that were observed between the UK and the Nigerian fire service

How does bias / scope influence the operational outcome of pressurised incident command decisions and can it be countered?

Effective fireground decision - making requires good situation awareness (SA) and appropriate selection from the information available to the incident commander. Individuals can display different information bias / scope in their view of the operational incident: either a liberal bias / scope towards accepting information as true with a risk of false alarm errors and / or a conservative bias / scope towards rejecting information with a risk of misses. Such decision - making bias / scope was examined over a series of five separate studies including operational fire fighters and incident commanders. The studies included a breathing apparatus (BA) exercise, two different table top operational incidents (domestic and commercial) and two exercises for flexible duty managers (FDM) in an assessable simulated fireground incident in 2012 and again in 2013. The studies were based on realistic incidents that both fire fighters and FDMs would be expected to respond to, in the final two studies each individual had to take over command and move towards a successful conclusion from an operational, environmental and social perspective. In all the studies, participants were required to answer true or false to a series of probe statements about the incident, which were analysed by a signal detection tool (QASA) to give a measure of actual situational awareness (ASA), perceived situational awareness (PSA) and bias / scope. The first exercise was a BA exercise undertaken to identify if bias was shown by FF’s when undertaking training, the data analysed by the QASA identified that most individuals displayed a high level of ASA about the incident, but also showed either a conservative bias / scope (with miss errors) or a liberal bias /scope (with false alarm errors). The results however also show that two individuals can appear to have similar ASA, but in fact still have very different bias / scope in regard to that knowledge. Once it was established that bias was identified this was developed using table top exercises as it allowed more participants and more control over undertaking the research within normal programmed training periods. The analysis of the two table top exercises showed ASA was high in both, but fire fighters perceived their PSA in a similar way if they had high confidence in one exercise they also had high confidence in the other exercise, or vice versa. However there was no significant correlation between the ASA scores and the PSA scores, with the pattern of bias / scope tendencies being differed across the two studies; with no significant correlation. In reviewing these results the identified difference in undertaking the 2 exercises was that in the second FF’s were familiar with the process and this allow a more relaxed approach, reducing pressure on the individual. While individuals showed bias patterns within the exercises undertaken, more pressurized exercises were identified to see if this bias was consistent for the individual when under pressure. Using the assessable incident commander exercises run by the FRS to test incident commander competence at a FDM level to undertake this. The exercises were used in 2012 and 2013 using the same individuals to compare their results, the outcome of these two simulated assessable fireground incident studies were; • for ASA: there was no significant correlation: r = -.120 and p= .623; • for PSA: there was a significant positive correlation: r =.577 and p = .012; • for bias / scope there was found a strongly positive significant correlation across the scores: r = .592 which is significant at the .008 level. The conclusion of the research is that individuals hold bias / scope tendencies and under pressure these tendencies are shown to be resting and will impact (condition) the individual’s decisions during periods of operational command during stressful conditions. The finding of bias / scope patterns is an important one that may have implications for understanding errors in incident ground decision - making. The finding of resting bias / scope patterns in FDM is an even more important one, which will have implications for understanding errors in incident ground decision - making and how we can help to reduce them. In semi structure interviews with FDMs who had undertaken the assessable exercises, they believed that knowing their bias was a first step to altering it to allow them to improve their decision making at pressurized incidents. Which supported the ultimate goal of the current research to further the understanding of bias / scope tendency, in order to support the training of effective fireground decision - making