Operators׳ adaptation to imperfect automation – Impact of miss-prone alarm systems on attention allocation and performance

Operators in complex environments are often supported by alarm systems that indicate when to shift attention to certain tasks. As alarms are not perfectly reliable, operators have to select appropriate strategies of attention allocation to compensate for unreliability and to maintain overall performance. This study explores how humans adapt to differing alarm reliabilities. Within a multi-task simulation consisting of a monitoring task and two other concurrent tasks, participants were assigned to one of five groups. In the manual control group none of the tasks was supported by an alarm system, whereas the four experimental groups were supported in the monitoring task by a miss-prone alarm system differing in reliability, i.e. 68.75%, 75%, 87.5%, 93.75%. Compared to the manual control group, all experimental groups benefited from the support by alarms, with best performance for the highest reliability condition. However, for the lowest reliability group the benefit was associated with an increased attentional effort, a more demanding attention allocation strategy, and a declined relative performance in a concurrent task. Results are discussed in the context of recent automation research

Response Criterion Placement Modulates the Effects of Graded Alerting Systems on Human Performance and Learning in a Target Detection Task

Human operators can perform better with the use of an automated diagnostic aid than without the use of an aid in a signal detection task. This experiment aimed to determine whether any differences existed among graded aids—automated diagnostic aids that use a scale of confidence levels reflecting a spectrum of probabilistic information or uncertainty when making a judgment—that enabled better human detection performance, and either binary or graded aid produced better learning. Participants performed a visual search framed as a medical decision making task. Stimuli were arrays of random polygons (“cells”) generated by distorting a prototype shape. The target was a shape more strongly distorted than the accompanying distracters. A target was present on half of the trials. Each participant performed the task with the assistance of either a binary aid, one of three graded aids, or no aid. The aids’ sensitivities were the same (d′ = 2); the difference between the aids lay in the placement of their decision criteria, which determines a tradeoff between the aid’s predictive value and the frequency with which it makes a diagnosis. The graded aid with 90% reliability provided a judgment on the greatest number of trials, the graded aid with 94% reliability gave a judgment on fewer trials, and the third graded aid with 96% reliability gave a judgment on the least number of trials. The binary aid with 84% reliability gave a judgment on each trial. All aids improved human detection performance, though the graded aids trended towards improving performance more than the binary aid. The binary and graded aids did not produce significantly better or worse learning than did unaided performance. The binary and graded aids did not significantly help learning, but they certainly did not worsen human detection performance when compared to the unaided condition. These results imply that the decision boundaries of a graded alert might be fixed to encourage appropriate reliance on the aid and improve human detection performance, and indicate employing either a graded or binary automated aid may be beneficial to learning in a detection task

The Effect of Task Load, Automation Reliability, and Environment Complexity on UAV Supervisory Control Performance

Over the last decade, military unmanned aerial vehicles (UAVs) have experienced exponential growth and now comprise over 40% of military aircraft. However, since most military UAVs require multiple operators (usually an air vehicle operator, payload operator, and mission commander), the proliferation of UAVs has created a manpower burden within the U.S. military. Fortunately, simultaneous advances in UAV automation have enabled a switch from direct control to supervisory control; future UAV operators will no longer directly control a single UAV subsystem but, rather, will control multiple advanced, highly autonomous UAVs. However, research is needed to better understand operator performance in a complex UAV supervisory control environment. The Naval Research Lab (NRL) developed SCOUT™ (Supervisory Control Operations User Testbed) to realistically simulate the supervisory control tasks that a future UAV operator will likely perform in a dynamic, uncertain setting under highly variable time constraints. The study reported herein used SCOUT to assess the effects of task load, environment complexity, and automation reliability on UAV operator performance and automation dependence. The effects of automation reliability on participants’ subjective trust ratings and the possible dissociation between task load and subjective workload ratings were also explored. Eighty-one Navy student pilots completed a 34:15 minute pre-scripted SCOUT scenario, during which they managed three helicopter UAVs. To meet mission goals, they decided how to best allocate the UAVs to locate targets while they maintained communications, updated UAV parameters, and monitored their sensor feeds and airspace. After completing training on SCOUT, participants were randomly sorted into low and high automation reliability groups. Within each group, task load (the number of messages and vehicle status updates that had to be made and the number of new targets that appeared) and environment complexity (the complexity of the payload monitoring task) were varied between low and high levels over the course of the scenario. Participants’ throughput, accuracy, and expected value in response to mission events were used to assess their performance. In addition, participants rated their subjective workload and fatigue using the Crew Status Survey. Finally, a four-item survey modeled after Lee and Moray’s validated (1994) scale was used to assess participants’ trust in the payload task automation and their self-confidence that they could have manually performed the payload task. This study contributed to the growing body of knowledge on operator performance within a UAV supervisory control setting. More specifically, it provided experimental evidence of the relationship between operator task load, task complexity, and automation reliability and their effects on operator performance, automation dependence, and operators’ subjective experiences of workload and fatigue. It also explored the relationship between automation reliability and operators’ subjective trust in said automation. The immediate goal of this research effort is to contribute to the development of a suite of domain-specific performance metrics to enable the development and/or testing and evaluation of future UAV ground control stations (GCS), particularly new work support tools and data visualizations. Long-term goals also include the potential augmentation of the current Aviation Selection Test Battery (ASTB) to better select future UAV operators and operational use of the metrics to determine mission-specific manpower requirements. In the far future, UAV-specific performance metrics could also contribute to the development of a dynamic task allocation algorithm for distributing control of UAVs amongst a group of operators

The Effects of Alarm System Errors on Dependence: Moderated Mediation of Trust With and Without Risk

Research on sensor-based signaling systems suggests that false alarms and misses affect operator dependence via two independent psychological processes, hypothesized as two types of trust. These two types of trust manifest in two categorically different behaviors: compliance and reliance. The current study links the theoretical perspective outlined by Lee and See (2004) to the compliance-reliance paradigm, and argues that trust mediates the false alarm-compliance relationship but not the miss-reliance relationship. Specifically, the key conditions to allow the mediation of trust are: The operator is presented with a salient choice to depend on the signaling system and the risk associated with non-dependence is recognized. Eighty-eight participants interacted with a primary flight simulation task and a secondary signaling system task. Participants were asked to evaluate their trust in the signaling system according to the informational bases of trust: Performance, process, and purpose. Half of the participants were in a high risk group and half were in a low risk group. The signaling systems varied by reliability (90%, 60%) within subjects and error bias (false alarm prone, miss prone) between subjects. Generally, analyses supported the hypotheses. Reliability affected compliance, but only in the false alarm prone group. Alternatively, reliability affected reliance, but only in the miss prone group. Higher reliability led to higher subjective trust. Conditional indirect effects indicated that individual factors of trust mediated the relationship between false alarm rate and compliance (i.e., purpose) and reliance (i.e., process), but only in the high risk groups. Serial mediation analyses indicated that the false alarm rate affected compliance and reliance through the sequential ordering of the factors of trust, all stemming from performance. Miss rate did not affect reliance through any of the factors of trust. The theoretical implications of this study suggest the compliance-reliance paradigm is not the reflection of two independent types of trust. The practical applications of this research could be to update training and design recommendations that are based upon the assumption of trust causing operator responses regardless of error bias

THE INFLUENCE OF CULTURAL FACTORS ON TRUST IN AUTOMATION

Human interaction with automation is a complex process that requires both skilled operators and complex system designs to effectively enhance overall performance. Although automation has successfully managed complex systems throughout the world for over half a century, inappropriate reliance on automation can still occur, such as the recent malfunction in Tesla autopilot mechanisms that resulted in a fatality. Research has shown that trust, as an intervening variable, is critical to the development of appropriate reliance on automated systems. Because automation inevitably involves uncertainty, trust in automation is related to a calibration between a user’s expectations and the capabilities of automation. Prior studies suggest that trust is dynamic and influenced by both endogenous (e.g., cultural diversity) and exogenous (e.g., system reliability) variables. To determine how cultural factors affect various aspects of trust in and reliance on automation, the present research has developed a cross-cultural trust questionnaire and an air traffic control simulator that incorporates a variety of scenarios identified from a review of relevant literature. The measures and tasks have been validated by a crowdsourcing system (Amazon Mechanical Turk), as well as through experimental studies conducted in the U.S., Turkey, and Taiwan, with approximately 1000 participants. The results indicate that the developed trust instrument can effectively measure human trust in automation across cultures. The findings reveal substantial cultural differences in human trust in automation, which have a significant impact on the design, implementation, and evaluation of automated systems to make them more trustworthy in determining the appropriate trust calibration for optimized reliance across cultures

Effects of Trust, Self-Confidence, and Feedback on the Use of Decision Automation

Operators often fail to rely sufficiently on alarm systems. This results in a joint human-machine (JHM) sensitivity below the one of the alarm system. The ‘confidence vs. trust hypothesis’ assumes the use of the system depends on the weighting of both values. In case of higher confidence, the task is performed manually, if trust is higher, the user relies on the system. Thus, insufficient reliance may be due to operators’ overconfidence in their own abilities and/or insufficient trust in the decision automation, but could be mitigated by providing feedback. That was investigated within a signal detection task, supported by a system with either higher sensitivity (HSS) or lower sensitivity (LSS) than the human, while being provided with feedback or not. We expected disuse of the LSS and insufficiently reliance on the HSS, in the condition without feedback. The feedback was expected to increase reliance on the HSS through an increase in trust and/or decreases in confidence, and thus, improve performance. Hypotheses were partly supported. Confidence in manual performance was similar to trust in the HSS even though humans’ sensitivity was significantly lower than systems’ sensitivity. While confidence had not effect on reliance or JHM sensitivity, trust was found to be positively related with both. We found disuse of the HSS, that could be improved through feedback, increasing also trust and JHM sensitivity. However, contrary to ‘confidence vs. trust’ expectations, participants were also found to make use of the LSS. This misuse could not be reduced by feedback. Results indicate the use of feedback being beneficial for the overall performance (with HSS only). Findings do not support the idea that misuse or disuse of the system may result from comparison of confidence and trust. We suppose it may rather be the product of users’ wrong strategy of function allocation, based on the underlying idea of team work in combination with missing assignment of responsibility. We discuss this alternative explanation

Effects of Transparency and Haze on Trust and Performance During a Full Motion Video Analysis Task

Automation is pervasive across all task domains, but its adoption poses unique challenges within the intelligence, surveillance, and reconnaissance (ISR) domain. When users are unable to establish optimal levels of trust in the automation, task accuracy, speed, and automation usage suffer (Chung & Wark, 2016). Degraded visual environments (DVEs) are a particular problem in ISR; however, their specific effects on trust and task performance are still open to investigation (Narayanaswami, Gandhe, & Mehra, 2010). Research suggests that transparency of automation is necessary for users to accurately calibrate trust levels (Lyons et al., 2017). Chen et al. (2014) proposed three levels of transparency, with varying amounts of information provided to the user at each level. Transparency may reduce the negative effects of DVEs on trust and performance, but the optimal level of transparency has not been established (Nicolau & McKnight, 2006). The current study investigated the effects of varying levels of transparency and image haze on task performance and user trust in automation. A new model predicting trust from attention was also proposed. A secondary aim was to investigate the usefulness of task shedding and accuracy as measures of trust. A group of 48 undergraduates attempted to identify explosive emplacement activity within a series of full motion video (FMV) clips, aided by an automated analyst. The experimental setup was intended to replicate Level 5 automation (Sheridan & Verplank, 1978). Reliability of the automated analyst was primed to participants as 78% historical accuracy. For each clip, participants could shed their decision to an automated analyst. Higher transparency of automation predicted significantly higher accuracy, whereas hazy visual stimuli predicted significantly lower accuracy and 2.24 times greater likelihood of task shedding. Trust significantly predicted accuracy, but not task shedding. Participants were fastest in the medium transparency condition. The proposed model of attention was not supported; however, participants’ scanning behavior differed significantly between hazy and zero haze conditions. The study was limited by task complexity due to efforts to replicate real-world conditions, leading to confusion on the part of some participants. Results suggested that transparency of automation is critical, and should include purpose, process, performance, reason, algorithm, and environment information. Additional research is needed to explain task shedding behavior and to investigate the relationship between degrade visual environments, transparency of automation, and trust in automation

The Role of Trust as a Mediator Between System Characteristics and Response Behaviors

There have been several theoretical frameworks that acknowledge trust as a prime mediator between system characteristics and automation reliance. Some researchers have operationally defined trust as the behavior exhibited. Other researchers have suggested that although trust may guide operator response behaviors, trust does not completely determine the behavior and advocate the use of subjective measures of trust. Recently, several studies accounting for temporal precedence failed to confirm that trust mediated the relationship between system characteristics and response behavior. The purpose of the current work was to clarify the roles that trust plays in response behavior when interacting with a signaling system. Forty-four participants interacted with a primary flight simulation task and a secondary signaling system task. The signaling system varied in reliability (90% and 60%) within subjects and error bias (false alarm prone and miss prone) between subjects. Analyses indicated that trust partially mediated the relationship between reliability and agreement rate. Trust did not, however, mediate the relationship between reliability and reaction time. Trust also did not mediate the relationships between error bias and reaction time or agreement rate. Analyses of variance generally supported specific behavioral and trust hypotheses, indicating that the paradigm employed produced similar effects on response behaviors and subjective estimates of trust observed in other studies. The results of this study indicate that other mediating variables may offer more predictive power in determining response behaviors. Additionally, strong assumptions of trust acting as the prime mediator and operationally defining trust as a type of behavior should be viewed with caution

The Effect of Culture on Trust in Automation: Reliability and Workload

Trust in automation has become a topic of intensive study over the past two decades. While the earliest trust experiments involved human interventions to correct failures/errors in automated control systems a majority of subsequent studies have investigated information acquisition and analysis decision aiding tasks such as target detection for which automation reliability is more easily manipulated. Despite the high level of international dependence on automation in industry and transport almost all current studies have employed Western samples primarily from the US. The present study addresses these gaps by running a large sample experiment in three (US, Taiwan and Turkey) diverse cultures using a ‘trust sensitive task’ consisting of both automated control and target detection subtasks. This paper presents results for the target detection subtask for which reliability and task load were manipulated. The current experiments allow us to determine whether reported effects are universal or specific to Western culture, vary in baseline or magnitude, or differ across cultures. Results generally confirm consistent effects of manipulations across the three cultures as well as cultural differences in initial trust and variation in effects of manipulations consistent with 10 cultural hypotheses based on Hofstede’s Cultural Dimensions and Leung and Cohen’s theory of Cultural Syndromes. These results provide critical implications and insights for enhancing human trust in intelligent automation systems across cultures. Our paper presents the following contributions: First, to the best of our knowledge, this is the first set of studies that deal with cultural factors across all the cultural syndromes identified in the literature by comparing trust in the Honor, Face, Dignity cultures. Second, this is the first set of studies that uses a validated cross-cultural trust measure for measuring trust in automation. Third, our experiments are the first to study the dynamics of trust across cultures