Agent Transparency for Intelligent Target Identification in the Maritime Domain, and its impact on Operator Performance, Workload and Trust

This item is only available electronically.Objective: To examine how increasing the transparency of an intelligent maritime target identification system impacts on operator performance, workload and trust in the intelligent agent. Background: Previous research has shown that operator accuracy improves with increased transparency of an intelligent agent’s decisions and recommendations. This can be at the cost of increased workload and response time, although this has not been found by all studies. Prior studies have predominately focussed on route planning and navigation, and it is unclear if the benefits of agent transparency would apply to other tasks such as target identification. Method: Twenty seven participants were required to identify a number of tracks based on a set of identification criteria and the recommendation of an intelligent agent at three transparency levels in a repeated-measures design. The intelligent agent generated an identification recommendation for each track with different levels of transparency information displayed and participants were required to determine the identity of the track. For each transparency level, 70% of the recommendations made by the intelligent agent were correct, with incorrect recommendation due to additional information that the agent was not aware of, such as information from the ship’s radar. Participants’ identification accuracy and identification time were measured, and surveys on operator subjective workload and subjective trust in the intelligent agent were collected for each transparency level. Results: The results indicated that increased transparency information improved the operators’ sensitivity to the accuracy of the agent’s decisions and produced a greater tendency Agent Transparency for Intelligent Target Identification 33 to accept the agent’s decision. Increased agent transparency facilitated human-agent teaming without increasing workload or response time when correctly accepting the intelligent agent’s decision, but increased the response time when rejecting incorrect intelligent agent’s decisions. Participants also reported a higher level of trust when the intelligent agent was more transparent. Conclusion: This study shows the ability of agent transparency to improve performance without increasing workload. Greater agent transparency is also beneficial in building operator trust in the agent. Application: The current study can inform the design and use of uninhabited vehicles and intelligent agents in the maritime context for target identification. It also demonstrates that providing greater transparency of intelligent agents can improve human-agent teaming performance for a previously unstudied task and domain, and hence suggests broader applicability for the design of intelligent agents.Thesis (M.Psych(Organisational & Human Factors)) -- University of Adelaide, School of Psychology, 201

Coping with unreliable automation: content/format/form in the design of human-automation systems

The research presented in this thesis is funded by the European Union and addresses the relationship between people and automated decision support in the context of Traffic Management. Given that automation might not always be 100% reliable, the first research question to be addressed is what effect does automation reliability have on human decision making? User trials contribute to addressing the question of, how can user interfaces be designed to cope with the effects of different levels of automation reliability. The thesis is developed around the concept of Content (the users’ information requirements), Format (the paradigm of interaction and communication protocols) and Form (how information is presented to the users). Results demonstrate that, even in the absence of explicit feedback, users are sensitive to automation reliability and can adapt their information search and decision making strategies accordingly. The user’s decision on whether or not to seek further information cannot be attributed only to information availability or accessing costs, but the visual appearance of the user interface can have a higher influence on user behaviour. These observations and conclusions led to the refinement of the Content/Format/Form concept to a broader sociotechnical design framework