Heikkilä, Eetu

Karvonen, Hannu

Wahlström, Mikael

English

VTT Research System

This document is downloaded from theVTT’s Research Information Portalhttps://cris.vtt.fiVTThttp://www.vtt.fiP.O. box 1000FI-02044 VTTFinlandBy using VTT’s Research Information Portal you are bound by thefollowing Terms & Conditions.I have read and I understand the following statement:This document is protected by copyright and other intellectualproperty rights, and duplication or sale of all or part of any of thisdocument is not permitted, except duplication for research use oreducational purposes in electronic or print form. You must obtainpermission for any other use. Electronic or print copies may not beoffered for sale.VTT Technical Research Centre of FinlandKey Human Factors Concepts of Artificial Intelligence AwarenessKarvonen, Hannu; Heikkilä, Eetu; Wahlström, MikaelPublished: 12/12/2018Document VersionPublisher's final versionLink to publicationPlease cite the original version:Karvonen, H., Heikkilä, E., & Wahlström, M. (2018). Key Human Factors Concepts of Artificial IntelligenceAwareness: Transparency, Communication, and Trust. Poster session presented at AI Day 2018, Espoo,Finland.Download date: 01. May. 2020KEY CONCEPTS AND ISSUES• Definition of AIA: The human’s perception of the current decisionmade by the AI, his/her comprehension of this decision, andhis/her estimate of the decision(s) by AI in the future.Key human factors concepts concerning human-AI interaction:A. Transparency:• How transparent is the functioning of the AI?i. Black box of ML → transparent and clear ML process The probabilities of the predictions in conducted tasks could bepresented to the user to enable the estimation of their reliability.ii. Design implications: Understandable explanations of the ML process Reasons behind certain decisions and results Simplifications and illustrative visualizations of used algorithmsB. Communication:1. The way in which the AI system communicates its functioning,intentions, capabilities, and limitations to the user2. The possibilities of the AI to understand human communication.i. E.g., simplifications and clear visualizations are needed Explicit information visualization techniques, Multimodal output user interfaces Interaction technologies that are natural for humansii. The communication of humans understood by the computer:natural ways of humans to communicate (e.g., voice/gestures)C. Appropriate trust:• Can the user trust the AI, based, on the knowledge about, e.g.,1. The capabilities of the AI-based system,2. The quality and relevancy of the used data, or3. Has the system learned the required skills w/o becoming biased.i. Calibration of trust into an appropriate level (see Fig 1) User has to have a clear idea of the capabilities of the usedalgorithms and what kind of data has been used in the ML Mitigating the problematic effects of overtrust or distrust in AI.DISCUSSION & CONCLUSIONSIncreasing the level of automation and artificial intelligence has effects to human-technology interaction: In addition to sufficient level of SA, human need to achieve also a sufficient level of AA and AIA AIA is a key concept to be considered when studying and designing future socio-technical systems.  How to best support human AIA through design choices?  E.g., system’s user interfaces and the provided training  Key concepts: System transparency, Computer-human communication and Appropriate trust Both theory and practice should be developed to consider AIA more holistically with systems utilizing ML Practical R&D case studies both in naturalistic and experimental settings are needed INTRODUCTION• Increase of AI usage (e.g., machine learning, ML) → More awareness-related interaction challenges for the users of these systems.• Situation awareness (SA), automation awareness (AA), andartificial intelligence awareness (AIA) as important concepts• Focus of this work: user AI awareness regarding ML processes• Related human-AI interaction issues:1. System transparency2. Computer-human communication3. Appropriate trust• Key question: How these phenomena relate to each other?ARTIFICIAL INTELLIGENCE AWARENESS AND AFFECTING MATTERSAIA is based on the ways for the human to gain understandingabout the AI’s functioning, which is affected by e.g.,1. AI system’s user interfaces2. Provided AI-specific training and education3. General knowledge of the principles of computer systems and AI4. Momentary high level of cognitive workload of the human5. Subjectively experienced level of complexity of the AI system.TAXONOMY OF SA, AA, AND AIA• We suggest that the discussed awareness-related phenomenaenclose each other in the following way:• AA encloses AIA while SA encloses these both• To illustrate the taxonomy, we present Fig 2, in which the stagesof awareness are based on the three-level SA model by Endsley.• The circular form of the illustration in Fig 2 refers to the formation ofSA, AA and AIA as a continuous process instead of a linear one.• Neisser’s action-perception cycles in human activity.• In line with the Situated Cognition perspective, emphasizing howcurrent awareness of a situation affects the process of acquiringand interpreting new awareness in an on-going cycle.Figure 1. Appropriate trust in automation. Modified based on Lee & See (2004).ContactHannu Karvonenhannu.karvonen@vtt.fiwww.vttresearch.comKey Human Factors Concepts of Artificial Intelligence Awareness: Transparency, Communication, and TrustHannu Karvonen, Eetu Heikkilä, Mikael WahlströmVTT Technical Research Centre of Finland LtdFigure 2. The Awareness Circles of SA, AA and AIA. The Figure is meant to reflect the relationship between 1. Situation Awareness, 2. Automation Awareness, and 3. AI Awareness. Explanations for the used codes:• 1A: Perception of the current situation• 1B: Comprehension of the current situation• 1C: Estimate of the future situation• 2A: Perception of the current status of the automation• 2B: Comprehension of the current status of the automation• 2C: Estimate of the future status of the automation• 3A: Perception of the current AI-based decision by the system• 3B: Comprehension of the current AI-based decision basis of the system• 3C: Estimate of the system’s AI-based decision(s) in the future and their basis

Key Human Factors Concepts of Artificial Intelligence Awareness:Transparency, Communication, and Trust

This document is downloaded from theVTT’s Research Information Portalhttps://cris.vtt.fiVTThttp://www.vtt.fiP.O. box 1000FI-02044 VTTFinlandBy using VTT’s Research Information Portal you are bound by thefollowing Terms & Conditions.I have read and I understand the following statement:This document is protected by copyright and other intellectualproperty rights, and duplication or sale of all or part of any of thisdocument is not permitted, except duplication for research use oreducational purposes in electronic or print form. You must obtainpermission for any other use. Electronic or print copies may not beoffered for sale.VTT Technical Research Centre of FinlandKey Human Factors Concepts of Artificial Intelligence AwarenessKarvonen, Hannu; Heikkilä, Eetu; Wahlström, MikaelPublished: 12/12/2018Document VersionPublisher's final versionLink to publicationPlease cite the original version:Karvonen, H., Heikkilä, E., & Wahlström, M. (2018). Key Human Factors Concepts of Artificial IntelligenceAwareness: Transparency, Communication, and Trust. Poster session presented at AI Day 2018, Espoo,Finland.Download date: 27. Apr. 2020KEY CONCEPTS AND ISSUES• Definition of AIA: The human’s perception of the current decisionmade by the AI, his/her comprehension of this decision, andhis/her estimate of the decision(s) by AI in the future.Key human factors concepts concerning human-AI interaction:A. Transparency:• How transparent is the functioning of the AI?i. Black box of ML → transparent and clear ML process The probabilities of the predictions in conducted tasks could bepresented to the user to enable the estimation of their reliability.ii. Design implications: Understandable explanations of the ML process Reasons behind certain decisions and results Simplifications and illustrative visualizations of used algorithmsB. Communication:1. The way in which the AI system communicates its functioning,intentions, capabilities, and limitations to the user2. The possibilities of the AI to understand human communication.i. E.g., simplifications and clear visualizations are needed Explicit information visualization techniques, Multimodal output user interfaces Interaction technologies that are natural for humansii. The communication of humans understood by the computer:natural ways of humans to communicate (e.g., voice/gestures)C. Appropriate trust:• Can the user trust the AI, based, on the knowledge about, e.g.,1. The capabilities of the AI-based system,2. The quality and relevancy of the used data, or3. Has the system learned the required skills w/o becoming biased.i. Calibration of trust into an appropriate level (see Fig 1) User has to have a clear idea of the capabilities of the usedalgorithms and what kind of data has been used in the ML Mitigating the problematic effects of overtrust or distrust in AI.DISCUSSION & CONCLUSIONSIncreasing the level of automation and artificial intelligence has effects to human-technology interaction: In addition to sufficient level of SA, human need to achieve also a sufficient level of AA and AIA AIA is a key concept to be considered when studying and designing future socio-technical systems.  How to best support human AIA through design choices?  E.g., system’s user interfaces and the provided training  Key concepts: System transparency, Computer-human communication and Appropriate trust Both theory and practice should be developed to consider AIA more holistically with systems utilizing ML Practical R&D case studies both in naturalistic and experimental settings are needed INTRODUCTION• Increase of AI usage (e.g., machine learning, ML) → More awareness-related interaction challenges for the users of these systems.• Situation awareness (SA), automation awareness (AA), andartificial intelligence awareness (AIA) as important concepts• Focus of this work: user AI awareness regarding ML processes• Related human-AI interaction issues:1. System transparency2. Computer-human communication3. Appropriate trust• Key question: How these phenomena relate to each other?ARTIFICIAL INTELLIGENCE AWARENESS AND AFFECTING MATTERSAIA is based on the ways for the human to gain understandingabout the AI’s functioning, which is affected by e.g.,1. AI system’s user interfaces2. Provided AI-specific training and education3. General knowledge of the principles of computer systems and AI4. Momentary high level of cognitive workload of the human5. Subjectively experienced level of complexity of the AI system.TAXONOMY OF SA, AA, AND AIA• We suggest that the discussed awareness-related phenomenaenclose each other in the following way:• AA encloses AIA while SA encloses these both• To illustrate the taxonomy, we present Fig 2, in which the stagesof awareness are based on the three-level SA model by Endsley.• The circular form of the illustration in Fig 2 refers to the formation ofSA, AA and AIA as a continuous process instead of a linear one.• Neisser’s action-perception cycles in human activity.• In line with the Situated Cognition perspective, emphasizing howcurrent awareness of a situation affects the process of acquiringand interpreting new awareness in an on-going cycle.Figure 1. Appropriate trust in automation. Modified based on Lee & See (2004).ContactHannu Karvonenhannu.karvonen@vtt.fiwww.vttresearch.comKey Human Factors Concepts of Artificial Intelligence Awareness: Transparency, Communication, and TrustHannu Karvonen, Eetu Heikkilä, Mikael WahlströmVTT Technical Research Centre of Finland LtdFigure 2. The Awareness Circles of SA, AA and AIA. The Figure is meant to reflect the relationship between 1. Situation Awareness, 2. Automation Awareness, and 3. AI Awareness. Explanations for the used codes:• 1A: Perception of the current situation• 1B: Comprehension of the current situation• 1C: Estimate of the future situation• 2A: Perception of the current status of the automation• 2B: Comprehension of the current status of the automation• 2C: Estimate of the future status of the automation• 3A: Perception of the current AI-based decision by the system• 3B: Comprehension of the current AI-based decision basis of the system• 3C: Estimate of the system’s AI-based decision(s) in the future and their basis

https://cris.vtt.fi/ws/files/22329248/AI_day_poster.pdf

Key Human Factors Concepts of Artificial Intelligence Awareness:Transparency, Communication, and Trust

Abstract

Similar works

Full text

Available Versions

VTT Research System

VTT Research System