2 research outputs found
The benefits and costs of explainable artificial intelligence in visual quality control: Evidence from fault detection performance and eye movements
Visual inspection tasks often require humans to cooperate with AI-based image
classifiers. To enhance this cooperation, explainable artificial intelligence
(XAI) can highlight those image areas that have contributed to an AI decision.
However, the literature on visual cueing suggests that such XAI support might
come with costs of its own. To better understand how the benefits and cost of
XAI depend on the accuracy of AI classifications and XAI highlights, we
conducted two experiments that simulated visual quality control in a chocolate
factory. Participants had to decide whether chocolate moulds contained faulty
bars or not, and were always informed whether the AI had classified the mould
as faulty or not. In half of the experiment, they saw additional XAI highlights
that justified this classification. While XAI speeded up performance, its
effects on error rates were highly dependent on (X)AI accuracy. XAI benefits
were observed when the system correctly detected and highlighted the fault, but
XAI costs were evident for misplaced highlights that marked an intact area
while the actual fault was located elsewhere. Eye movement analyses indicated
that participants spent less time searching the rest of the mould and thus
looked at the fault less often. However, we also observed large interindividual
differences. Taken together, the results suggest that despite its potentials,
XAI can discourage people from investing effort into their own information
analysis