Towards Effective Human-AI Decision-Making: The Role of Human Learning in Appropriate Reliance on AI Advice

The true potential of human-AI collaboration lies in exploiting the complementary capabilities of humans and AI to achieve a joint performance superior to that of the individual AI or human, i.e., to achieve complementary team performance (CTP). To realize this complementarity potential, humans need to exercise discretion in following AI’s advice, i.e., appropriately relying on the AI’s advice. While previous work has focused on building a mental model of the AI to assess AI recommendations, recent research has shown that the mental model alone cannot explain appropriate reliance. We hypothesize that, in addition to the mental model, human learning is a key mediator of appropriate reliance and, thus, CTP. In this study, we demonstrate the relationship between learning and appropriate reliance in an experiment with 100 participants. This work provides fundamental concepts for analyzing reliance and derives implications for the effective design of human-AI decision-making

ML-Based Teaching Systems: A Conceptual Framework

As the shortage of skilled workers continues to be a pressing issue, exacerbated by demographic change, it is becoming a critical challenge for organizations to preserve the knowledge of retiring experts and to pass it on to novices. While this knowledge transfer has traditionally taken place through personal interaction, it lacks scalability and requires significant resources and time. IT-based teaching systems have addressed this scalability issue, but their development is still tedious and time-consuming. In this work, we investigate the potential of machine learning (ML) models to facilitate knowledge transfer in an organizational context, leading to more cost-effective IT-based teaching systems. Through a systematic literature review, we examine key concepts, themes, and dimensions to better understand and design ML-based teaching systems. To do so, we capture and consolidate the capabilities of ML models in IT-based teaching systems, inductively analyze relevant concepts in this context, and determine their interrelationships. We present our findings in the form of a review of the key concepts, themes, and dimensions to understand and inform on ML-based teaching systems. Building on these results, our work contributes to research on computer-supported cooperative work by conceptualizing how ML-based teaching systems can preserve expert knowledge and facilitate its transfer from SMEs to human novices. In this way, we shed light on this emerging subfield of human-computer interaction and serve to build an interdisciplinary research agenda

Machines that Learn and Teach Seamlessly

This paper describes an investigation into creating agents that can learn how to perform a task by observing an expert, then seamlessly turn around and teach the same task to a less proficient person. These agents are taught through observation of expert performance and thereafter refined through unsupervised practice of the task, all on a simulated environment. A less proficient human is subsequently taught by the now-trained agent through a third approach-coaching, executed through a haptic device. This approach addresses tasks that involve complex psychomotor skills. A machine-learning algorithm called PIGEON is used to teach the agents. A prototype is built and then tested on a task involving the manipulation of a crane to move large container boxes in a simulated shipyard. Two evaluations were performed-a proficiency test and a learning rate test. These tests were designed to determine whether this approach improves the human learning more than self-experimentation by the human. While the test results do not conclusively show that our approach provides improvement over self-learning, some positive aspects of the results suggest great potential for this approach

Machines That Learn And Teach Seamlessly

This paper describes an investigation into creating agents that can learn how to perform a task by observing an expert, then seamlessly turn around and teach the same task to a less proficient person. These agents are taught through observation of expert performance and thereafter refined through unsupervised practice of the task, all on a simulated environment. A less proficient human is subsequently taught by the now-trained agent through a third approach-coaching, executed through a haptic device. This approach addresses tasks that involve complex psychomotor skills. A machine-learning algorithm called PIGEON is used to teach the agents. A prototype is built and then tested on a task involving the manipulation of a crane to move large container boxes in a simulated shipyard. Two evaluations were performed-a proficiency test and a learning rate test. These tests were designed to determine whether this approach improves the human learning more than self-experimentation by the human. While the test results do not conclusively show that our approach provides improvement over self-learning, some positive aspects of the results suggest great potential for this approach. © 2008-2011 IEEE

Machines that learn and teach seamlessly

This paper describes an investigation into creating agents that can learn how to perform a task by observing an expert, then seamlessly turn around and teach the same task to a less proficient person. These agents are taught through observation of expert performance and thereafter refined through unsupervised practice of the task, all on a simulated environment. A less proficient human is subsequently taught by the now-trained agent through a third approach-coaching, executed through a haptic device. This approach addresses tasks that involve complex psychomotor skills. A machine-learning algorithm called PIGEON is used to teach the agents. A prototype is built and then tested on a task involving the manipulation of a crane to move large container boxes in a simulated shipyard. Two evaluations were performed-a proficiency test and a learning rate test. These tests were designed to determine whether this approach improves the human learning more than self-experimentation by the human. While the test results do not conclusively show that our approach provides improvement over self-learning, some positive aspects of the results suggest great potential for this approach. © 2008-2011 IEEE