Managing AI Advice in Crowd Decision-Making

In this paper, we examine how advice from an AI algorithm should be provided to decision-makers that work in a crowd setting. With a theoretical model and numerical experiments we show that the harmful effect of incorrect advice relative to the beneficial effect of correct advice increases with increasing crowd size. Thus, for larger crowds, more advice should be withheld so that it does not negatively affect the crowd accuracy. We propose a mechanism for AI advice personalization that takes the crowd size into account. In an experimental study where subjects classified images, we demonstrate that the crowd size-dependent advice personalization reduces the detrimental effects of incorrect advice and leads to an increase in crowd accuracy

Calibrating Users’ Mental Models for Delegation to AI

Artificial intelligence (AI) has the potential to dramatically change the way decisions are made and organizations are managed. As of today, AI is mostly applied as a collaboration partner for humans, amongst others through delegation of tasks. However, it remains to be explored how AI should be optimally designed to enable effective human-AI collaboration through delegation. We analyze influences on human delegation behavior towards AI by studying whether increasing users\u27 knowledge of AI\u27s error boundaries leads to improved delegation behavior and trust in AI. Specifically, we analyze the effect of showing AI\u27s certainty score and outcome feedback alone and in combination using a 2x2 between-subject experiment with 560 subjects. We find that providing both pieces of information can have a positive effect on collaborative performance, delegation behavior, and users\u27 trust in AI. Our findings contribute to the design of AI for collaborative settings and motivate research on factors promoting delegation to AI

Division of Labor between Humans and Algorithms in Healthcare: The Case of Surgery Duration Predictions

For many healthcare applications a collaboration of humans and algorithms has been shown to be superior to pure automation in terms of performance. However, the healthcare sector is characterized by shortages in personnel, which can lead to an excessive workload for the employees and thus makes automation highly beneficial to reduce human workload. In our paper, we consider a combination of different work modes and evaluate whether humans have to be involved in every instance of a task or whether they can be replaced by an AI for some instances. We analyze the potential of segmenting tasks based on who is involved in their completion: Either an AI or a human complete the task individually, or they complete the task together. Considering the case of surgery duration predictions and using a dataset from a university hospital, we observe that human effort could be decreased while maintaining a high prediction performance

New team mates in the warehouse: Human interactions with automated and robotized systems

Despite all the technological progress in the arena of automated and robotized systems, humans will continue to play a significant role in the warehouse of the future, due to their distinctive skills and economic advantages for certain tasks. Although industry and engineering have mainly dealt with the design and functionalities of automated warehouses, the role of human factors and behavior is still underrepresented. However, many novel warehousing systems require human-machine interactions, leading to a growing scientific and managerial necessity to consider human factors and behavior, particularly for operational activities. This is the first study that comprehensively identifies and analyzes relevant behavioral issues of interactions between warehouse operators and machines. To do so, we develop a systematic framework that links human-machine interactions with behavioral issues and implications on system performance across all operational warehouse activities. Insights generated by interviews with warehousing experts are applied to identify the most important issues. We develop a comprehensive research agenda, consisting of a set of potential research questions associated to the identified behavioral issues. The discussion is enriched by providing theoretical and managerial insights from related domains and existing warehousing research. Ultimately, we consolidate our findings by developing overarching theoretical foundations and deriving unifying themes

WILL HUMANS-IN-THE-LOOP BECOME BORGS? MERITS AND PITFALLS OF WORKING WITH AI

We analyze how advice from an AI affects complementarities between humans and AI, in particular what humans know that an AI does not know: unique human knowledge. In a multi-method study consisting of an analytical model, experimental studies, and a simulation study, our main finding is that human choices converge toward similar responses improving individual accuracy. However, as overall individual accu-racy of the group of humans improves, the individual unique human knowledge decreases. Based on this finding, we claim that humans interacting with AI behave like Borgs, that is, cyborg creatures with strong individual performance but no human individuality. We argue that the loss of unique human knowledge may lead to several undesirable outcomes in a host of human-AI decision environments. We demonstrate this harmful impact on the wisdom of crowds. Simulation results based on our experimental data suggest that groups of humans interacting with AI are far less effective as compared to human groups without AI assistance. We suggest mitigation techniques to create environments that can provide the best of both worlds (e.g., by personalizing AI advice). We show that such interventions perform well individually as well as in wisdom of crowds settings

Online rescheduling of physicians in hospitals

Scheduling physicians is a complex task. Legal requirements, different levels of qualification, and preferences for different working hours increase the difficulty of determining a solution that simultaneously fulfills all requirements. Unplanned absences, e.g., due to illness, additionally drive the complexity. In this study, we discuss an approach to deal with the following trade-off. Changes to the existing plan should be kept as small as possible. However, an updated plan should still meet the requirements regarding work regulation, qualifications needed, and physician preferences. We present a mixed-integer linear programming model to create updated duty and workstation rosters simultaneously following absences of scheduled personnel. To enable a comparison with previous sequential approaches, we separate our model into two models for the duty and workstation roster which generate plans sequentially. In a case study, we apply our integrated and sequential models to real-life data from a German university hospital with 133 physicians, 17 duties, and 20 workstations. We consider a planning horizon of 4 weeks and reschedule physicians on each day for three different cost settings for the trade-off between plan quality (in terms of preferences, fairness, coverage and training) and plan stability, resulting in a total of 4201 model runs. We demonstrate that our integrated model can achieve near-optimal results with reasonable computational efforts. In each of these runs our model reschedules physicians within 1-21 s. We run the sequential models on the same data, but for only one cost setting, resulting in 1401 runs. The results indicate that our integrated model manages to respect interdependencies between duty and workstation roster whereas the sequential models will always optimize for the plan which is created first. Overall, results indicate that our integrated model parameters allow managing the trade-off between plan quality goals and plan stability

Will Humans-in-the-Loop Become Borgs? Merits and Pitfalls of Working with AI

We analyze how advice from an AI affects complementarities between humans and AI, in particular what humans know that an AI does not know: “unique human knowledge.” In a multi-method study consisting of an analytical model, experimental studies, and a simulation study, our main finding is that human choices converge toward similar responses improving individual accuracy. However, as overall individual accuracy of the group of humans improves, the individual unique human knowledge decreases. Based on this finding, we claim that humans interacting with AI behave like “Borgs,” that is, cyborg creatures with strong individual performance but no human individuality. We argue that the loss of unique human knowledge may lead to several undesirable outcomes in a host of human–AI decision environments. We demonstrate this harmful impact on the “wisdom of crowds.” Simulation results based on our experimental data suggest that groups of humans interacting with AI are far less effective as compared to human groups without AI assistance. We suggest mitigation techniques to create environments that can provide the best of both worlds (e.g., by personalizing AI advice). We show that such interventions perform well individually as well as in wisdom of crowds settings

A column generation approach for the integrated shift and task scheduling problem of logistics assistants in hospitals

In order to cope with steadily increasing healthcare costs, hospitals introduce a new type of employee taking over logistics tasks from specialized nurses, namely logistics assistants. In the light of the introduction, hospitals are faced with the question of dimensioning their number. We present a mixed-integer program that allows defining the optimal number of logistics assistants, given predefined task requirements. We combine flexible shift scheduling with a task scheduling problem. We incorporate flexibility both in terms of shift scheduling as well as task scheduling in order to define the minimum number of workers. We present a column generation based solution approach that finds optimal solutions, and compare decomposition approaches with one and two subproblems. Neither the general model nor the solution approach are limited to logistics assistants but can also be applied to other problem settings in the healthcare industry and beyond. The approach is tested with 48 problem instances in total and compared to benchmarks. As part of our solution approach, we present a lower bound for staff minimization problems with an unknown number of available workers. We show that flexibility in shift scheduling and task scheduling leads to a decrease of 40-49% of the required workforce, compared to the non-flexible case. (C) 2016 Elsevier B.V. All rights reserved

Exploring User Heterogeneity in Human Delegation Behavior towards AI

As artificial intelligence (AI) can increasingly be used to support decision-making in various areas, enhancing the understanding of human-AI collaboration is more important than ever. We study delegation between humans and AI as one form of collaboration. Specifically, we investigate whether there exist distinct patterns of human delegation behavior towards AI. In a laboratory experiment, subjects performed an image classification task with 100 images to be classified. For the last 50 images, the treatment group had the option to delegate images to an AI. By performing a cluster analysis on this treatment, we find four types of delegation behavior towards AI that differ in their overall performance, delegation rate, and their accuracy of self-assessment. Our results motivate further research on delegation of humans to AI and act as a starting point to research on human collaboration with AI on an individual level

The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals (vol 22, pg 145, 2020)

The original version of this article unfortunately contained errors. The first column of Tables 5 and 6 in the Appendix section should contain the year of publication instead of the reference number in brackets. The reference citations were then placed in the second column