Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-making

Data associated with the paper with the same titl

Representational Smoothing to Improve Medical Image Decision Making

We demonstrate how medical-image classification decisions can be denoised by aggregating decisions on similar images. In our algorithm, the final decision on a target image is cancerous if a percentage t of the k most similar images are cancerous, else it is not cancerous. Similarity between images is calculated as the distance between representations from an artificial neural network. We vary k and t for novice and expert participants using data from Trueblood et al. (2018) and Trueblood et al. (2021). We show that increasing k improves performance for novices, with their performance approaching that of experts. We also show that the algorithm is biased towards identifying cancerous cells, which is reflected in the representational space. The percentage t allows greater control over sensitivity and specificity and can be used to debias decisions. This algorithm is less effective for experts, partially explained by them giving similar responses on similar images

The Role of Salience in Multialternative Multiattribute Choice

Attention plays a central role in multi-alternative multiat- tribute decision-making but the cognitive mechanisms for it are elusive (Yang & Krajbich, 2023; Molter, Thomas, Huet- tel, Heekeren, & Mohr, 2022; Trueblood, 2022). In this project, we explored the role of bottom-up attention by manipulating the salience of different options in a multi-alternative, multi-attribute choice display. Behaviorally, we observed that salience interacts with choice, where the salient option is selected more often, especially in quick decisions. Using computational modeling, we tested two different hypotheses for how salience impacts decision-making for different individuals. We tested (i) if salience created an initial bias in the decision-making process, and (ii) if salience impacted the comparisons that are made during the decision-making process. We find that there are large individual differences in the mechanism through which salience impacts choice. For many individuals, there was no impact of salience. However, for a sizable minority, salience created an initial boost in selecting the salient option. We do not find strong evidence for the impact of salience in the comparison process. In exploratory analyses, we observe that the impact of salience in decision-making is correlated with thinking styles. Our results indicate that salience-driven attention might impact decision-making in different ways for individuals

A registered report on presentation factors that influence the attraction effect

Context effects occur when the preference between two alternatives is affected by the presence of an extra alternative. These effects are some of the most well studied phenomena in multi-alternative, multi-attribute decision making. Recent research in this area has revealed an intriguing pattern of results. On the one hand, these effects are robust and ubiquitous. That is, they have been demonstrated in many domains and different choice settings. On the other hand, they are fragile and they disappear or even reverse under different conditions. This pattern of results has spurred debate and speculation about the cognitive mechanisms that drive these choices. The attraction effect, where the preference for an option increases in the presence of a dominated decoy, has generated the most controversy. In this registered report, we systematically vary factors that are known to be associated with the attraction effect to build a solid foundation of empirical results to aid future theory development. We find a robust attraction effect across the different conditions. The strength of this effect is modulated by the display order (e.g., decoy top, target middle, competitor bottom) and mode (numeric vs. graphical) but not display layout (by-attribute vs. by-alternative)

Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-making

Improving the accuracy of medical image interpretation is critical to improving the diagnosis of many diseases. Using both novices (undergraduates) and experts (medical professionals), we investigated methods for improving the accuracy of a single decision maker and a group of decision makers by aggregating repeated decisions in different ways. Participants made classification decisions (cancerous versus non-cancerous) and confidence judgments on a series of cell images, viewing and classifying each image twice. We first examined whether it is possible to improve individual-level performance by using the maximum confidence slating algorithm (Koriat, 2012b), which leverages metacognitive ability by using the most confident response for an image as the ‘final response’. We find maximum confidence slating improves individual classification accuracy for both novices and experts. Building on these results, we show that aggregation algorithms based on confidence weighting scale to larger groups of participants, dramatically improving diagnostic accuracy, with the performance of groups of novices reaching that of individual experts. In sum, we find that repeated decision making and confidence weighting can be a valuable way to improve accuracy in medical image decision-making and that these techniques can be used in conjunction with each other

Improving Medical Image Decision Making by Leveraging Metacognitive Processes and Representational Similarity

Improving the accuracy of medical image interpretation is critical to improving the diagnosis of many diseases. Using both novices (undergraduates) and experts (medical professionals), we investigate methods for improving the accuracy of a single decision maker by aggregating repeated decisions from an individual in different ways. Our participants made classification decisions (cancerous versus non-cancerous) and confidence judgments on a series of cell images, viewing and classifying each image twice. We first applied the maximum confidence slating algorithm (Koriat, 2012), which leverages metacognitive ability by using the most confident response for an image as the `final response'. We also examined algorithms that aggregated decisions based on image similarity, leveraging neural network models to determine similarity. We found maximum confidence slating improves classification accuracy for both novices and experts. However, aggregating responses on similar images improves classification accuracy for novices and not experts, suggesting differences in the decision mechanisms of novices and experts