Do I trust my machine teammate? An investigation from perception to decision

© 2019 Copyright held by the owner/author(s). In the human-machine collaboration context, understanding the reason behind each human decision is critical for interpreting the performance of the human-machine team. Via an experimental study of a system with varied levels of accuracy, we describe how human trust interplays with system performance, human perception and decisions. It is revealed that humans are able to perceive the performance of automatic systems and themselves, and adjust their trust levels according to the accuracy of systems. The 70% system accuracy suggests to be a threshold between increasing and decreasing human trust and system usage. We have also shown that trust can be derived from a series of users' decisions rather than from a single one, and relates to the perceptions of users. A general framework depicting how trust and perception affect human decision making is proposed, which can be used as future guidelines for human-machine collaboration design

Calibration analysis within the cognitive and personality domains: individual differences in confidence, accuracy, and bias

[Abstract] Calibration research is concerned with the accuracy of confidence judgments made by individuals when responding to various cognitive tasks. Individuals are scored as accurate or inaccurate based on the objective criterion of whether their responses are correct. Within the personality domain, judging the accuracy of trait self-ratings is more complicated than in the cognitive area as there are no perfect criteria for evaluating the accuracy of these types of judgements (Colvin & Funder, 1991). Because cognitive calibration research findings formed the anchor for the current studies, the decision was made to initially scrutinize the cognitive domain for mis-calibration, and whether individual differences in gender, age, personality, and ability, influenced cognitive confidence and bias scores. In order to achieve the aims, of this dissertation, three studies were conducted with a total of 831 individuals being tested. To determine accuracy within the personality domain, the current studies constructed a situation wherein Big Five personality assessments could be scored as accurate or inaccurate. Results showed that when consistency measures were used, accuracy scores for each Big Five trait were reasonably high across Studies 1, 2, and 3. Prior to the studies conducted in this dissertation, no techniques using calibration procedures had been established to assess Big Five confidence or bias. Within the cognitive domain, calibration research has demonstrated the existence of a trait of self-confidence that appears to be independent of the type of activity being investigated. This result was replicated in Studies 1 and 3. However, the generality of this trait across other domains, such as personality assessments, remains largely unexplored. Two measures were designed to obtain confidence ratings in relation to Big Five personality judgments. Results from three studies showed that the benchmark for peoples‟ confidence in Big Five judgments was around 80%. Data from Studies 1, 2, and 3 also demonstrated a one-factor solution when confidence scores for each Big Five trait were factor analysed. In studies 1 and 3 the factorial structure of cognitive and Big Five confidence scores was examined and both studies produced a two-factor solution. The calibration paradigm also uses bias scores as a measure of how well calibrated individuals are when self-monitoring their performance on various cognitive tasks. Whether people are well-calibrated within the Big Five domain has not been investigated by previous researchers. The current studies examined whether people were mis-calibrated when making Big Five judgments about themselves. The data from two studies indicate that people were well-calibrated for each of the Big Five traits. Factor analyses of Big Five bias scores revealed a one-factor solution. When study 3 examined the factorial structure of cognitive and Big Five bias scores, the analyses showed that bias across these domains were separate but correlated processes. Across all three studies, individual differences in gender and age did not influence Big Five confidence, Big Five accuracy or Big Five bias scores. One of the most significant implications of this dissertation, for calibration researchers, who are striving to understand the mis-calibration phenomenon, was that the structural analyses of cognitive and Big Five bias scores yielded a two-factor solution (i.e., Personality and Cognitive Bias), that was moderately correlated. Also, in the current studies, simple methods were used to obtain Big Five confidence ratings. These procedures could now be used to investigate the factorial structure of confidence in much more detail, and across other domains such as interests, attitudes and values. Practical implications of the current research within the field of clinical psychology were also discussed