This report documents the program and the outcomes of Dagstuhl Seminar 21381 "Conversational Agent as Trustworthy Autonomous System (Trust-CA)". First, we present the abstracts of the talks delivered by the Seminar’s attendees. Then we report on the origin and process of our six breakout (working) groups. For each group, we describe its contributors, goals and key questions, key insights, and future research. The themes of the groups were derived from a pre-Seminar survey, which also led to a list of suggested readings for the topic of trust in conversational agents. The list is included in this report for references

Følstad, Asbjørn

Guribye, Frode

Kontogiorgos, Dimosthenis

Kraus, Matthias

Laban, Guy

Lee, Minha

Moore, Roger K.

Nesset, Birthe

Paiva, Ana

Pelachaud, Catherine

Porcheron, Martin

Ruane, Elayne

Schuller, Björn

Usbeck, Ricardo

van der Goot, Margot

English

OPUS Augsburg

E e Lai-Chong Law, Asbjørn Følstad, Jonathan Grudin, and Björn Schuller 1014.6 Breakout Group 5: Definition, Conceptualization and Measurementof TrustGroup 5License Creative Commons BY 4.0 International license© Group 5Contributors:Group 5a (in person): Martin Porcheron, Minha Lee, Birthe Nesset, Frode GuribyeGroup 5b (online): Margot van der Goot, Roger K. Moore, Ricardo Usbeck, Ana Paiva,Catherine Pelachaud, Elayne RuaneGroup 5c (in person): Björn Schuller, Guy Laban, Dimosthenis Kontogiorgos, MatthiasKraus, Asbjørn Følstad4.6.1 Goal and key questionsGoal: Enable assessment and measurement of trust in CAsKey question: How to define, conceptualize and measure trust in CA?Relevant aspects: Modelling frameworks – antecedents / consequents; basis in knowledge onhuman-human communication; psychology of trust – over-trust / intuition4.6.2 Key insightsDefining trust. Trust is addressed in di erent disciplines, both as a general concept withinpsychology, sociology, and management research (e.g., Rousseau, 1998; Mayer et al., 1995)and – more recently – as a term of relevance for users’ perceptions of technology (e.g.,Corritore et al., 2003; McKnight et al., 2011). A range of definitions exists for trust. There isvariation in definitions concerning whether trust should be construed as a belief or attitude(Lewis et al., 2018; Lee & See 2004), and the degree to which there is a behavioural elementin trust (Söllner et al., 2016; Malle & Ullman, 2021).For conceptual clarity, it may be useful to consider trust an attitude which may be foundedby trusting beliefs, and which may lead to trusting behaviour.Trusting behaviour is determined by trust and may as such be an indicator of trust – providedusers have a choice. Trusting behaviour is also moderated by environment, user group, and usecase. An example of trusting behaviour is self-disclosure. Trust may also impact engagementlevel in behaviour and tendency to repeated use.Developing trust through conversational interactions. The notion of trust in technologyarguably is of particular relevance to CAs, due to their conversational interaction withusers. Conversations are humanlike which has implications for users trusting beliefs andbehaviours. Furthermore, conversations may be relational, leading to expectations of evolvingcapabilities in agent. Conversations may also be cooperative, leading to expectations ofmutual adaptations in the user and conversational agent to achieve a common goal.On this background, trust in CA may be considered as gradually built through conversations.In consequence, four trust concepts may be of particular relevance for CAs:Initial trust: trust required for users to initiate interaction. Initial trust corresponds tothe notion of calculating trust in Rousseau et al. (1998)21381102 21381 – Conversational Agent as Trustworthy Autonomous System (Trust-CA)Sharing trust: trust required for sharing information with chatbot. The relevance ofsharing trust may depend on varying levels of perceived sensitivity in the domain or topicof CA interaction.Reliance trust: trust required for relying on chatbot recommendations or decision support,that is, trust impacting user beliefs or behaviour beyond the context of the CA interaction.Long-term trust: trust required for repeated / routine use. Long term trust correspondsto the notion of relational trust in Rousseau et al. (1998)Extending the trust model of Rousseau et al. (1998), the four trust concepts for CAs may bemapped out on a timeline of the evolving relation between user and CA as follows:Figure 4 Extended trust model.Balancing trust and trustworthiness. When considering trust, it is critical to distinguishbetween perceived trust and trustworthiness.Perceived trust is held by the trustor, typically the user. Perceived trust and related trustbeliefs may be measured through a range of self-report measurements, for example frominformation systems research (e.g. Lankton et al., 2015), social robotics (c.f. review inHancock et al., 2020). Perceived trust may be impacted by the trustworthiness of the trustor.However, as information on this may not be available, other characteristics may impact trust.For CAs, anthropomorphism may be such a characteristic, as it may impact trust thoughnot be correlated with trustworthiness.Trustworthiness is a characteristic of the trustee, typically the service provider. Trustworthi-ness may depend on factors such as transparency, reliability, consistency, sincerity, honesty,integrity, benevolence, competence, and cooperation. These factors, though not necessarilystatic, may be considered observable characteristics in a trustee.There is a need to study trustworthiness and perceived trust in parallel – to address potentialovertrust (low trustworthiness and high perceived trust) and undertrust (high trustworthinessand low perceived trust). There is a lack of approaches or measurements for the integratedstudy of trustworthiness and trust.Measuring trust by integrating self report measures and behavioural measures. Inexisting scales and measurements, trust is typically construed as personal, mainly availableto researchers through self report measurements. Nevertheless, trust can be interpreted asreflected in and through people’s behaviour, rather than merely a stance prior to the useof some device or system. Trust as reflected in trusting behaviour may enables trust to bemeasured also on the basis of user behaviour. There seem however to be a lack of distinctbehaviour scales for trust assessment.Possibly, trust may be measured by having a CA asking about sensitive information andmonitor users’ disclosing behaviour. Specifically, a tiered approach may be useful, based onasking questions of personal information of increasing level of sensitivity to infer a person’sE e Lai-Chong Law, Asbjørn Følstad, Jonathan Grudin, and Björn Schuller 103level or trust. However, the choice of behavioural measures of trust may depend on thecontext of the CA.An integrated approach, combining self-report measures and tiered behavioural measuresseems a promising approach for future research.A proposed integrating framework for measuring trust and trusting behaviour. Followingfrom the above, instruments and data sources for measuring trust may be divided into twobroad groups: Subjective and objective measures:Subjective measures concern the measurement of trust determinants / trusting beliefs orbehavioural intent (e.g., Lankton et al., 2015; Yagoda & Gillan, 2012). As a subjectivemeasurement, perceptions of trust are expected to be explicit from the subject’s report,corresponding to the subject’s trust beliefs. Nevertheless, these might not be consistentwith the subject’s trusting behaviour due to personal perceptions and attitudes of thesubject regarding the conversational AI system – e.g., due to scepticism of AI (Araujo etal., 2020).Objective measures include measures of physiological states, speech / voice, interactionwith agent (e.g., sharing behaviour), changes in beliefs due to agent, behaviour in theworld due to agent. Accordingly, the subject’s behaviour would implicitly indicate higheror lower levels of trust. THe association between trusting behaviour and trust shouldbe studied individually, depending on context, settings and task. Within the scope ofconversational AI, behaviour such as self-disclosure (e.g., Laban et al., 2021a), reciprocity(e.g., Zonca et al., 2021), and changes in disclosure and expression over time (e.g., Labanet al., 2021b) could implicitly indicate changes in trust. These behaviours, however, mightnot be consistent with one’s trust beliefs due to, for example, habits and needs (e.g.,having the need to share, or being an impulsive individual) or a ect-based factors of trustlike the system’s heuristics and demonstrated social cues (e.g., one might be more likelyto share information with a more persuasive system despite not trusting it; e.g., Ghazaliet al., 2019).Subjective and objective measures may be included in a framework of trusting beliefs andtrusting behaviour as follows:4.6.3 Future ResearchThe following questions require further research e ort to address:Developing a comprehensive framework to capture how trust evolves across long-termuse.Refining the framework for trusting beliefs and trusting behaviour.Developing integrated approaches and measures for studying users perceived trust andthe trustworthiness of service providers, to mitigate overtrust and undertrust.Developing integrated measures of trust and trusting behaviour, combining self reportmeasures and tiered behavioural measures to support standardised measure for trust inconversational agents, and incorporating this in conversational systems.21381104 21381 – Conversational Agent as Trustworthy Autonomous System (Trust-CA)Figure 5 Framework of trusting beliefs and trusting behaviour.References1 Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020). In AI we trust?Perceptions about automated decision-making by artificial intelligence. AI and Society,35(3), 611 – 623.2 Corritore, C. L., Kracher, B., & Wiedenbeck, S. (2003). On-line trust: concepts, evolvingthemes, a model. International Journal of Human-Computer Studies, 58(6), 737-758.3 Ghazali, A. S., Ham, J., Barakova, E., & Markopoulos, P. (2019). Assessing the e ect ofpersuasive robots interactive social cues on users’ psychological reactance, liking, trustingbeliefs and compliance. Advanced Robotics, 33(7 – 8), 325 – 337.4 Hancock, P. A., Kessler, T. T., Kaplan, A. D., Brill, J. C., & Szalma, J. L. (2020). Evolvingtrust in robots: specification through sequential and comparative meta-analyses. Humanfactors, 0018720820922080.5 Laban, G., George, J.-N., Morrison, V., & Cross, E. S. (2021a). Tell me more! Assessinginteractions with social robots from speech. Paladyn, Journal of Behavioral Robotics, 12(1),136 – 159.6 Laban, G., Kappas, A., Morrison, V., & Cross, E. S. (2021b). Protocol for a MediatedLong-Term Experiment with a Social Robot. PsyArXiv.7 Lankton, N. K., McKnight, D. H., & Tripp, J. (2015). Technology, humanness, and trust:Rethinking trust in technology. Journal of the Association for Information Systems, 16(10).8 Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance.Human factors, 46(1), 50-80.9 Lewis, M., Sycara, K., & Walker, P. (2018). The role of trust in human-robot interaction.In Foundations of trusted autonomy (pp. 135-159). Springer, Cham.10 Malle, B. F., & Ullman, D. (2021). A multidimensional conception and measure of human-robot trust. In Trust in Human-Robot Interaction (pp. 3-25). Academic Press.11 Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizationaltrust. Academy of management review, 20(3), 709-734.E e Lai-Chong Law, Asbjørn Følstad, Jonathan Grudin, and Björn Schuller 10512 McKnight, D. H., Carter, M., Thatcher, J. B., & Clay, P. F. (2011). Trust in a specifictechnology: An investigation of its components and measures. ACM Transactions onmanagement information systems (TMIS), 2(2), 1-25.13 Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. (1998). Not so di erent after all:A cross-discipline view of trust. Academy of Management Review, 23(3), 393-404.14 Söllner, M., Ho mann, A., & Leimeister, J. M. (2016). Why di erent trust relationshipsmatter for information systems users. European Journal of Information Systems, 25(3),274-287.15 Yagoda, R. E., & Gillan, D. J. (2012). You Want Me to Trust a ROBOT? The Developmentof a Human – Robot Interaction Trust Scale. International Journal of Social Robotics, 4(3),235 – 248.16 Zonca, J., Folsø, A., Sciutti, A. (2021). The role of reciprocity in human-robot socialinfluence. iScience, 24(12), 103424.4.7 Breakout Group 6: Interaction DesignGroup 6License Creative Commons BY 4.0 International license© Group 6Contributors:Group 6a (in person): Oliver Bendel, Sebastian Hobert, Ryan Schuetzler, Elisabeth André,Leigh Clark, Clayton Lewis, Stefan Scha er, Eren Yildiz, E e LawGroup 6b (online): Stefan Morana, Heloisa Candello, Christine Liebrecht, Zhou Yu, DakuoWang, Michelle Zhou, Ana Paula Chaves, Cosmin Munteanu, Soomin Kim4.7.1 Goal and key questionsGoal: Identify interaction designs to strengthen trust in CA.Key questions: How to design trusted conversational user interfaces?Relevant aspects: UX – human-AI sociability; Human-in-the-loop; Evaluation – reliability/ac-ceptance; Group interaction.4.7.2 Key insights4.7.2.1 Group 6aThe group started with reflecting on the following aspects of trust:Brand and UX: The producer of a chatbot a ects our perception of trust; we trust certainproducts because we trust certain brands; the implication of UX-trust relationGroup e ect: If we trust people, and those people trust a chatbot, then we are morelikely to trust it as well.Domain-dependency: Certain domains are more sensitive to trust fluctuationModality: Intricate relations between modality, risk and trustNext, the group focused on some specific aspects of conversational interactions. The mul-tifaceted nature of trust and the numerous factors of people’s interactions with CAs thatcan impact on perceptions and behaviours, complicating our understanding of how, whyand when to design for trust and subsequently evaluate it. We present a discussion of somecritical aspects of CA interactions and highlight the need for a holistic approach to creatingtrustworthy CAs.21381View publication stats

Definition, conceptualisation and measurement of trust

https://opus.bibliothek.uni-augsburg.de/opus4/files/101368/101368.pdf

Definition, conceptualisation and measurement of trust

Abstract

Similar works

Full text

Available Versions

OPUS Augsburg