To whom to explain and what? : Systematic literature review on empirical studies on Explainable Artificial Intelligence (XAI)

Expectations towards artificial intelligence (AI) have risen continuously because of machine learning models’ evolution. However, the models’ decisions are often not intuitively understandable. For this reason, the field of Explainable AI (XAI) has emerged, which tries to create different techniques to help users understand AI better. As AI’s use spreads more broadly in society, it becomes like a co-worker that people need to understand. For this reason, AI-human interaction in research is of broad and current interest. This thesis outlines the current empirical XAI research literature themes from the human-computer interaction (HCI) perspective. This study's method is an explorative, systematic literature review carried out following the PRISMA (Preferred Research Items for Systematic Reviews) method. In total, 29 articles that concluded an empirical study into XAI from the HCI perspective were included in the review. The material was collected based on database searches and snowball sampling. The articles were analyzed based on their descriptive statistics, stakeholder groups, research questions, and theoretical approaches. This study aims to determine what factors made users consider XAI transparent, explainable, or trustworthy and to whom the XAI research was intended. Based on the analysis, three stakeholder groups to whom the current XAI literature was aimed for emerged: end-users, domain experts, and developers. This study’s findings show that domain experts’ needs towards XAI vary greatly between domains, whereas developers need better tools to create XAI systems. The end-users, on their part, considered case-based explanations unfair and wanted to have explanations that “speak their language”. Also, the results indicate that the effect of current XAI solutions on users’ trust towards AI systems is relatively small or even non-existing. The studies’ direct theoretical contributions and the number of theoretical lenses used were both found out to be relatively low. This thesis’s most immense contribution is to provide a synthesis of the extant empirical XAI literature from the HCI perspective, which previous studies have rarely brought together. Continuing this thesis, researchers can further investigate research avenues such as explanation quality methodologies, algorithm auditing methods, users’ mental models, and prior conceptions about AI.Odotukset tekoälyä kohtaan ovat kohonneet jatkuvasti koneoppimismallien kehittymisen vuoksi. Mallien tekemät päätökset eivät usein ole ihmiskäyttäjälle vaistonvaraisesti ymmärrettävissä. Tätä ongelmaa ratkomaan on syntynyt selittävän tekoälyn tutkimuskenttä, joka luo erilaisia tekniikoita käyttäjien ymmärryksen tueksi. Kun tekoälyn käyttö yhteiskunnassa yleistyy laajemmin, tulee siitä ikään kuin työkaveri, jota ihmisten tulee ymmärtää. Tästä syystä tekoälyn ja ihmisen välisen vuorovaikutuksen tutkiminen on nyt laajan mielenkiinnon kohteena. Tässä pro gradu -tutkielmassa hahmotellaan selittävän tekoälyn tutkimuskentän ajankohtaisia teemoja, ihmisen ja tietokoneen välisen vuorovaikutuksen näkökulmasta. Tutkielman metodi on tutkiva, systemaattinen kirjallisuuskatsaus, ja se suoritettiin seuraten PRISMA-ohjeistusta. Katsaukseen valikoitui yhteensä 29 ihmisen ja tietokoneen vuorovaikutuksen näkökulmasta selittävää tekoälyä empiirisesti tutkinutta artikkelia. Aineisto kerättiin tietokantahakujen ja lumipallo-otannan avulla. Tutkimuksia eriteltiin artikkeleja kuvailevien tietojen, niiden kohdeyleisön, tutkimuskysymysten sekä teoreettisten lähestymistapojen kautta. Tutkielman tarkoituksena on selvittää, millaiset tekijät saivat käyttäjät pitämään tekoälyä läpinäkyvänä, selitettävissä olevana tai luotettavana, sekä kenelle aihepiirin tutkimus oli suunnattu. Analyysin perusteella löytyi kolme ryhmää, joille nykyistä kirjallisuutta on suunnattu: loppukäyttäjät, toimialojen asiantuntijat sekä tekoälyn kehittäjät. Tutkielman tulokset osoittavat, että asiantuntijoiden tarpeet selittävää tekoälyä kohtaan vaihtelevat laajasti toimialojen välillä, kun taas sen kehittäjät kaipaisivat parempia työkaluja tuekseen. Loppukäyttäjien havaittiin pitävän tekoälyn antamia tapauskohtaisia esimerkkejä epäreiluina, ja haluavan juuri heitä puhuttelevia selityksiä. Tulokset ilmaisevat, että nykyisten selittävien tekoälytekniikoiden vaikutukset käyttäjien luottamukseen tekoälyä kohtaan ovat vähäisiä. Tutkimusten tieteellisen panosten ja niiden käyttämien teoreettisten näkökulmien määrän havaittiin olevan suhteellisen pieniä. Tämän tutkielman suurin tieteellinen panos on luoda yhteenveto empiiriseen, selittävän tekoälyn tutkimuskirjallisuuteen, ihmisen ja tietokoneen välisen vuorovaikutuksen näkökulmasta. Tätä näkökulmaa aiempi kirjallisuus on vain harvoin saattanut kokoon. Tutkielma avaa useita näkymiä jatkotutkimukselle, esimerkiksi selitysten laatumetodien, algoritmien auditointimenetelmien, käyttäjien ajatusmallien sekä aiempien käsitysten vaikutusten näkökulmista

Understanding the Role of Interactivity and Explanation in Adaptive Experiences

Adaptive experiences have been an active area of research in the past few decades, accompanied by advances in technology such as machine learning and artificial intelligence. Whether the currently ongoing research on adaptive experiences has focused on personalization algorithms, explainability, user engagement, or privacy and security, there is growing interest and resources in developing and improving these research focuses. Even though the research on adaptive experiences has been dynamic and rapidly evolving, achieving a high level of user engagement in adaptive experiences remains a challenge. %????? This dissertation aims to uncover ways to engage users in adaptive experiences by incorporating interactivity and explanation through four studies. Study I takes the first step to link the explanation and interactivity in machine learning systems to facilitate users\u27 engagement with the underlying machine learning model with the Tic-Tac-Toe game as a use case. The results show that explainable machine learning (XML) systems (and arguably XAI systems in general) indeed benefit from mechanisms that allow users to interact with the system\u27s internal decision rules. Study II, III, and IV further focus on adaptive experiences in recommender systems in specific, exploring the role of interactivity and explanation to keep the user “in-the-loop” in recommender systems, trying to mitigate the ``filter bubble\u27\u27 problem and help users in self-actualizing by supporting them in exploring and understanding their unique tastes. Study II investigates the effect of recommendation source (a human expert vs. an AI algorithm) and justification method (needs-based vs. interest-based justification) on professional development recommendations in a scenario-based study setting. The results show an interaction effect between these two system aspects: users who are told that the recommendations are based on their interests have a better experience when the recommendations are presented as originating from an AI algorithm, while users who are told that the recommendations are based on their needs have a better experience when the recommendations are presented as originating from a human expert. This work implies that while building the proposed novel movie recommender system covered in study IV, it would provide a better user experience if the movie recommendations are presented as originating from algorithms rather than from a human expert considering that movie preferences (which will be visualized by the movies\u27 emotion feature) are usually based on users\u27 interest. Study III explores the effects of four novel alternative recommendation lists on participants’ perceptions of recommendations and their satisfaction with the system. The four novel alternative recommendation lists (RSSA features) which have the potential to go beyond the traditional top N recommendations provide transparency from a different level --- how much else does the system learn about users beyond the traditional top N recommendations, which in turn enable users to interact with these alternative lists by rating the initial recommendations so as to correct or confirm the system\u27s estimates of the alternative recommendations. The subjective evaluation and behavioral analysis demonstrate that the proposed RSSA features had a significant effect on the user experience, surprisingly, two of the four RSSA features (the controversial and hate features) perform worse than the traditional top-N recommendations on the measured subjective dependent variables while the other two RSSA features (the hipster and no clue items) perform equally well and even slightly better than the traditional top-N (but this effect is not statistically significant). Moreover, the results indicate that individual differences, such as the need for novelty and domain knowledge, play a significant role in users’ perception of and interaction with the system. Study IV further combines diversification, visualization, and interactivity, aiming to encourage users to be more engaged with the system. The results show that introducing emotion as an item feature into recommender systems does help in personalization and individual taste exploration; these benefits are greatly optimized through the mechanisms that diversify recommendations by emotional signature, visualize recommendations on the emotional signature, and allow users to directly interact with the system by tweaking their tastes, which further contributes to both user experience and self-actualization. This work has practical implications for designing adaptive experiences. Explanation solutions in adaptive experiences might not always lead to a positive user experience, it highly depends on the application domain and the context (as studied in all four studies); it is essential to carefully investigate a specific explanation solution in combination with other design elements in different fields. Introducing control by allowing for direct interactivity (vs. indirect interactivity) in adaptive systems and providing feedback to users\u27 input by integrating their input into the algorithms would create a more engaging and interactive user experience (as studied in Study I and IV). And cumulatively, appropriate direct interaction with the system along with deliberate and thoughtful designs of explanation (including visualization design with the application environment fully considered), which are able to arouse user reflection or resonance, would potentially promote both user experience and user self-actualization

Customers’ loyalty model in the design of e-commerce recommender systems

Recommender systems have been adopted in most modern online platforms to guide users in finding more suitable items that match their interests. Previous studies showed that recommender systems impact the buying behavior of e-commerce customers. However, service providers are more concerned about the continuing behavior of their customers, specifically customers’ loyalty, which is an important factor to increase service providers’ share of wallet. Therefore, this study aimed to investigate the customers’ loyalty factors in online shopping towards e-commerce recommender systems. To address the research objectives, a new research model was proposed based on the Cognition-Affect-Behavior model. To validate the research model, a quantitative methodology was utilized to gather the relevant data. Using a survey method, a total of 310 responses were gathered to examine the impacts of the identified factors on customers’ loyalty towards Amazon’s recommender system. Data was analysed using Partial Least Square Structural Equation Modelling. The results of the analysis indicated that Usability (P=0.467, t=5.139, p<0.001), Service Interaction (P=0.304, t=4.42, p<0.001), Website Quality (P=0.625, t=15.304, p<0.001), Accuracy (P=0.397, t=6.144, p<0.001), Novelty (P=0.289, t=4.406, p<0.001), Diversity (P=0.142, t=2.503, p<0.001), Recommendation Quality (P=0.423, t=7.719, p<0.001), Explanation (P=0.629, t=15.408, p<0.001), Transparency (P=0.279, t=5.859, p<0.001), Satisfaction (P=0.152, t=3.045, p<0.001) and Trust (P=0.706, t=14.14, p<0.001) have significant impacts on customers’ loyalty towards the recommender systems in online shopping. Information quality, however, did not affect the quality of the website that hosted the recommender system. The findings demonstrated that accuracy-oriented measures were insufficient in understanding customer behavior, and other quality factors, such as diversity, novelty, and transparency could improve customers’ loyalty towards recommender systems. The outcomes of the study indicated the significant impact of the website quality on customers’ loyalty. The developed model would be practical in helping the service providers in understanding the impacts of the identified factors in the proposed customers’ loyalty model. The outcomes of the study could also be used in the design of recommender systems and the deployed algorithm

Explainable Recommendations in Intelligent Systems: Delivery Methods, Modalities and Risks

With the increase in data volume, velocity and types, intelligent human-agent systems have become popular and adopted in different application domains, including critical and sensitive areas such as health and security. Humans’ trust, their consent and receptiveness to recommendations are the main requirement for the success of such services. Recently, the demand on explaining the recommendations to humans has increased both from humans interacting with these systems so that they make an informed decision and, also, owners and systems managers to increase transparency and consequently trust and users’ retention. Existing systematic reviews in the area of explainable recommendations focused on the goal of providing explanations, their presentation and informational content. In this paper, we review the literature with a focus on two user experience facets of explanations; delivery methods and modalities. We then focus on the risks of explanation both on user experience and their decision making. Our review revealed that explanations delivery to end-users is mostly designed to be along with the recommendation in a push and pull styles while archiving explanations for later accountability and traceability is still limited. We also found that the emphasis was mainly on the benefits of recommendations while risks and potential concerns, such as over-reliance on machines, is still a new area to explore

DESIGN AND EXPLORATION OF NEW MODELS FOR SECURITY AND PRIVACY-SENSITIVE COLLABORATION SYSTEMS

Collaboration has been an area of interest in many domains including education, research, healthcare supply chain, Internet of things, and music etc. It enhances problem solving through expertise sharing, ideas sharing, learning and resource sharing, and improved decision making. To address the limitations in the existing literature, this dissertation presents a design science artifact and a conceptual model for collaborative environment. The first artifact is a blockchain based collaborative information exchange system that utilizes blockchain technology and semi-automated ontology mappings to enable secure and interoperable health information exchange among different health care institutions. The conceptual model proposed in this dissertation explores the factors that influences professionals continued use of video- conferencing applications. The conceptual model investigates the role the perceived risks and benefits play in influencing professionals’ attitude towards VC apps and consequently its active and automatic use

Design revolutions: IASDR 2019 Conference Proceedings. Volume 4: Learning, Technology, Thinking

In September 2019 Manchester School of Art at Manchester Metropolitan University was honoured to host the bi-annual conference of the International Association of Societies of Design Research (IASDR) under the unifying theme of DESIGN REVOLUTIONS. This was the first time the conference had been held in the UK. Through key research themes across nine conference tracks – Change, Learning, Living, Making, People, Technology, Thinking, Value and Voices – the conference opened up compelling, meaningful and radical dialogue of the role of design in addressing societal and organisational challenges. This Volume 4 includes papers from Learning, Technology and Thinking tracks of the conference

Supporting Serendipity through Interactive Recommender Systems in Higher Education

Serendipiteetin käsite viittaa onnekkaisiin sattumuksiin, jossa hyödyllistä tietoa tai muita arvokkaita asioita löydetään yllättäen. Suosittelujärjestelmien tutkimuksessa serendipiteetistä on tullut keskeinen kokemuksellinen tavoite. Ihmisen ja tietokoneen vuorovaikutuksen kannalta olennainen kysymys siitä, kuinka käyttöliittymäsuunnittelu suosittelujärjestelmissä voisi tukea serendipiteetin kokemusta, on kuitenkin saanut vain vähän huomiota. Tässä työssä tutkitaan, kuinka suosittelijajärjestelmän mahdollistamaa serendipiteetin kokemusta voidaan soveltaa tutkimusartikkelien suositteluihin korkeakouluopetuksen kontekstissa. Erityisesti työ tarkastelee suositusjärjestelmäsovellusten käyttöä kehittyvissä maissa, sillä suurin osa kehittyvissä maissa tehdyistä tutkimuksista on keskittynyt pelkästään järjestelmien toteutukseen. Tässä väitöskirjassa kuvataan suosittelujärjestelmien käyttöliittymien suunnittelua ja kehittämistä, tavoitteena ymmärtää paremmin serendipiteetin kokemuksen tukemista käyttöliittymäratkaisuilla. Tutkimalla näitä järjestelmiä kehittyvässä maassa (Pakistan), tämä väitöskirja asettaa suosittelujärjestelmien käytön vastakkain aikaisempien teollisuusmaissa tehtyjen tutkimusten kanssa, ja siten mahdollistaa suositusjärjestelmien soveltamiseen liittyvien kontekstuaalisten ja kulttuuristen haasteiden tarkastelua. Väitöskirja koostuu viidestä empiirisestä käyttäjätutkimuksesta ja kirjallisuuskatsausartikkelista, ja työ tarjoaa uusia käyttöliittymäideoita, avoimen lähdekoodin ohjelmistoratkaisuja sekä empiirisiä analyyseja suositusjärjestelmiin liittyvistä käyttäjäkokemuksista pakistanilaisessa korkeakoulussa. Onnekkaita löytöjä tarkastellaan liittyen tutkimusartikkelien löytämiseen suositusjärjestelmän avulla. Väitöstyö kattaa sekä konstruktiivista että kokeellista tutkimusta. Väitöskirjan artikkelit esittelevät alkuperäistä tutkimusta, jossa kokeillaan erilaisia käyttöliittymämalleja, pohditaan sidosryhmien vaatimuksia, arvioidaan käyttäjien kokemuksia suositelluista artikkeleista ja esitellään tutkimusta suositusjärjestelmien tehtäväkuormitusanalyysistä.Serendipity is defined as the surprising discovery of useful information or other valuable things. In recommender systems research, serendipity has become an essential experiential goal. However, relevant to Human-Computer Interaction, the question of how the user interfaces of recommender systems could facilitate serendipity has received little attention. This work investigates how recommender system-facilitated serendipity can be applied to research article recommendation processes in the context of higher education. In particular, this work investigates the use of recommender system applications in developing countries as most studies in developing countries have focused solely on implementation, rather than user experiences. This dissertation describes the design and development of several user interfaces for recommender systems in an attempt to improve our understanding of serendipity facilitation with the help of user interfaces. By studying these systems in a developing country, this dissertation contrasts the study of recommender systems in developed countries, examining the contextual and cultural challenges associated with the application of recommender systems. This dissertation consists of five empirical user studies and a literature review article, contributing novel user interface designs, open-source software, and empirical analyses of user experiences related to recommender systems in a Pakistani higher education institution. The fortunate discoveries of recommendations are studied in the context of exploring research articles with the help of a recommender system. This dissertation covers both constructive and experimental research. The articles included in this dissertation present original research experimenting with different user interface designs in recommender systems facilitating serendipity, discuss stakeholder requirements, assess user experiences with recommended articles, and present a study on task load analysis of recommender systems. The key findings of this research are that serendipity of recommendations can be facilitated to users with the user interface. Recommender systems can become an instrumental technology in the higher education research and developing countries can benefit from recommender systems applications in higher education institutions

E-Learning Adoption: Designing a Network-Based Educational and Methodological Course on "Humans and Their Health"

This study aims to explore the factors influencing the adoption of e-learning platforms in biology education and examine the impact of online learning on students’ performance. This study investigates the relationships between perceived usefulness, perceived ease of use, attitude toward e-learning, flexibility, content quality, and students’ behavioral intention to adopt e-learning activities. A mixed-methods approach was employed consisting of two phases: a questionnaire survey with structural equation modeling (SEM) to analyze data and an experiment with an independent sample t-test to assess the impact of online learning on student performance. Findings disclosed that perceived usefulness, perceived ease of use, attitude toward e-learning, flexibility, and content quality positively impacted students’ behavioral intention to adopt e-learning and their performance. This study contributes to the theoretical understanding of the factors influencing e-learning adoption in biology education. Practical recommendations are provided for educators, instructional designers, and policymakers to facilitate the implementation of e-learning platforms in biology education. These recommendations include promoting the perceived usefulness and ease of use of e-learning platforms, fostering a positive attitude toward e-learning, enhancing flexibility, ensuring high-quality content, providing training and support for educators, and considering the needs of students with disabilities. Doi: 10.28991/ESJ-2023-07-06-014 Full Text: PD