Understanding the Role of Interactivity and Explanation in Adaptive Experiences

Adaptive experiences have been an active area of research in the past few decades, accompanied by advances in technology such as machine learning and artificial intelligence. Whether the currently ongoing research on adaptive experiences has focused on personalization algorithms, explainability, user engagement, or privacy and security, there is growing interest and resources in developing and improving these research focuses. Even though the research on adaptive experiences has been dynamic and rapidly evolving, achieving a high level of user engagement in adaptive experiences remains a challenge. %????? This dissertation aims to uncover ways to engage users in adaptive experiences by incorporating interactivity and explanation through four studies. Study I takes the first step to link the explanation and interactivity in machine learning systems to facilitate users\u27 engagement with the underlying machine learning model with the Tic-Tac-Toe game as a use case. The results show that explainable machine learning (XML) systems (and arguably XAI systems in general) indeed benefit from mechanisms that allow users to interact with the system\u27s internal decision rules. Study II, III, and IV further focus on adaptive experiences in recommender systems in specific, exploring the role of interactivity and explanation to keep the user “in-the-loop” in recommender systems, trying to mitigate the ``filter bubble\u27\u27 problem and help users in self-actualizing by supporting them in exploring and understanding their unique tastes. Study II investigates the effect of recommendation source (a human expert vs. an AI algorithm) and justification method (needs-based vs. interest-based justification) on professional development recommendations in a scenario-based study setting. The results show an interaction effect between these two system aspects: users who are told that the recommendations are based on their interests have a better experience when the recommendations are presented as originating from an AI algorithm, while users who are told that the recommendations are based on their needs have a better experience when the recommendations are presented as originating from a human expert. This work implies that while building the proposed novel movie recommender system covered in study IV, it would provide a better user experience if the movie recommendations are presented as originating from algorithms rather than from a human expert considering that movie preferences (which will be visualized by the movies\u27 emotion feature) are usually based on users\u27 interest. Study III explores the effects of four novel alternative recommendation lists on participants’ perceptions of recommendations and their satisfaction with the system. The four novel alternative recommendation lists (RSSA features) which have the potential to go beyond the traditional top N recommendations provide transparency from a different level --- how much else does the system learn about users beyond the traditional top N recommendations, which in turn enable users to interact with these alternative lists by rating the initial recommendations so as to correct or confirm the system\u27s estimates of the alternative recommendations. The subjective evaluation and behavioral analysis demonstrate that the proposed RSSA features had a significant effect on the user experience, surprisingly, two of the four RSSA features (the controversial and hate features) perform worse than the traditional top-N recommendations on the measured subjective dependent variables while the other two RSSA features (the hipster and no clue items) perform equally well and even slightly better than the traditional top-N (but this effect is not statistically significant). Moreover, the results indicate that individual differences, such as the need for novelty and domain knowledge, play a significant role in users’ perception of and interaction with the system. Study IV further combines diversification, visualization, and interactivity, aiming to encourage users to be more engaged with the system. The results show that introducing emotion as an item feature into recommender systems does help in personalization and individual taste exploration; these benefits are greatly optimized through the mechanisms that diversify recommendations by emotional signature, visualize recommendations on the emotional signature, and allow users to directly interact with the system by tweaking their tastes, which further contributes to both user experience and self-actualization. This work has practical implications for designing adaptive experiences. Explanation solutions in adaptive experiences might not always lead to a positive user experience, it highly depends on the application domain and the context (as studied in all four studies); it is essential to carefully investigate a specific explanation solution in combination with other design elements in different fields. Introducing control by allowing for direct interactivity (vs. indirect interactivity) in adaptive systems and providing feedback to users\u27 input by integrating their input into the algorithms would create a more engaging and interactive user experience (as studied in Study I and IV). And cumulatively, appropriate direct interaction with the system along with deliberate and thoughtful designs of explanation (including visualization design with the application environment fully considered), which are able to arouse user reflection or resonance, would potentially promote both user experience and user self-actualization

Supporting flexible workflow processes with a progression model

Users require flexibility when interacting with information systems to contend with changing business processes and to support diverse workflow. Model-based user interface design can accommodate flexible business processes by integrating workflow modelling with other modelling approaches. We present a workflow model, the progression model, to help in developing systems that support flexible business processes. The progression model tracks a user’s interaction with an application as a set of data elements we refer to as a workflow transaction. The steps a user takes to create a workflow transaction and the state of the workflow transaction at each step is made explicit. By making the workflow status and workflow transaction state explicit, the user can change the order of the steps in a process, manage multiple workflow transactions, keep track of data as it is accumulated, and so on. The intent is to provide the user with a mechanism to deal with partial information, interrupted and concurrent workflow transaction entry, and the processing of multiple workflow transactions. This thesis describes the progression model, an XML-compliant notation to specify the progression model, and a prototype system

Task models as basis for requirements engineering and software execution

ABSTRACT In this paper we discuss an approach linking GUI specifications to abstract dialog models. Both specifications are based on task models describing behavioral features. It will be shown how first prototypes of interactive systems, which are generated from user interface models, can help to capture requirements. Users can interactively play with prototypes. Tool support is also provided for co-operative work of different users, which starts with abstract canonical prototypes that can evolve to concrete GUI specifications

Refinement for user interface designs

Formal approaches to software development require that we correctly describe (or specify) systems in order to prove properties about our proposed solution prior to building it. We must then follow a rigorous process to transform our specification into an implementation to ensure that the properties we have proved are retained. Different transformation, or refinement, methods exist for different formal methods, but they all seek to ensure that we can guide the transformation in a way which preserves the desired properties of the system. Refinement methods also allow us to subsequently compare two systems to see if a refinement relation exists between the two. When we design and build the user interfaces of our systems we are similarly keen to ensure that they have certain properties before we build them. For example, do they satisfy the requirements of the user? Are they designed with known good design principles and usability considerations in mind? Are they correct in terms of the overall system specification? However, when we come to implement our interface designs we do not have a defined process to follow which ensures that we maintain these properties as we transform the design into code. Instead, we rely on our judgement and belief that we are doing the right thing and subsequent user testing to ensure that our final solution remains useable and satisfactory. We suggest an alternative approach, which is to define a refinement process for user interfaces which will allow us to maintain the same rigorous standards we apply to the rest of the system when we implement our user interface designs

Ifaces: Adaptative user interfaces for ambient intelligence

Proceedings of the IADIS International Conference on Interfaces and Human Computer Interaction. Amsterdam, The Netherlands 25-27 July 2008In this paper we present an ontology language to model an environment and its graphical user interface in the field of ambient intelligence. This language allows a simple definition of the environment and automatically produces its associated interaction interface. The interface dynamically readjusts to the characteristics of the environment and the available devices. Therefore it adapts to the necessities of the people who have to use it and their resources. The system has been developed and tested employing a real ambient intelligence environment.This work has been partly funded by HADA project number TIN2007 – 64718 and the UAM – Indra Chair in Ambient Intelligence

FlexiXML : a portable user interface rendering engine for UsiXML

A considerable amount of effort in software development is dedicated to the user interaction layer.Given the complexity inherent to the development of this layer, it is important to be able to analyse the concepts and ideas being used in the development of a given user interface. This analysis should be performed as early as possible. Model- based user interface development provides a solution to this problem by providing developers with tools that enable both modeling, and reasoning about, user interfaces at different levels of abstraction. Of particular interest here, is the possibility of animating the models to generate actual user interfaces. This paper describes FlexiXML, a tool that performs the rendering and animation of user interfaces described in the UsiXML modeling language

Using formal models to design user interfaces a case study

The use of formal models for user interface design can provide a number of benefits. It can help to ensure consistency across designs for multiple platforms, prove properties such as reachability and completeness and, perhaps most importantly, can help incorporate the user interface design process into a larger, formally-based, software development process. Often, descriptions of such models and examples are presented in isolation from real-world practice in order to focus on particular benefits, small focused examples or the general methodology. This paper presents a case study of developing the user interface to a new software application using a particular pair of formal models, presentation models and presentation interaction models. The aim of this study was to practically apply the use of formal models to the design process of a UI for a new software application. We wanted to determine how easy it would be to integrate such models into our usual development process and to find out what the benefits, and difficulties, of using such models were. We will show how we used the formal models within a user-centred design process, discuss what effect they had on this process and explain what benefits we perceived from their use

User Interface Abstraction for enabling TV set based Inclusive Access to the Information Society

199 p.The television (TV) set is present in most homes worldwide, and is the most used Information and Communication Technology (ICT). Despite its large implantation in the market, the interactive services consumption on TV set is limited. This thesis focuses on overcoming the following limiting factors: (i) limited Human Computer Interaction and (ii) lack of considering user’s real life context in the digital television (dTV) service integration strategy. Making interactive services accessible to TV set’s large user base, and especially to the most vulnerable ones, is understood as the path to integrate the mankind with the information society. This thesis explores the use of user interface abstraction technologies to reach the introduced goals. The main contributions of this thesis are: (i) an approach to enable the universally accessible remote control of the TV set, (ii) an approach for the provision of universally accessible interactive services through TV sets, and (iii) an approach for the provision of universally accessible services in the TV user’s real life context. We have implemented the contributing approaches for different use cases, and we have evaluated them with real users, achieving good results

A pattern-oriented and model-driven architecture for interactive systems

Day-to-day experiences suggest that it is not enough to approach a complex design equipped with design tips, guidelines, and hints. Developers must also be able to use proven solutions emerging from the best design practices to solve new design challenges. Without these, the designer is unable to properly apply guidelines or take full advantage of the power of technology, resulting therefore in poor performance, poor scalability, and poor usability. Furthermore, the designer might "reinvent the wheel" when attempting to implement a design solution. A number of design problems continue to arise, such as: (1) decoupling the various aspects of interactive systems (for example, business logic, the UI, navigation, and information architecture) and (2) isolating platform specifics from the concerns common to all interactive systems. In the context of a proposal for a Pattern-Oriented and Model-driven Architecture (POMA) for interactive systems, this thesis identifies an extensive list of pattern categories and types of models aimed at providing a pool of proven solutions to these problems. The models of patterns span several levels of abstraction, such as domain, task, dialog, presentation and layout. The proposed POMA architecture illustrates how several individual models can be combined at different levels of abstraction into heterogeneous structures which can then be used as building blocks in the development of interactive systems. This document is divided into six chapters: the first chapter presents a background and related work on "Patterns" in general and on various architectures for interactive systems development such as "N-tiers architectures", "Pattern-Oriented Design" (POD), "Pattern- Supported Approach" (PSA), and "Model-Driven Architecture" (MDA). The second chapter introduces the research topic with its objectives, its limits, the research methodology, and research steps. The third chapter describes primarily the most important parts of the research which is the development of a new architecture called Pattern-Oriented and Model-Driven Architecture, facilitating the development of interactive systems including fundamentals and key concepts, an overview, justifications versus N-tiers, POD, PSA, and MDA architectures and specifications. The fourth chapter describes architectural levels and categories of patterns used in POMA. The fifth chapter describes the categories of models used in POMA. The sixth chapter presents an exploratory case study applied to the architecture proposed in this research. The last chapter presents a conclusion on this research work and its expected evolution in the future