Canvil: Designerly Adaptation for LLM-Powered User Experiences

Advancements in large language models (LLMs) are poised to spark a proliferation of LLM-powered user experiences. In product teams, designers are often tasked with crafting user experiences that align with user needs. To involve designers and leverage their user-centered perspectives to create effective and responsible LLM-powered products, we introduce the practice of designerly adaptation for engaging with LLMs as an adaptable design material. We first identify key characteristics of designerly adaptation through a formative study with designers experienced in designing for LLM-powered products (N=12). These characteristics are 1) have a low technical barrier to entry, 2) leverage designers' unique perspectives bridging users and technology, and 3) encourage model tinkering. Based on this characterization, we build Canvil, a Figma widget that operationalizes designerly adaptation. Canvil supports structured authoring of system prompts to adapt LLM behavior, testing of adapted models on diverse user inputs, and integration of model outputs into interface designs. We use Canvil as a technology probe in a group-based design study (6 groups, N=17) to investigate the implications of integrating designerly adaptation into design workflows. We find that designers are able to iteratively tinker with different adaptation approaches and reason about interface affordances to enhance end-user interaction with LLMs. Furthermore, designers identified promising collaborative workflows for designerly adaptation. Our work opens new avenues for collaborative processes and tools that foreground designers' user-centered expertise in the crafting and deployment of LLM-powered user experiences

An adaptive technique for content-based image retrieval

We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs—a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the user's need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the user's need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search

Content adaptation for an adaptive hypermedia system

The aim of this paper is presenting the recommendation module of the Mathematics Collaborative Learning Platform (PCMAT). PCMAT is an Adaptive Educational Hypermedia System (AEHS), with a constructivist approach, which presents contents and activities adapted to the characteristics and learning style of students of mathematics in basic schools. The recommendation module is responsible for choosing different learning resources for the platform, based on the user's characteristics and performance. Since the main purpose of an adaptive system is to provide the user with content and interface adaptation, the recommendation module is integral to PCMAT’s adaptation model

Intelligent Context-Aware and Adaptive Interface for Mobile LBS

Context-aware user interface plays an important role in many human-computer Interaction tasks of location based services. Although spatial models for context-aware systems have been studied extensively, how to locate specific spatial information for users is still not well resolved, which is important in the mobile environment where location based services users are impeded by device limitations. Better context-aware human-computer interaction models of mobile location based services are needed not just to predict performance outcomes, such as whether people will be able to find the information needed to complete a human-computer interaction task, but to understand human processes that interact in spatial query, which will in turn inform the detailed design of better user interfaces in mobile location based services. In this study, a context-aware adaptive model for mobile location based services interface is proposed, which contains three major sections: purpose, adjustment, and adaptation. Based on this model we try to describe the process of user operation and interface adaptation clearly through the dynamic interaction between users and the interface. Then we show how the model applies users’ demands in a complicated environment and suggested the feasibility by the experimental results

Fusione Termonucleare Controllata - Il punto sulla ricerca

Adapting user interfaces to different contexts of use is essential to enhance usability. Adaptation enhances user satisfaction by meeting changing context of use requirements. However, given the variety of contexts of use, and the significant amount of involved information and contextual treatments, transformations of user interface models that consider adaptation become complex. This complexity becomes a challenge when trying to add new adaptation rules or modify the transformation. In this paper, we present “Adapt-first”, an adaptation approach intended to simplify adaptation within model based user interfaces. It capitalizes on differentiating adaptations and concretization via two transformation techniques: Concretization and translation. First-Adapt approach aims at reducing complexity and maintenance efforts of transformations from a model to another

Adapt-First: a MDE Transformation Approach for Supporting User Interface Adaptation

Adapting user interfaces to different contexts of use is essential to enhance usability. Adaptation enhances user satisfaction by meeting changing context of use requirements. However, given the variety of contexts of use, and the significant amount of involved information and contextual treatments, transformations of user interface models that consider adaptation become complex. This complexity becomes a challenge when trying to add new adaptation rules or modify the transformation. In this paper, we present “Adapt-first”, an adaptation approach intended to simplify adaptation within model based user interfaces. It capitalizes on differentiating adaptations and concretization via two transformation techniques: concretization and translation. First-Adapt approach aims at reducing complexity and maintenance efforts of transformations from a model to another

A Decision Theoretic Approach for Interface Agent Development

The complexity of current software applications is overwhelming users. The need exists for intelligent interface agents to address the problem of increasing taskload that is overwhelming the human user. Interface agents could help alleviate user taskload by extracting and analyzing relevant information, and providing information abstractions of that information, and providing timely, beneficial assistance to users. These agents could communicate with the user through the existing user interface and also adapt to user needs and behaviors. Central to providing assistance to a user is the issue of correctly determining the user\u27s intent. This dissertation presents an effective, efficient, and extensible decision-theoretic architecture for user intent ascription. The multi-agent architecture, the Core Interface Agent architecture, provides a dynamic, uncertainty-based knowledge representation for modeling the inherent ambiguity in ascribing user intent. The knowledge representation, a Bayesian network, provides an intuitive, mathematically sound way of determining the likelihood a user is pursuing a goal. This likelihood, combined with the utility of offering assistance to the user, provides a decision-theoretic approach to offering assistance to the user. The architecture maintains an accurate user model of the user\u27s goals within a target system environment. The on-line maintenance of the user model is performed by a collection of correction adaptation agents. Because the decision-theoretic methodology is domain-independent, this new methodology for user intent ascription is readily extensible over new application domains. Furthermore, it also offers the ability to bootstrap intent understanding without the need for often lengthy and costly knowledge elicitation. Thus, as a side benefit, the process can mitigate the classic knowledge acquisition bottleneck problem

An analysis of the generative user interface pattern structure

Current business information systems extensively rely on graphical user interfaces (GUIs). These sub-systems enable the interaction between the end user and application kernel services that are essential for the business process instances. Due to dynamic and rapid changes of both business processes and their required services, a strong need for the quick adaptation of GUIs to the occurring changes arose. As both efficiency and usability are essential for the GUI adaptation, model-based development processes that involve patterns and their instantiation for specific GUI contexts have been suggested by ongoing research. Being based on human computer interaction patterns, the new kind of pattern needs to be formalized in order to enable the automated processing of configurable instances by generator tools. However, current research is still at the edge to express the concepts for such generative user interface patterns. Crucial factors and impacts of those patterns have not been described sufficiently yet so that a standardized format for the expression of variability is still missing. With our work, we briefly review the current state on modeling user interface patterns and their requirement aspects. The ultimate objective of this paper is the development of an analysis model that is able to express both the structure and variability concerns of user interface patterns in detail. To evaluate and illustrate the analysis model concepts, selected user interface pattern instances are modeled via object models. As result, a detailed description of generative user interface patterns is achieved, which can be applied as a basis for the verification of recent approaches of model- and pattern-based GUI development or even the synthesis of a dedicated user interface pattern language