A proposal for the evaluation of adaptive information retrieval systems using simulated interaction

The Centre for Next Generation Localisation (CNGL) is involved in building interactive adaptive systems which combine Information Retrieval (IR), Adaptive Hypermedia (AH) and adaptive web techniques and technologies. The complex functionality of these systems coupled with the variety of potential users means that the experiments necessary to evaluate such systems are difficult to plan, implement and execute. This evaluation requires both component-level scientific evaluation and user-based evaluation. Automated replication of experiments and simulation of user interaction would be hugely beneficial in the evaluation of adaptive information retrieval systems (AIRS). This paper proposes a methodology for the evaluation of AIRS which leverages simulated interaction. The hybrid approach detailed combines: (i) user-centred methods for simulating interaction and personalisation; (ii) evaluation metrics that combine Human Computer Interaction (HCI), AH and IR techniques; and (iii) the use of qualitative and quantitative evaluations. The benefits and limitations of evaluations based on user simulations are also discussed

Supporting authoring of adaptive hypermedia

It is well-known that students benefit from personalised attention. However, frequently teachers are unable to provide this, most often due to time constraints. An Adaptive Hypermedia (AH) system can offer a richer learning experience, by giving personalised attention to students. The authoring process, however, is time consuming and cumbersome. Our research explores the two main aspects to authoring of AH: authoring of content and adaptive behaviour. The research proposes possible solutions, to overcome the hurdles towards acceptance of AH in education. Automation methods can help authors, for example, teachers could create linear lessons and our prototype can add content alternatives for adaptation. Creating adaptive behaviour is more complex. Rule-based systems, XML-based conditional inclusion, Semantic Web reasoning and reusable, portable scripting in a programming language have been proposed. These methods all require specialised knowledge. Hence authoring of adaptive behaviour is difficult and teachers cannot be expected to create such strategies. We investigate three ways to address this issue. 1. Reusability: We investigate limitations regarding adaptation engines, which influence the authoring and reuse of adaptation strategies. We propose a metalanguage, as a supplement to the existing LAG adaptation language, showing how it can overcome such limitations. 2. Standardisation: There are no widely accepted standards for AH. The IMSLearning Design (IMS-LD) specification has similar goals to Adaptive Educational Hypermedia (AEH). Investigation shows that IMS-LD is more limited in terms of adaptive behaviour, but the authoring process focuses more on learning sequences and outcomes. 3. Visualisation: Another way is to simplify the authoring process of strategies using a visual tool. We define a reference model and a tool, the Conceptual Adaptation Model (CAM) and GRAPPLE Authoring Tool (GAT), which allow specification of an adaptive course in a graphical way. A key feature is the separation between content, strategy and adaptive course, which increases reusability compared to approaches that combine all factors in one model

Adaptive hypermedia for education and training

Adaptive hypermedia (AH) is an alternative to the traditional, one-size-fits-all approach in the development of hypermedia systems. AH systems build a model of the goals, preferences, and knowledge of each individual user; this model is used throughout the interaction with the user to adapt to the needs of that particular user (Brusilovsky, 1996b). For example, a student in an adaptive educational hypermedia system will be given a presentation that is adapted specifically to his or her knowledge of the subject (De Bra & Calvi, 1998; Hothi, Hall, & Sly, 2000) as well as a suggested set of the most relevant links to proceed further (Brusilovsky, Eklund, & Schwarz, 1998; Kavcic, 2004). An adaptive electronic encyclopedia will personalize the content of an article to augment the user's existing knowledge and interests (Bontcheva & Wilks, 2005; Milosavljevic, 1997). A museum guide will adapt the presentation about every visited object to the user's individual path through the museum (Oberlander et al., 1998; Stock et al., 2007). Adaptive hypermedia belongs to the class of user-adaptive systems (Schneider-Hufschmidt, Kühme, & Malinowski, 1993). A distinctive feature of an adaptive system is an explicit user model that represents user knowledge, goals, and interests, as well as other features that enable the system to adapt to different users with their own specific set of goals. An adaptive system collects data for the user model from various sources that can include implicitly observing user interaction and explicitly requesting direct input from the user. The user model is applied to provide an adaptation effect, that is, tailor interaction to different users in the same context. In different kinds of adaptive systems, adaptation effects could vary greatly. In AH systems, it is limited to three major adaptation technologies: adaptive content selection, adaptive navigation support, and adaptive presentation. The first of these three technologies comes from the fields of adaptive information retrieval (IR) and intelligent tutoring systems (ITS). When the user searches for information, the system adaptively selects and prioritizes the most relevant items (Brajnik, Guida, & Tasso, 1987; Brusilovsky, 1992b)

Applying digital content management to support localisation

The retrieval and presentation of digital content such as that on the World Wide Web (WWW) is a substantial area of research. While recent years have seen huge expansion in the size of web-based archives that can be searched efficiently by commercial search engines, the presentation of potentially relevant content is still limited to ranked document lists represented by simple text snippets or image keyframe surrogates. There is expanding interest in techniques to personalise the presentation of content to improve the richness and effectiveness of the user experience. One of the most significant challenges to achieving this is the increasingly multilingual nature of this data, and the need to provide suitably localised responses to users based on this content. The Digital Content Management (DCM) track of the Centre for Next Generation Localisation (CNGL) is seeking to develop technologies to support advanced personalised access and presentation of information by combining elements from the existing research areas of Adaptive Hypermedia and Information Retrieval. The combination of these technologies is intended to produce significant improvements in the way users access information. We review key features of these technologies and introduce early ideas for how these technologies can support localisation and localised content before concluding with some impressions of future directions in DCM

Defining adaptation in a generic multi layer model : CAM: the GRAPPLE conceptual adaptation model

Authoring of Adaptive Hypermedia is a difficult and time consuming task. Reference models like LAOS and AHAM separate adaptation and content in different layers. Systems like AHA! offer graphical tools based on these models to allow authors to define adaptation without knowing any adaptation language. The adaptation that can be defined using such tools is still limited. Authoring systems like MOT are more flexible, but usability of adaptation specification is low. This paper proposes a more generic model which allows the adaptation to be defined in an arbitrary number of layers, where adaptation is expressed in terms of relationships between concepts. This model allows the creation of more powerful yet easier to use graphical authoring tools. This paper presents the structure of the Conceptual Adaptation Models used in adaptive applications created within the GRAPPLE adaptive learning environment, and their representation in a graphical authoring tool