Evaluation of Recommender Systems for Technology-Enhanced Learning: Challenges and Possible Solutions

Heleou, S., Drachsler, H., & Gillet, D. (2009). Evaluation of Recommender Systems for Technology-Enhanced Learning: Challenges and Possible Solutions. 1st workshop on Context-aware Recommender Systems for Learning at the Alpine Rendez-Vous. November, 30-December, 3, 2009, Garmisch-Patenkirchen, Germany.This paper discusses challenges and possible solutions of recommender systems for Technology-Enhanced Learning (TEL). It also briefly presents the the 3A contextual recommender system and explores its applicability and evaluation in the context of learners using multiple Web 2.0 applications

Preface to Proceedings of the 1st Workshop on Recommender Systems in Technology Enhanced Learning (RecSysTEL 2010)

AbstractTechnology enhanced learning (TEL) aims to design, develop and test socio-technical innovations that will support and enhance learning practices of both individuals and organisations. It is an application domain that generally addresses all types of technology research & development aiming to support teaching and learning activities. Information retrieval is a pivotal activity in TEL, and the deployment of recommender systems has attracted increased interest during the past years.Recommendation methods, techniques and systems open an interesting new approach to facilitate and support learning and teaching. There are plenty of resources available on the Web, both in terms of digital learning content and people resources (e.g. other learners, experts, tutors) that can be used to facilitate teaching and learning tasks. The challenge is to develop, deploy and evaluate systems that provide learners and teachers with meaningful guidance in order to help identify suitable learning resources from a potentially overwhelming variety of choices.The 1st Workshop on Recommender Systems for Technology Enhanced Learning (RecSysTEL) builds upon the tradition of a series of workshops on Social Information Retrieval for Technology Enhanced Learning (SIRTEL), Context-Aware Recommendation for Learning and Towards User Modelling and Adaptive Systems for All (TUMAS-A)a. RecSysTEL was organised jointly by the 4th ACM Conference on Recommender Systems (RecSys 2010) and the 5th European Conference on Technology Enhanced Learning (EC-TEL 2010), on 29–30 September 2010 in Barcelona, Spain. Its main goal was to bring together researchers and practitioners who are working on topics related to the design, development and testing of recommender systems in educational settings as well as present the current status of research in this area and create cross-disciplinary liaisons between the RecSys and ECTEL communities. Overall, its contributions outline the rich potential of TEL as an application area for recommender systems and identify the challenges of developing such systems in a TEL context

Identifying the Goal, User model and Conditions of Recommender Systems for Formal and Informal Learning

Drachsler, H., Hummel, H. G. K., & Koper, R. (2009). Identifying the Goal, User model and Conditions of Recommender Systems for Formal and Informal Learning. Journal of Digital Information, 10(2), 4-24.The following article addresses open questions of the discussions in the first SIRTEL workshop at the EC-TEL conference 2007. It argues why personal recommender systems have to be adjusted to the specific characteristics of learning to support lifelong learners. Personal recommender systems strongly depend on the context or domain they operate in, and it is often not possible to take one recommender system from one context and transfer it to another context or domain. The article describes a number of distinct differences for personalized recommendation to consumers in contrast to recommendations to learners. Similarities and differences are translated into specific demands for learning and specific requirements for personal recommendation systems. It further suggests an evaluation approach for recommender systems in technology-enhanced learning.The work on this publication has been sponsored by the TENCompetence Integrated Project that is funded by the European Commission's 6th Framework Programme, priority IST/Technology Enhanced Learning. Contract 027087 [http://www.tencompetence.org

A trust-based social recommender for teachers

Fazeli, S., Drachsler, H., Brouns, F., & Sloep, P. B. (2012). A trust-based social recommender for teachers. In N. Manouselis, H. Drachsler, K. Verbert, & O. C. Santos (Eds.), 2nd Workshop on Recommender Systems for Technology Enhanced Learning (RecSysTEL 2012) in conjunction with the 7th European Conference on Technology Enhanced Learning (EC-TEL 2012) (pp. 49-60). September, 18-19, 2012, Saarbrücken, Germany.Online communities and networked learning provide teachers with social learning opportunities to interact and collaborate with others in order to develop their personal and professional skills. In this paper, Learning Networks are presented as an open infrastructure to provide teachers with such learning opportunities. However, with the large number of learning resources produced everyday, teachers need to find out what are the most suitable resources for them. In this paper, recommender systems are introduced as a potential solution to address this issue. Unfortunately, most of the educational recommender systems cannot make accurate recommendations due to the sparsity of the educational datasets. To overcome this problem, we propose a research approach that describes how one may take advantage of the social data which are obtained from monitoring the activities of teachers while they are using our social recommender.NELLL, Open Discovery Space (ODS

Issues and considerations regarding sharable data sets for recommender systems in technology enhanced learning

Drachsler, H., Bogers, T., Vuorikari, R., Verbert, K., Duval, E., Manouselis, N., Beham, G., Lindstaedt, S., Stern, H., Friedrich, M., & Wolpers, M. (2010, 28 September). Issues and Considerations regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning. Presentation at the 1st Workshop Recommender Systems in Technology Enhanced Learning (RecSysTEL) in conjunction with 5th European Conference on Technology Enhanced Learning (EC-TEL 2010): Sustaining TEL: From Innovation to Learning and Practice, Barcelona, Spain.The presentation is based on the positioning paper of the dataTEL Theme Team of the STELLAR Network of Excellence (http://www.teleurope.eu/pg/groups/9405/datatel/) that addresses the lack of educational data sets in TEL and present ideas to overcome this situation. The accompanying paper: Issues and Considerations regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning, can be found at http://www.sciencedirect.com/science/journal/18770509 and a pre-print is available in our Dspace repository and at scribd. The presentation starts with a description of the current situation where almost none educational data sets are publicly available. This is a strange situation as plenty of data is saved on a daily base in LMS like Moodle, Blackboard. In other domains like e-commerce it is a common practice to use publicly available data sets from different application environments (e.g. Yahoo, MovieLens) in order to evaluate algorithms and create new data products. These data sets are for instance used as benchmarks to develop new recommendation algorithms and compare them to other algorithms in certain settings. Recommender systems are also increasingly applied in Technology Enhanced Learning field but it is still an application area that lacks such publicly available data sets. Although there is a lot of research conducted on recommender systems in TEL, they lack data sets that would allow the experimental evaluation of the performance of different recommendation algorithms using comparable, interoperable, and reusable data sets. This leads to awkward experimentation and testing such as using data sets from movies in order to evaluate educational recommendation algorithms.Stella

STELLAR Alpine Rendez-Vous White Paper

Drachsler, H., Verbert, K., Sicilia, M. A., Wolpers, M., Manouselis, N., Vuorikari, R., Lindstaedt, S., & Fischer, F. (2011). dataTEL - Datasets for Technology Enhanced Learning. STELLAR Alpine Rendez-Vous White Paper. Alpine Rendez-Vous 2011 White paper collection, Nr. 13., France (2011) Accessible at: http://oa.stellarnet.eu/open-archive/browse?resource=6756_v1The dataTEL white paper develop during the dataTEL workshop at the ARV2011. The workshop was motivated by the issue that very less educational datasets are publicly available in TEL, so that the outcomes of different TEL adaptive applications and recommender systems that support personalised learning are hardly comparable. In other domains like in e-commerce it is a common practise to use different datasets as benchmarks to evaluate recommender systems algorithms to make the results comparable (MovieLens, Book-Crossing, EachMovie dataset). So far, no universally valid knowledge exists in TEL on algorithm that can be successfully applied in a certain learning setting to personalise learning. Having a collection of datasets could be a first major step towards a theory of personalisation within TEL that can be based on empirical experiments with verifiable and valid results. Therefore, the main objective of the dataTEL workshop was to explore suitable datasets for TEL with a specific focus on recommender and adaptive information systems that can take advantage of these datasets. In this context, new challenges emerge like unclear legal protection rights and privacy issues, suitable policies and formats to share data, required preprocessing procedures and rules to create sharable datasets, common evaluation criteria for recommender systems in TEL and how a dataset driven future in TEL could look like.dataTEL, NeLLL AlterEgo, STELLAR, MAVSE

A Hybrid Recommender Strategy on an Expanded Content Manager in Formal Learning

The main topic of this paper is to find ways to improve learning in a formal Higher Education Area. In this environment, the teacher publishes or suggests contents that support learners in a given course, as supplement of classroom training. Generally, these materials are pre-stored and not changeable. These contents are typically published in learning management systems (the Moodle platform emerges as one of the main choices) or in sites created and maintained on the web by teachers themselves. These scenarios typically include a specific group of students (class) and a given period of time (semester or school year). Contents reutilization often needs replication and its update requires new edition and new submission by teachers. Normally, these systems do not allow learners to add new materials, or to edit existing ones. The paper presents our motivations, and some related concepts and works. We describe the concepts of sequencing and navigation in adaptive learning systems, followed by a short presentation of some of these systems. We then discuss the effects of social interaction on the learners’ choices. Finally, we refer some more related recommender systems and their applicability in supporting learning. One central idea from our proposal is that we believe that students with the same goals and with similar formal study time can benefit from contents' assessments made by learners that already have completed the same courses and have studied the same contents. We present a model for personalized recommendation of learning activities to learners in a formal learning context that considers two systems. In the extended content management system, learners can add new materials, select materials from teachers and from other learners, evaluate and define the time spent studying them. Based on learner profiles and a hybrid recommendation strategy, combining conditional and collaborative filtering, our second system will predict learning activities scores and offers adaptive and suitable sequencing learning contents to learners. We propose that similarities between learners can be based on their evaluation interests and their recent learning history. The recommender support subsystem aims to assist learners at each step suggesting one suitable ordered list of LOs, by decreasing order of relevance. The proposed model has been implemented in the Moodle Learning Management System (LMS), and we present the system’s architecture and design. We will evaluate it in a real higher education formal course and we intend to present experimental results in the near future