Quality evaluation of open educational resources

Open Educational Resources (OER) are free and open-licensed educational materials widely used for learning. OER quality assessment has become essential to support learners and teachers in finding high-quality OERs, and to enable online learning repositories to improve their OERs. In this work, we establish a set of evaluation metrics that assess OER quality in OER authoring tools. These metrics provide guidance to OER content authors to create high-quality content. The metrics were implemented and evaluated within SlideWiki, a collaborative OpenCourseWare platform that provides educational materials in presentation slides format. To evaluate the relevance of the metrics, a questionnaire is conducted among OER expert users. The evaluation results indicate that the metrics address relevant quality aspects and can be used to determine the overall OER quality

OER Development and Promotion. Outcomes of an International Research Project on the OpenCourseWare Model

In this paper, we describe the successful results of an international research project focused on the use of Web technology in the educational context. The article explains how this international project, funded by public organizations and developed over the last two academic years, focuses on the area of open educational resources (OER) and particularly the educational content of the OpenCourseWare (OCW) model. This initiative has been developed by a research group composed of researchers from three countries. The project was enabled by the Universidad Politécnica de Madrid OCW Office�s leadership of the Consortium of Latin American Universities and the distance education know-how of the Universidad Técnica Particular de Loja (UTPL, Ecuador). We give a full account of the project, methodology, main outcomes and validation. The project results have further consolidated the group, and increased the maturity of group members and networking with other groups in the area. The group is now participating in other research projects that continue the lines developed her

Quality Prediction of Open Educational Resources A Metadata-based Approach

In the recent decade, online learning environments have accumulated millions of Open Educational Resources (OERs). However, for learners, finding relevant and high quality OERs is a complicated and time-consuming activity. Furthermore, metadata play a key role in offering high quality services such as recommendation and search. Metadata can also be used for automatic OER quality control as, in the light of the continuously increasing number of OERs, manual quality control is getting more and more difficult. In this work, we collected the metadata of 8,887 OERs to perform an exploratory data analysis to observe the effect of quality control on metadata quality. Subsequently, we propose an OER metadata scoring model, and build a metadata-based prediction model to anticipate the quality of OERs. Based on our data and model, we were able to detect high-quality OERs with the F1 score of 94.6%. © 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Detecting Communities and Analysing Interactions with Learning Objects in Online Learning Repositories

The widespread use of online learning object repositories has raised the need of studies that assess the quality of their contents, and their user’s performance and engagement. The present research addresses two fundamental problems that are central to that need: the need to explore user interaction with these repositories and the detection of emergent communities of users. The current dissertation approaches those directions through investigating and mining the Khan Academy repository as a free, open access, popular online learning repository addressing a wide content scope. It includes large numbers of different learning objects such as instructional videos, articles, and exercises. In addition to a large number of users. Data was collected using the repository’s public application programming interfaces combined with Web scraping techniques to gather data and user interactions. Different research activities were carried out to generate useful insights out of the gathered data. We conducted descriptive analysis to investigate the learning repository and its core features such as growth rate, popularity, and geographical distribution. A number of statistical and quantitative analysis were applied to examine the relation between the users’ interactions and different metrics related to the use of learning objects in a step to assess the users’ behaviour. We also used different Social Network Analysis (SNA) techniques on a network graph built from a large number of user interactions. The resulting network consisted of more than 3 million interactions distributed across more than 300,000 users. The type of those interactions is questions and answers posted on Khan Academy’s instructional videos (more than 10,000 video). In order to analyse this graph and explore the social network structure, we studied two different community detection algorithms to identify the learning interactions communities emerged in Khan Academy then we compared between their effectiveness. After that, we applied different SNA measures including modularity, density, clustering coefficients and different centrality measures in order to assess the users’ behaviour patterns and their presence. Using descriptive analysis, we discovered many characteristics and features of the repository. We found that the number of learning objects in Khan Academy’s repository grows linearly over time, more than 50% of the users do not complete the watched videos, and we found that the average duration for video lessons 5 to 10 minutes which aligns with the recommended duration in literature. By applying community detection techniques and social network analysis, we managed to identify learning communities in Khan Academy’s network. The size distribution of those communities found to follow the power-law distribution which is the case of many real-world networks. Those learning communities are related to more than one domain which means the users are active and interacting across domains. Different centrality measures we applied to focus on the most influential players in those communities. Despite the popularity of online learning repositories and their wide use, the structure of the emerged learning communities and their social networks remain largely unexplored. Our findings could be considered initial insights that may help researchers and educators in better understanding online learning repositories, the learning process inside those repositories, and learner behaviou

Metadata analysis of open educational resources

Open Educational Resources (OERs) are openly licensed educational materials that are widely used for learning. Nowadays, many online learning repositories provide millions of OERs. Therefore, it is exceedingly difficult for learners to find the most appropriate OER among these resources. Subsequently, the precise OER metadata is critical for providing high-quality services such as search and recommendation. Moreover, metadata facilitates the process of automatic OER quality control as the continuously increasing number of OERs makes manual quality control extremely difficult. This work uses the metadata of 8,887 OERs to perform an exploratory data analysis on OER metadata. Accordingly, this work proposes metadata-based scoring and prediction models to anticipate the quality of OERs. Based on the results, our analysis demonstrated that OER metadata and OER content qualities are closely related, as we could detect high-quality OERs with an accuracy of 94.6%. Our model was also evaluated on 884 educational videos from Youtube to show its applicability on other educational repositories

Open Educational Content for Digital Public Libraries

If the production of digital content for teaching -- particularly free content -- is to expand substantially, there must be mechanisms to establish a link to fame and fortune that was not perceived in a pre-digital world. How that might be done is the central question this report addresses, in the context of examining the movement for open educational content. Understanding that movement requires delving into the history of what may seem, on first pass, a totally unrelated field of endeavor. The reader's patience is requested....

OER Recommendations to Support Career Development

This Work in Progress Research paper departs from the recent, turbulent changes in global societies, forcing many citizens to re-skill themselves to (re)gain employment. Learners therefore need to be equipped with skills to be autonomous and strategic about their own skill development. Subsequently, high-quality, on-line, personalized educational content and services are also essential to serve this high demand for learning content. Open Educational Resources (OERs) have high potential to contribute to the mitigation of these problems, as they are available in a wide range of learning and occupational contexts globally. However, their applicability has been limited, due to low metadata quality and complex quality control. These issues resulted in a lack of personalised OER functions, like recommendation and search. Therefore, we suggest a novel, personalised OER recommendation method to match skill development targets with open learning content. This is done by: 1) using an OER quality prediction model based on metadata, OER properties, and content; 2) supporting learners to set individual skill targets based on actual labour market information, and 3) building a personalized OER recommender to help learners to master their skill targets. Accordingly, we built a prototype focusing on Data Science related jobs, and evaluated this prototype with 23 data scientists in different expertise levels. Pilot participants used our prototype for at least 30 minutes and commented on each of the recommended OERs. As a result, more than 400 recommendations were generated and 80.9% of the recommendations were reported as useful.Comment: This paper has been accepted to be published in the proceedings of IEEE Frontiers In Education (FIE) 2020 by IEEE Xplor

Hybrid human-AI driven open personalized education

Attaining those skills that match labor market demand is getting increasingly complicated as prerequisite knowledge, skills, and abilities are evolving dynamically through an uncontrollable and seemingly unpredictable process. Furthermore, people's interests in gaining knowledge pertaining to their personal life (e.g., hobbies and life-hacks) are also increasing dramatically in recent decades. In this situation, anticipating and addressing the learning needs are fundamental challenges to twenty-first century education. The need for such technologies has escalated due to the COVID-19 pandemic, where online education became a key player in all types of training programs. The burgeoning availability of data, not only on the demand side but also on the supply side (in the form of open/free educational resources) coupled with smart technologies, may provide a fertile ground for addressing this challenge. Therefore, this thesis aims to contribute to the literature about the utilization of (open and free-online) educational resources toward goal-driven personalized informal learning, by developing a novel Human-AI based system, called eDoer. In this thesis, we discuss all the new knowledge that was created in order to complete the system development, which includes 1) prototype development and qualitative user validation, 2) decomposing the preliminary requirements into meaningful components, 3) implementation and validation of each component, and 4) a final requirement analysis followed by combining the implemented components in order develop and validate the planned system (eDoer). All in all, our proposed system 1) derives the skill requirements for a wide range of occupations (as skills and jobs are typical goals in informal learning) through an analysis of online job vacancy announcements, 2) decomposes skills into learning topics, 3) collects a variety of open/free online educational resources that address those topics, 4) checks the quality of those resources and topic relevance using our developed intelligent prediction models, 5) helps learners to set their learning goals, 6) recommends personalized learning pathways and learning content based on individual learning goals, and 7) provides assessment services for learners to monitor their progress towards their desired learning objectives. Accordingly, we created a learning dashboard focusing on three Data Science related jobs and conducted an initial validation of eDoer through a randomized experiment. Controlling for the effects of prior knowledge as assessed by the pretest, the randomized experiment provided tentative support for the hypothesis that learners who engaged with personal eDoer recommendations attain higher scores on the posttest than those who did not. The hypothesis that learners who received personalized content in terms of format, length, level of detail, and content type, would achieve higher scores than those receiving non-personalized content was not supported as a statistically significant result

Overview and Analysis of Practices with Open Educational Resources in Adult Education in Europe

OER4Adults aimed to provide an overview of Open Educational Practices in adult learning in Europe, identifying enablers and barriers to successful implementation of practices with OER. The project was conducted in 2012-2013 by a team from the Caledonian Academy, Glasgow Caledonian University, funded by The Institute for Prospective Technological Studies (IPTS). The project drew on data from four main sources: • OER4Adults inventory of over 150 OER initiatives relevant to adult learning in Europe • Responses from the leaders of 36 OER initiatives to a detailed SWOT survey • Responses from 89 lifelong learners and adult educators to a short poll • The Vision Papers on Open Education 2030: Lifelong Learning published by IPTS Interpretation was informed by interviews with OER and adult education experts, discussion at the IPTS Foresight Workshop on Open Education and Lifelong Learning 2030, and evaluation of the UKOER programme. Analysis revealed 6 tensions that drive developing practices around OER in adult learning as well 6 summary recommendations for the further development of such practices