Dashboard for Evaluating the Quality of Open Learning Courses

Universities are developing a large number of Open Learning projects that must be subject to quality evaluation. However, these projects have some special characteristics that make the usual quality models not respond to all their requirements. A fundamental part in a quality model is a visual representation of the results (a dashboard) that can facilitate decision making. In this paper, we propose a complete model for evaluating the quality of Open Learning courses and the design of a dashboard to represent its results. The quality model is hierarchical, with four levels of abstraction: components, elements, attributes and indicators. An interesting contribution is the definition of the standards in the form of fulfillment levels, that are easier to interpret and allow using a color code to build a heat map that serves as a dashboard. It is a regular nonagon, divided into sectors and concentric rings, in which each color intensity represents the fulfillment level reached by each abstraction level. The resulting diagram is a compact and visually powerful representation, which allows the identification of the strengths and weaknesses of the Open Learning course. A case study of an Ecuadorian university is also presented to complete the description and draw new conclusions.This research was funded by Universidad Central del Ecuador, through the agreement with Universidad de Alicante for the direction of doctoral theses.” “The APC was funded by Cátedra Santander-UA de Transformación Digital and Smart Learning Research Group, Universidad de Alicante”

Performance Management in Action

Outlines how collecting and sharing data enables educators to assess student performance in real time and predict and address issues early in order to improve achievement levels and graduation rates. Profiles successful programs in three urban districts

Project based learning on industrial informatics: applying IoT to urban garden

Copyright (c) 2018 IEEEThe fast evolution of technologies forces teachers to trade content off for self-learning. PBL is one of the best ways to promote self-learning and simultaneously boost motivation. In this paper, we present our experience introducing project-based learning in the last year subject. New Internet of Things (IoT) topic allows us to carry out complete projects, integrating different technologies and tools. Moreover, the selection of open-source and standard free technologies makes easy and cheap the access to hardware and software platforms used. We carefully have picked communication, data management, and programming tools that we think would be attractive to our students. They can start making fast prototyping with little initial skills and, at the same time, these are serious and popular tools widely used in the industry. In this paper, we report on the design of a project-based learning for our course and the impact this has on the student satisfaction and motivation. Surveys taught us that tuning the courses towards developing real projects on the field, has a large impact on acceptance, learning objectives achievements and motivation towards the course content.”I Plan Propio Integral de Docencia de la Universidad de Málaga” y Proyecto de Innovación Educativa PIE17/085, de la Universidad de Málaga. Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Student-Centered Learning: Functional Requirements for Integrated Systems to Optimize Learning

The realities of the 21st-century learner require that schools and educators fundamentally change their practice. "Educators must produce college- and career-ready graduates that reflect the future these students will face. And, they must facilitate learning through means that align with the defining attributes of this generation of learners."Today, we know more than ever about how students learn, acknowledging that the process isn't the same for every student and doesn't remain the same for each individual, depending upon maturation and the content being learned. We know that students want to progress at a pace that allows them to master new concepts and skills, to access a variety of resources, to receive timely feedback on their progress, to demonstrate their knowledge in multiple ways and to get direction, support and feedback from—as well as collaborate with—experts, teachers, tutors and other students.The result is a growing demand for student-centered, transformative digital learning using competency education as an underpinning.iNACOL released this paper to illustrate the technical requirements and functionalities that learning management systems need to shift toward student-centered instructional models. This comprehensive framework will help districts and schools determine what systems to use and integrate as they being their journey toward student-centered learning, as well as how systems integration aligns with their organizational vision, educational goals and strategic plans.Educators can use this report to optimize student learning and promote innovation in their own student-centered learning environments. The report will help school leaders understand the complex technologies needed to optimize personalized learning and how to use data and analytics to improve practices, and can assist technology leaders in re-engineering systems to support the key nuances of student-centered learning

Interactive learning online: Challenges and opportunities

Since the early 1990s online education and online learning systems have held the promise of increasing instructional productivity and reducing costs without sacrificing educational quality. There is no evidence to date that such promise has materialized. The impetus of the newest developments with free online courses to hundreds of thousands of students might drastically transform how we teach more and better with less. The innovation that prompted this panel is called Interactive Learning Online (ILO), and has the distinctive feature of highly interactive, machine-guided instruction that can be scaled to accommodate a large number of students who benefit from targeted and personalized learning. The panelists have experimented with online learning in different ways. Their perspectives will address challenges and opportunities with the adoption of ILO systems

Squaring the circle: a new alternative to alternative-assessment

Many quality assurance systems rely on high-stakes assessment for course certification. Such methods are not as objective as they might appear; they can have detrimental effects on student motivation and may lack relevance to the needs of degree courses increasingly oriented to vocational utility. Alternative assessment methods can show greater formative and motivational value for students but are not well suited to the demands of course certification. The widespread use of virtual learning environments and electronic portfolios generates substantial learner activity data to enable new ways of monitoring and assessing students through Learning Analytics. These emerging practices have the potential to square the circle by generating objective, summative reports for course certification while at the same time providing formative assessment to personalise the student experience. This paper introduces conceptual models of assessment to explore how traditional reliance on numbers and grades might be displaced by new forms of evidence-intensive student profiling and engagement

학습자 동기 부여 지원을 위한 MOOCs 인터페이스 개발

학위논문(석사) -- 서울대학교대학원 : 사범대학 교육학과(교육공학전공), 2022.2. Young Hoan Cho.With the development of information and communication technology (ICT), many people get educated not only in traditional classrooms but also online nowadays. As one way of online education, the market of MOOCs (Massive Open Online Courses) has been growing continuously since the first platform was opened to the public in 2012. Today in 2021, the number of MOOCs learners has reached up to 220 million, and MOOCs are playing an irreplaceable role in higher education, lifelong education, corporate education, etc. Although we can expect that MOOCs will become increasingly important in the field of education, with its rapid growth during the last decade, some issues of it have been exposed. One of the issues is the low completion rate. Compared to traditional education, MOOCs learners are reported more likely to drop out, which leads to the average completion rate at around 10%. According to previous studies, one of the reasons that causes this phenomenon is lacking motivation. As a part that interacts directly with users of an application, interface is crucial because it offers affordance and determines the way users use the application. And the interface becomes even more important when it comes to E-learning because motivators, which can affect learners' motivation, can be designed in the user interface. However, studies have shown that the current interface design of MOOCs lacks motivation factors and fails to facilitate interactive communication among MOOCs learners. Therefore, in this research, a MOOCs interface that focuses on improving learners' motivation was designed. To achieve this goal, the research questions considered were: 1) What are the interface design guidelines and interface functions to motivate MOOCs learners to sustain their learning? 2) What is the interface to motivate MOOCs learners to sustain their learning? and 3) What are the learners' responses to the interface? To answer the research questions, the type 1 design and development methodology proposed by Richey and Klein was followed. First, MOOCs interface design guidelines were derived by literature review and followed by 2 rounds of expert review conducted by 4 experts to ensure the internal validity. Second, a prototype of MOOCs interface was designed based on the guidelines by using prototyping tool Figma. Third, the prototype was given to 5 learners along with a series of tasks for learner response tests to ensure the external validity of the design guidelines, and based on the result, both the prototype and the guidelines were revised. The final version of the MOOCs interface design guidelines consists of 3 motivational design principles （Autonomy, Competence, and Relatedness）, 12 motivational design guidelines (5 autonomy-supported, 4 competence-supported, and 3 relatedness-supported) along with 34 design guidelines developed for MOOCs interface. Based on these design guidelines, the functions of the MOOCs interface in this research were designed. Based on the autonomy-supported guidelines, functions such as learning mode selection (self-paced, scheduled, premiere), learning group, learning activity, goal setting, dashboard, reminder, recommendation, and feedback were designed. And based on the competence-supported guidelines, functions such as account register, course enrollment, learning path, team activity support, dashboard, and goal setting were designed. Meanwhile, based on the relatedness-supported guidelines, functions such as dashboard, feedback, keyword checklist, learning group, chatting window, group/team activity, course evaluation, team assignment, mind map, and note were designed. The participating learners were satisfied with the design. The survey data showed that learners’ general perceptions of the MOOCs interface reached 4.44, perceived autonomy reached 4.40, perceived competence reached 4.52, and perceived relatedness reached 4.66 (5 points Likert scale). The in-depth interview data was open coded into three categories: 1) Advantages of the MOOCs interface, 2) Problems with the MOOCs interface, and 3) Suggestions for improvement. The advantages include providing choices for autonomy support, providing scaffolding and adaptive learning for competence support, providing interactive learning for relatedness support, and providing novel meanwhile helpful functions that existing platforms don’t have. The problems include lacking tutorials for novel functions, inconsistent icons and choice of words, and improper positioning and interaction. The suggestions for improvement include adding the wiki function, adding the reminder function, and visualizing the timetable. The significance of this research can be summarized as follows: 1) proposed an intrinsic motivation oriented MOOCs interface. 2) introduced three learning modes to the MOOCs learning environment. 3) introduced the learning group and learning team to the MOOCs environment to facilitate learners’ interaction. 4) provided an example of the dashboard for the context of MOOCs. And 5) provided insight into how to help learners achieve personalized learning in the MOOCs environment.정보통신기술(ICT)의 발달로 오늘날 많은 사람이 전통적인 교실에서뿐만 아니라 온라인으로도 교육을 받고 있다. 온라인 교육의 한 방법으로 MOOC(Massive Open Online Courses)의 시장은 2012년 첫 플랫폼이 공개된 이후 지속적으로 성장하고 있으며, 교육학자에게 많은 관심을 받고 있다. 특히 MOOC는 고등교육, 평생교육, 기업교육 등에서 대체할 수 없는 역할을 한다는 연구 결과가 속속히 나오고 있다. 교육 분야에서 MOOC가 점점 더 중요해질 것이라고 예상되지만, 지난 10년 동안 MOOC의 급속한 성장과 함께 MOOC의 문제점들이 일부 발견되었다. 그중 하나는 낮은 이수율로서, 기존 교육에 비해 MOOC 학습자는 평균 이수율이 약 10%에 머무를 정도로 중도 탈락할 가능성이 높은 것으로 보고되었다. 선행 연구에 따르면 이러한 현상을 일으키는 원인 중 하나는 부족한 동기부여 때문이다. 사용자 인터페이스는 애플리케이션에서 사용자와 직접 상호 작용하는 부분으로서 어포던스를 제공하고, 사용자가 애플리케이션을 사용하는 방식을 결정하기 때문에 매우 중요하다고 말할 수 있다. 특히, E-learning 상황에서는 인터페이스가 더욱 중요한데, 이는 동기 부여에 영향을 미칠 수 있는 동기 요소들이 사용자 인터페이스에 설계될 수 있기 때문이다. 그러나 선행 연구에 따르면 현재의 MOOC 인터페이스 디자인은 동기 요소가 부족하고 MOOC 학습자 간의 상호 작용을 촉진하지 못하고 있다. 이에 따라서 본 연구에서는 학습자의 동기부여에 초점을 맞추어 MOOC 인터페이스를 설계하였고, 이를 위해 다음의 연구 문제들을 고려하였다. 1) MOOC 학습자가 학습을 지속하도록 동기를 부여하는 인터페이스 디자인 가이드라인은 무엇인가? 2) 인터페이스 디자인 가이드라인에 따른 예시적인 인터페이스는 무엇인가? 3) 인터페이스에 대한 학습자의 반응은 무엇인가? 연구 질문에 답하기 위해 Richey와 Klein이 제안한 설계ㆍ개발 방법론 중 유형 1을 따랐다. 먼저 문헌검토를 통해 MOOC 인터페이스 디자인 가이드라인을 도출하고, 내적 타당성을 확보하기 위해 4명의 전문가가 2차례에 걸쳐 전문가검토를 실시하였다. 다음으로, 프로토타이핑 도구인 Figma를 사용하여 가이드라인에 따라 MOOC 인터페이스의 프로토타입을 개발했다. 마지막으로, 디자인 가이드라인의 외적 타당성을 확보하기 위해 2차례에 걸쳐 사용성 평가를 하고 학습자의 반응을 측정했다. 일련의 과제와 함께 프로토타입을 5명의 학습자에게 제공하고, 그 결과를 바탕으로 프로토타입과 가이드라인을 모두 수정했다. MOOC 인터페이스 디자인 가이드라인의 최종 버전은 3가지 동기 부여 디자인 원리 (자율성, 유능성 및 관계성), 12가지 동기 부여 디자인 가이드라인 (5개 자율성 지원, 4개 유능성 지원, 3개 관계성 지원)과 34개의 MOOC 인터페이스 디자인 가이드라인으로 구성되었다. 본 연구에서 개발한 MOOC 인터페이스는 이러한 MOOC 인터페이스 디자인 가이드라인에 따른 기능을 포함하고 있다. 자율성 지원 가이드라인을 반영하는 기능으로 학습 모드 선택 (self-paced, scheduled, premiere), 학습 그룹, 학습활동 선택, 학습목표 설정, 대시보드, 리마인더, 추천, 피드백 등 기능이 있었고, 유능성 지원 가이드라인을 반영하는 기능으로 레지스터, 수업 등록, 학습 경로, 팀 활동 지원, 대시보드, 학습목표 설정 등 기능이 있었으며, 관계성 지원 가이드라인을 반영하는 기능으로 대시보드, 피드백, 키워드 체크리스트, 학습 그룹, 채팅창, 그룹/팀 활동, 수업 평가, 팀 과제, 마인드맵, 노트 등 기능이 있었다. 사용성 평가에 참여한 학습자들은 개발한 MOOCs 인터페이스에 만족했으며, 5점 척도 구성한 설문 문항으로 조사한 결과, 인터페이스 전반에 대한 인식 4.44점, 자율성에 대한 인식 4.40점, 유능성에 대한 인식 4.52점, 관계성에 대한 인식 4.66점이었다. 그리고 심층 인터뷰에서 취득한 질적 데이터를 오픈 코딩을 통해서 정리한 결과, 개발한 MOOCs 인터페이스의 장점, 문제점, 개선점을 도출했다. 장점으로 자율성 지원을 위한 선택 제공, 유능성 지원을 위한 스캐폴딩과 적응형 학습, 관계성 지원을 위해서 제공하는 상호작용, 그리고 기존 플랫폼과의 의미있는 차이점 등이 있었다. 문제점으로 기존 플랫폼들이 제공하지 않은 새로운 기능에 대한 가이드가 필요하다는 점, 불일치한 아이콘과 용어, 부적절한 포지셔닝과 인터랙션 등이 있었다. 개선점으로 위키 기능, 알림 메시지 기능을 추가하고 시간표를 시각화 등이 있었다. 본 연구의 의의는 다음과 같이 요약될 수 있다. 1) 내재적 동기부여 지향한 MOOC 인터페이스를 제안하였다. 2) MOOC 학습 환경에 3가지 학습 모드를 도입했다. 3) 학습자의 상호 작용을 촉진하기 위해 학습 그룹과 학습 팀을 MOOC 학습 환경에 도입했다. 4) MOOC 상황에 대시보드의 예시를 제공했다. 그리고 5) 학습자가 MOOC 환경에서 맞춤형 학습을 달성하도록 돕는 방법에 대한 인사이트를 제공했다.Chapter 1. Introduction 1 1.1. Problem Statement 1 1.2. Purpose of the Research 4 1.3. Research Questions 8 1.4. Definition of Terms 9 1.4.1. MOOCs 9 1.4.2. Motivation 11 1.4.3. User Interface 12 Chapter 2. Literature Review 14 2.1. MOOCs 14 2.1.1. Characteristics and Meaning 14 2.1.2. History of MOOCs 16 2.1.3. MOOCs Platforms and MOOCs Learners 20 2.1.4. The Critiques and Drop Out Phenomenon 26 2.2. Motivation 35 2.2.1. Motivation in Learning 35 2.2.2. Motivation Theories 37 2.3.1. User Interface Design & Interface Design for Education 45 2.3.2 Motivation Supported User Interface Design 48 Chapter 3. Research Method 54 3.1. Design and Development Methodology 54 3.2. Research Participants 58 3.3. Research Tools 60 3.3.1. Internal Validation Tools 60 3.3.2. Prototyping Tool 60 3.3.3. External Validation Tools 62 3.4. Data Collection and Analysis 65 3.4.1. Expert Review 65 3.4.2. Learners' Responses 66 Chapter 4. Findings 68 4.1. The MOOCs Interface Design Guidelines 68 4.1.1. The Final Version of the MOOCs Interface Design Guidelines 69 4.1.2. The Results of the Expert Review 80 4.2. Prototype of the MOOCs Interface 85 4.3. Learners' Responses to the MOOCs Interface 118 4.3.1. Learners' Response to the MOOCs Interface (First Usability Test) 119 4.3.2. Learners' Responses to the Revised Interface (Second Usability Test) 143 Chapter 5. Discussion and Conclusion 145 5.1. Discussion 146 5.2. Conclusion 151 Reference 154 APPENDIX 1 175 APPENDIX 2 189 APPENDIX 3 197 APPENDIX 4 206 APPENDIX 5 213 APPENDIX 6 220 국문초록 224석