Quality Matters (QM) is a faculty-centered, peer review process designed to certify the quality of online courses. Quality Matters reviews are processes that can be taxing and become unsustainable for several institutions. What if QM standards could be more easily enforced during the course design and development phases? This could potentially reduce missteps that require re-design. In this paper, we propose a methodology that does exactly that. QM4Design structures the various steps of instructional design into a format where QM are explicitly re-enforced during the process

Porto, Stella C.S.

Suarez, Carolina Vega

Repositorio Institucional de la Universidad de La Laguna

Online Course Design: Quality DOES Matter Stella C.S. Porto Knowledge & Learning Sector Inter-American Development Bank Washington, D.C., United States stellap@iadb.org   Carolina Vega Suarez Knowledge & Learning Sector Inter-American Development Bank Washington, D.C., United States csuarez@iadb.org  Abstract— Quality Matters (QM) is a faculty-centered, peer review process designed to certify the quality of online courses. Quality Matters reviews are processes that can be taxing and become unsustainable for several institutions. What if QM standards could be more easily enforced during the course design and development phases? This could potentially reduce missteps that require re-design. In this paper, we propose a methodology that does exactly that. QM4Design structures the various steps of instructional design into a format where QM are explicitly re-enforced during the process. Keywords—online course design; quality; quality control; standards I. INTRODUCTION Established in 1959, the Inter-American Development Bank (IDB) has become a leading source of financing of social and economic development in the LAC region. The generation and dissemination of knowledge is a fundamental part of IDB's development mission. The best-known goal of a development bank is to fund projects in target countries that will support social and economic development. Less known is its focus on providing technical assistance and offering knowledge products on strategic areas such as development of institutions and the financial sector, economic development, social development, infrastructure and climate change, and integration and commerce. (IDB, 2018). Today IDB offers a variety of online courses through instructor led virtual classes as well as MOOCS. This paper presents instructional design methodology where QM Standards are explicitly reinforced.  In the next section we summarize the QM Rubric with particular focus on alignment aspects. Next, we describe the QM for design methodology and its benefits for quality control. II.   WHAT IS QM? Quality control is a critical aspect in the design, production, and delivery phases of any e-learning initiative. The literature has shown repeatedly that adopting guidelines regarding minimum standards for course design and development is crucial to achieving desired learning outcomes. (Hannover Research, 2009). QM is a faculty-centered, peer review process, designed to certify the quality of online courses.   (Quality Matters, 2018) QM can be summarized as the integration of three main components: the QM Rubric, QM Professional Development, and QM Review Process. The QM Rubric is a set of standards created based on literature reviews of online learning research. These standards also incorporate best practices identified by those in the field: course developers and instructors. These are often reviewed and updated.  There are eight General Standards in the QM Rubric, each one with a certain number of Specific Standards, as shown below:  1. Course Overview and Introduction:  These standards refer to the way the course introduction provides learners a clear guidance on what to expect and how to begin the course. (9 specific standards).  2. Learning Objectives (Competencies): These standards cover how should the learning objectives be presented from the learner’s point of view. (5 specific standards). 3. Assessment and Measurement: These standards describe the way assessment aligns to the learning objectives, stablishing a coherent relationship between them. (5 specific standards).  4. Instructional Materials: These standards focus on identifying the direct relationship between course resources and course learning objectives. (6 specific standards). 5. Course Activities and Learner Interaction: These standards aim to guarantee that activities promote learner engagement and hence active learning. (4 specific standards) 6. Course Technology: These standards refer to how the technologies used in the course facilitate the learning process. (5 specific standards). 7. Learner Support: These standards identify the services that support learner throughout their learning process, including aspects such as technical, accessibility and academics. (4 specific standards). 8. Accessibility and Usability. These standards focus on the accessibility and usability in the online. (5 specific standards).   Specific Standards are organized by levels of importance and corresponding points, namely: essential (3 points), very important (2 points) and important (1 point). The total score for the rubric is a max of 99 points as shown in the table below:     TABLE I. - Distribution of points to Specific Standards. QM Rubric Specific Standards Level of Importance Number of standards Points per Standard Total of Points Essential  21 3 63 Very Important  14 2 28 Important 8 1 8     TOTAL 43  99  When a standard is met, the corresponding points are added to the course QM score. To be considered QM certified, the course should achieve a final score of a minimum of 85% and all the essential standards need to be met. The essential standards are associated to all the alignment aspects of the rubric. Given that alignment is at the heart of the QM rubric, we depict below the Specific Standards that cover alignment within the rubric (QM Rubric, 2014): 2.1 “The course learning objectives, or course/program competencies, describe outcomes that are measurable”.  2.2 “The module/unit learning objectives or competencies describe outcomes that are measurable and consistent with the course-level objectives or competencies”.  3.1 “The assessments measure the stated learning objectives or competencies”.  4.1 “The instructional materials contribute to the achievement of the stated course and module/unit learning objectives or competencies”.  5.1 “The learning activities promote the achievement of the stated learning objectives or competencies”.  6.1 “The tools used in the course support the learning objectives or competencies”.    Fig.1 QM Rubric Alignment The QM rubric has taken an important role in providing a benchmark for quality control of online courses. (G. Hollowell, R. Brooks, and Y. Anderson, 2017). In IDB, training staff and conducting QM external reviews were the first steps taken to engrain the QM standards. Internal review processes have been established, where several INDES members coordinate activities of recommendations and implementing needed changes. This process, although more sustainable than the external official review, depend on the limited availability of INDES staff and can therefore create a long waiting period.  In many cases, after a review, re-design and development are required to fix mistakes, which could have been prevented if QM standards were considered forcefully during the design and development phases. Some of these corrections can be costly both in terms of new investments as well as delays in the course delivery. Moreover, conducting external review of all courses is not economically sustainable. Thus, the question of how to create a culture of adopting QM standards during the design phase becomes critical. Given the on-demand nature of INDES’ role in course production, more frequently than not, INDES is faced with managing projects with several stakeholders with no knowledge of QM, much less QM training. What if QM standards could be more easily enforced during the design and development phases, including to those with no prior knowledge of the QM rubric? This could potentially reduce missteps that require re-design later. This has pushed INDES QM coordinator and other staff members to devise a tool that sets the steps of instructional design into a format where QM standards are explicitly re-enforced during process. The tool, which is implemented as a comprehensive Workbook, ensures that at the end of the process, one have all the information needed to implement the course in any LMS, and that all QM standards are considered. (Porto, 2017) During this session, participants went through the motions of identifying how the QM rubric can be used for the design and development phase of a new course. Participants considered a variety of proposals and the challenges to use the rubric in this fashion, and further assess how this could be adopted within their own institution.  III. WHAT IS QM4DESIGN? Using both internal and external QM review processes, we revised and improved all our online tutor-led courses and a few of the existing MOOCS. This experience provided us with an inside look into the time and cost demands for completing such reviews.  It also made clear that many of the standards should be considered during the design phase. Our main concern was to find a way to improve the design process, so that at the end of a QM review, the course would meet QM expectations, because standards would be at the forefront during the design practice. We were searching for a methodology that would inform all those involved in the design about the critical quality elements depicted in the QM Rubric. And so, QM4Design was born.  During a usual QM Review, a reviewer inspects an already designed and developed course (QM Peer Review Course, 2018) For each QM standard, s/he looks for evidence that demonstrates that the course complies with each given standard. In QM4Design, we take the QM review process and adapt it, so it serves the purpose of designing a new course that adheres to the same standards. QM4Design takes a reversed engineering approach to the QM Review process. In the QM-guided course design process, the course designer designs the course, while concurrently ensuring that if at the end a reviewer were to inspect this course, s/he would immediately and easily find the evidence that guarantees that each QM standard is met. This methodology is an instrument to help course designers follow a QM guided design process in such a way, that at the end, the course will be fully QM compliant. The methodology establishes order and requirements concerning information for each step of the instructional design process. The course designer is guided to fill out information, and then check if s/he has provided enough evidence that assures that the course meets QM standards.  QM4Design methodology has been used by course designers (online tutors and instructional designers) who have designed or re-designed tutor-led online courses. Their experience using this methodology has been positive, showing how it has helped them structure their course design process, making it smoother, assuring quality and avoiding pitfalls right from the start.  We expect to continue using QM4Design tool to easily introduce future instructional designers and online tutors from IDB into the best practices of course design, with a focus on alignment and student centered pedagogical practices as critical elements towards high quality online courses.  REFERENCES [1] G. Hollowell, R. Brooks, and Y. Anderson, “Course design, Quality Matters training, and student outcomes,” in American Journal of Distance Education,  vol XXXI , 2017, pp. 207-216. Retrieved from http://www.tandfonline.com/doi/full/10.1080/08923647.2017.1301144?scroll=top&needAccess=true [2] Hanover Research, “Best practices in online teaching strategies”. Retrieved from https://www.uwec.edu/AcadAff/resources/edtech/upload/Best-Practices-in-Online-Teaching-Strategies-Membership.pdf 2009. [3] Interamerican Development Bank, “What we do”, Retrieved from https://www.iadb.org/en/about-us/departments/about%2C1342.html%3Fdept_id%3DKNL, 2018. [4] Quality Matters, “Rubric and standards”, retrieved from https://www.qualitymatters.org/qa-resources/rubric-standards , 2018. [5] Quality Matters, “Peer review course”, retrieved from https://www.qualitymatters.org/professional-development/courses/higher-ed-prc , 2018 [6] Quality Matters, Standars from the QM Higher Education Rubric, figth edition, 2014. [7] S. Porto, “QM Internal Review Workshop,” (2017),unpublished.  

Online Course Design : Quality DOES Matter

https://riull.ull.es/xmlui/bitstream/915/8026/1/Online%20Course%20Design%2c%20Quality%20DOES%20Matter%20JICV18_paper_9.pdf

Online Course Design : Quality DOES Matter

Abstract

Similar works

Full text

Available Versions

Repositorio Institucional de la Universidad de La Laguna