Aleven, Vincent

Baker, Ryan

Crosslin, Matthew

Ferschke, Oliver

Gasevic, Dragan

Howley, Iris

Rose, Carolyn Penstein

Siemens, George

Tomar, Gaurav

Yang, Diyi

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Challenges and Opportunities of Dual-Layer MOOCs: Reflectionsfrom an edX Deployment StudyCitation for published version:Rose, CP, Ferschke, O, Tomar, G, Yang, D, Howley, I, Aleven, V, Siemens, G, Crosslin, M, Gasevic, D &Baker, R 2015, Challenges and Opportunities of Dual-Layer MOOCs: Reflections from an edX DeploymentStudy. in Proceedings of the 11th International Conference on Computer Supported Collaborative Learning(CSCL 2015). vol. 2, International Society of the Learning Sciences, Gothenburg, Sweden, pp. 848-851.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionPublished In:Proceedings of the 11th International Conference on Computer Supported Collaborative Learning (CSCL 2015)Publisher Rights Statement:The papers are published through open access on the publisher's website and can be deposited and publishedopenly in the institutional repositories:www.isls.org/cscl2015/papers/1309-InteractiveEvent-Rose.pdfGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019 Challenges and Opportunities of Dual-Layer MOOCs: Reflections from an edX Deployment Study  Carolyn P. Rosé, Oliver Ferschke, Gaurav Tomar, Diyi Yang, Iris Howley, & Vincent Aleven,  Carnegie Mellon University, {cprose,ferschke,gtomar,ihowley,aleven}@cs.cmu.edu  George Siemens and Matthew Crosslin, University of Texas at Arlington, gsiemens@gmail.com Dragan Gasevic, University of Edinburgh, dragan.gasevic@ed.ac.uk Ryan Baker, Teacher’s College at Columbia, ryanshaunbaker@gmail.com   Abstract: This interactive event is meant to engage the CSCL community in brainstorming about what affordances in MOOCs would enable application of and research extending theories and best practices from our field.  To provide a concrete focus as a foundation for this discussion, we present the innovative design of a recent edX MOOC entitled Data, Analytics, and Learning (DALMOOC).  We have integrated several innovative forms of support for discussion based learning, social learning, and self-regulated learning.  In particular, we have integrated a layer referred to as ProSolo, which supports social learning and self-directed learning.  In further support of self-directed learning, intelligent tutor style exercises have been integrated, which offer immediate feedback and hints to students.  We have integrated a social recommendation approach to support effective help seeking in the threaded discussion forums as well as collaborative reflections in the form of synchronous chat exercises facilitated by software agents.  The event will include an overview, offering the opportunity for active engagement in the MOOC, structured brainstorming, and interactive, whole group feedback.  Keywords: massive open online course, discussion-based learning, self-regulated learning, social learning  Overview  This interactive event is meant to engage the CSCL community in brainstorming about what affordances in MOOCs would enable application of and research extending theories and best practices from the field of Computer Supported Collaborative Learning in particular, and the Learning Sciences more generally. It is one facet of a broader effort we are spear-heading to design and build out MOOC platform affordances for collaborative learning.   Our intention is to engage as many alternative perspectives as possible in order to work towards a platform that will enable both accelerated scientific advances in our field in this new context as well as efficient transitions from research into practice. We termed this MOOC a “dual layer” MOOC because students had the option of following a more standard path housed in the edX platform for moving through the course in one layer or to follow a more self-directed path in the ProSolo environment, that formed a second layer.  This MOOC, entitled Data, Analytics, and Learning (DALMOOC), was launched in October of 2014 and extended for 9 weeks in addition to an orientation week as part of in an iterative, design based research process the author team is engaged in to integrate affordances for richer, theory motivated learning experiences into xMOOCs. In each cycle, we base the design of our interventions on analyses of data collected from existing MOOCs, then we deploy those interventions, and then iterate on our designs based on our interpretation of the new data we collected.   This interactive event offers the opportunity to engage the CSCL community in our design based research process beginning with one concrete example MOOC.  However, the goal is to work towards a broader, longer term vision for work in this area. Prosolo  In the context of the current DALMOOC deployment, Siemens and Gasevic were responsible for undertaking work in two primary areas: a) increase the social learning experiences of self-regulated learners in MOOCs through social competency approaches to learning and b) develop learner knowledge graphs that reflect what a  learner knows and how they have come to know it.  This work built on an existing software product (ProSolo) that was integrated in DALMOOC and used by several thousand students in that context as well as other earlier MOOCs and pilots in corporate environments (Volkswagen).  This ProSolo layer served as a striking alternative to the more scripted xMOOC approach to MOOC learning, which remained available to students through the edX layer.  Learning in ProSolo is social in the sense that students follow and communicate with other students during the process of setting their learning objectives.  However, beyond that, the learning experiences students engage in within the ProSolo environment are still largely individual in character. ProSolo utilizes what is termed a competency-based approach to learning. The content of a traditional course/MOOC is translated into competencies and required learning activities are then attached to each competency. Learners can either take the existing set of competencies for a particular course or they can create personal learning goals. The cycle of planning learning, sharing learning and validating is social - learners share completed competencies, including the pathways and approaches used to achieve those competencies. Additionally ProSolo offers a "credential pipeline" that assists faculty in assessing successful completion of competencies. When a learner has completed a particular competency, she submits the competency with evidence of completion to peers or faculty. Upon review of evidence, faculty/peers can provide confirmation of successful completion or send the competency back to the learner for additional evidence of learning. This process also allows for the inclusion of learning that happens outside of the university. Learning that happens through formal courses, MOOCs, personal hobbies, and work can be included in ProSolo and within the credential pipeline. From a motivational standpoint, ProSolo is meant to give learners the opportunity to identify learning goals, connect with others around shared goals, and create a pathway for recognition of learning. A second aspect of ProSolo is the creation of learner profiles so students can find others with shared interests.  DALMOOC has been designed to model a distributed information structure. As such, learners are encouraged to participate in any space they would like: blogs, Facebook, Twitter, edX discussion forums, etc.  A key challenge that arises as learners engage in different spaces is one of fragmentation. Learning is a coherence forming process and knowledge is a state of connecting information pieces. As such, we have adopted an aggregation approach similar to what Stephen Downes pioneered with early MOOCs: gRSShopper. Content is aggregated and shared in a daily email to learners. By aggregating learner content and providing persistent profiles, we anticipate higher levels of learner engagement.  An important affordance provided by ProSolo is persistence.  The content of the course will remain available for students to access post-course, particularly the summary emails and learner profiles in ProSolo. Learners will have the option to search context relevant resources in ProSolo. The hope is that this will assist in creating a persistent practitioner community where learners will access resources post-course and continue to engage with each other on social media and in ProSolo.  Early results from DALMOOC indicate that learners experience some uncertainty and discomfort in transitioning from instructor-driven to self-regulated learning. Learners within the edX course found the distributed structure of DALMOOC confusing at times. Better scaffolding and support is needed to assist learners in transitioning between structured instruction and open engagement in social systems (either the open web or within ProSolo). The Cognitive Tutor Authoring Tools  The importance of immediate feedback and hints on demand during problem solving has been well demonstrated through decades of research on intelligent tutoring systems.  As MOOCs have evolved, an important direction for enhancing their ability to keep students engaged and help students check their understanding is to incorporate comprehension check questions, normally implemented as multiple choice activities.  In a recent Pyschology MOOC, enhanced learning and decreased attrition were both associated with students who chose to participate in intelligent tutor style structured activities with hints and feedback (Koedinger et al., 2015).  In DALMOOC, Baker from Teacher’s College in collaboration with Aleven at CMU integrated intelligent tutoring style activities in Baker’s unit on predictive modeling.  These scaffolded problem solving exercises were implemented using the Cognitive Tutor Authoring Tools (CTAT) (Koedinger et al., 1997; Koedinger & Aleven, 2007), which has been developed with the goal of efficiently authoring intelligent tutoring style learning experiences for students, to be used broadly in online learning.  While these activities were not collaborative, earlier work with student learning in CTAT tutors has been conducted either individually or collaboratively, and thus opportunities for incorporation of additional collaborative learning opportunities around these activities is a potential direction for future work.  The Quick Helper  Earlier research on MOOC discussion forums indicates that the experience of confusion as well as exposure to other students’ confusion are both associated with elevated attrition in MOOCs (Yang et al., 2015), and attempts at resolving confusion by making help requests in the threaded discussions are frequently left without a satisfactory response. In response to these two problems, a specifically discussion-focused intervention, called the Quick Helper, was integrated by Rosé, Ferschke, and the broader CMU team with DALMOOC to support help seeking as well as increase the probability that help requests will be met with a satisfactory response. While virtually all MOOCs offer threaded discussion affordances where students can post help requests, some students are reticent to ask for help, and even when students do post help requests, many of these requests go unanswered.  Our help seeking intervention connects students, whose questions may go unresolved, with student peers who may be able to answer their question. The Quick Helper is continuously available to students by means of a button.  When they click, they are guided to formulate a help request.  The help request is posted to the MOOC discussion board, and the text and metadata are forwarded to our Quick Helper system. Using this help request, a social recommendation algorithm selects three potential help providers from the pool of student peers.  The student is then given the option to invite one or more of these potential helpers to their thread as shown in Figure 1. Once selected, an email with a link to the help request thread is then automatically sent to the selected helpers inviting them to participate in the thread. At writing, 77 unique students elected to use our Quick Helper system approximately 127 times. Further discussion of our initial interventions applied to Quick Helper and its results are discussed in a separate paper (Ferschke et al., in press).               Figure 1. A screenshot of the Quick Helper helpers selection window (left) and the Bazaar Collaborative Reflection synchronous chat (right). Bazaar Collaborative Reflection  A final intervention, referred to as Bazaar Collaborative Reflection, makes synchronous collaboration opportunities available to students in a MOOC context. Research in Computer-Supported Collaborative Learning has demonstrated that conversational computer agents can serve as effective automated facilitators of synchronous collaborative learning (Dyke et al., 2013).  However, typical MOOC providers do not offer students opportunities for synchronous collaboration, and therefore have not so far benefitted from this technology.  Rosé’s team integrated such activities into DALMOOC by means of a Lobby program.  Students click on the Lobby button and are matched with one other student that is also logged in to it.              Upon entering the lobby, students are asked to enter the name that will be displayed in the chat. When successfully matched with another learner, the student and their partner are then presented with a link to a chat room created for them.  If another student does not enter the Lobby within a couple minutes, they are requested to return later. A visualization is presented to the student that illustrates the frequency of student clicks on the button at different times of the day on the various days of the week so that they are able to determine the best time to return. Students enter the synchronous chat room via the link, and interact with each other as well as a  conversational agent who appears as a regular user in the chat, as shown in Figure 1. This chat setup has been used in earlier classroom research (Dyke et al., 2013; Adamson et al., 2014).   In our initial investigation in the edX MOOC, we make use of statically scripted agents who guide the students through course-related discussion questions but future investigations may include agents that dynamically react to the students. Students interact and discuss a topic until the pair selects the “We’re ready” button, then the agent then proceeds with the next prompt.  While we have not yet assessed the influence of Bazaar on learning, we performed a survival analysis to evaluate the influence of this collaborative learning activity on attrition. Using the number of clicks on videos and the participation in discussion forums as control variables, we found that the participation in chats lowers the risk of dropout almost by 50%.  Reflections  As we observed learners experiencing the automated and social support integrated with this innovative edX MOOC, we observed students providing benefit to one another when they connected, both in the ProSolo Layer and the more standard edX layer.  As we observe a pattern of students gravitating towards the variety of communication media made available through ProSolo and in DAMOOC more broadly, we have begun to investigate the choices students make for where they conduct their social interaction, and what trade-offs we find in terms of the content focus and nature of discussions in the different environments.   As we read their communication, we see evidence of confusion and frustration in the process of wayfinding through the choices.  Thus, we find many opportunities to better support students in finding and maintaining desirable connections, support, and direction throughout the course.  The use of social recommender systems and group collaboration tools (such as Bazaar) are expected to lead to higher levels of metacognitive monitoring, which in turn is associated with an increase of feeling of knowing, increased confidence, judgment of learning, and monitoring of progress toward goals.  We continue to seek ways to integrate the technologies we have developed to achieve a more positive experience for students. References  Adamson, D., Dyke, G., Jang, H. J., Rosé, C. P. (2014). Towards an Agile Approach to Adapting Dynamic Collaboration Support to Student Needs, International Journal of Artificial Intelligence in Education, 24(1), pp. 91-121.  Dyke, G., Howley, I., Adamson, D., Kumar, R., & Rosé, C. P. (2013). Towards academically productive talk supported by conversational agents. In Productive multivocality in the analysis of group interactions, pp. 459-476. Springer US.  Ferschke, O., Howley, I., Tomar, G., Yang, D., Rosé, C. P. (in press).  Fostering Discussion across Communication Media in Massive Open Online Courses, Proceedings of Computer Supported Collaborative Learning 2015 Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, pp. 30-43. Koedinger, K. R. & Aleven, V. (2007).  Exploring the assistance dilemma in experiments with Cognitive Tutors.  Educational Psychology Review, 19 (3), pp. 239-264. Koedinger, K., Kim, J., Zhuxin, J., McLaughlin, E., and Bier, N. (2015). Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC, Proceedings of the ACM Conference on Learning @ Scale. Yang, D., Wen, M., Howley, I., Kraut, R., and Rose, C. P. (2015).  Exploring the Effect of Confusion in Discussion Forums of Massive Open Online Courses, Proceedings of the ACM Conference on Learning @ Scale. 

Challenges and Opportunities of Dual-Layer MOOCs: Reflections from an edX Deployment Study

https://www.pure.ed.ac.uk/ws/files/19117279/CSCL_DALMOOC_InteractiveEvent_Description_1_.pdf

Challenges and Opportunities of Dual-Layer MOOCs: Reflections from an edX Deployment Study

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer