I investigate the demographics and behavior of students in a series of four Astrophysics Massive Open Online Courses
(MOOCs) offered in 2014-15 by the Australian National University, via the edX Consortium. 135,000 students enrolled in one or
more of the courses. These students came from 175 countries: only 6% came from within Australia and less than half had
heard of the Australian National University before enrolling. MOOCs are thus an effective way to increase awareness of
Australian universities amongst populations where such awareness is extremely low. Only 30% of those who enrolled in the first
course ever logged in, and only 10% passed the first homework. Amongst those who passed the first homework, however, 75%
would go on to pass the course. Students in the age range 20-30 years were particularly likely to enroll in the course but never
log in. The students found the MOOCs to be an effective educational experience: 87% said that they learned as much or more
from these MOOCs as they had from on-campus courses. They particularly praised the ability to pause and re-wind the short
videos used in the course

Francis, Paul

English

The Australian National University

STUDENTS DEMOGRAPHICS AND BEHAVIOUR IN A SERIES OF AUSTRALIAN MASSIVE OPEN ONLINE COURSES (MOOCS)  Paul J. Francisa  Paul.Francis@anu.edu.au aResearch School of Astronomy and Astrophysics, and the Physics Education Centre, The Australian National University, Canberra 2601, Australia  KEYWORDS: MOOCs, Online learning, Student Demographics  ABSTRACT I investigate the demographics and behavior of students in a series of four Astrophysics Massive Open Online Courses (MOOCs) offered in 2014-15 by the Australian National University, via the edX Consortium. 100,000 students enrolled in one or more of the courses. These students came from 175 countries: only 6% came from within Australia and less than half had heard of the Australian National University before enrolling. MOOCs are thus an effective way to increase awareness of Australian universities amongst populations where such awareness is extremely low. Only 30% of those who enrolled in the first course ever logged in, and only 10% passed the first homework. Amongst those who passed the first homework, however, 75% would go on to pass the course. Students in the age range 20-30 years were particularly likely to enroll in the course but never log in. The students found the MOOCs to be an effective educational experience: 87% said that they learned as much or more from these MOOCs as they had from on-campus courses. They particularly praised the ability to pause and re-wind the short videos used in the course.  Proceedings of the Australian Conference on Science and Mathematics Education, Curtin University, Sept 30th to Oct 1st, 2015, page X, ISBN Number 978-0-9871834-4-6.   1. Introduction Massive Open Online Courses (MOOCs) are a new type of online learning, with courses offered for free to tens of thousands of students around the world (e.g. Liyanagunawardena, Adams and Williams 2013, Ebben and Murphy 2014). These courses can reach audiences poorly served by existing higher education (Dillahunt, Wang and Teasley 2014), and can produce learning outcomes at least as good as traditional courses (Colvin et al. 2014). Several enormous and well-funded university consortia such as edX, Coursera and FutureLearn are competing to offer such courses to tens of millions of students from around the world. Increasingly Australian universities are joining these consortia or considering joining.  Perhaps the greatest unknown for a university about to publish its first MOOCs is the nature of the student cohort they will attract. International experience (e.g. Breslow et al. 2013, Despujol et al. 2014, Jordan 2014, Jo et al. 2015) suggests that these courses will attract tens of thousands of already highly educated students from all over the world, but that completion rates will only be around 3%.  In this paper, I analyse the student cohort and behavior in a series of four Astrophysics MOOCs offered starting in March 2014 by the Australian National University (ANU). These were among the first generation of MOOCs offered by any Australian university, and provide a chance to see whether the Australian experience is consistent with the international experience.  The series of four MOOCs together provide the equivalent of our first-year on-campus core astrophysics course (Francis 2006). Each course consisted of around 10 hours of video footage (one per week), broken into multiple 5-10 minute clips interspersed with conceptual multiple-choice questions (short videos like these have been shown to be most effective in the MOOC context: see Guo, Kim and Rubin 2014). Each course also involved webcast worked examples (Francis 2013), weekly on-line homework and a fantasy universe exercise (Francis 2015). The courses required students to have a strong background in high-school maths and physics. Each course ran over a ten week period, but was then re-opened as a self-paced course, which students could complete in their own time.   2. Student Demographics Up until May 2015, a total of 59778 students had enrolled in at least one of the courses (1839 had enrolled in all four). Numbers enrolling in the individual courses are shown in Table 1. These are comparable to or slightly lower than typical for a recent MOOC (Jordan 2014, Ho et al. 2015), perhaps due to the strong maths and physics prior knowledge assumed.  Table 1: Enrolment numbers  Enrolled Started Course Passed (by May 2015) Course 1: Unsolved Mysteries 26770 2960 1640 (55% of those who started) Course 2: Exoplanets 12790 3345 1172 (35% of those who started) Course 3: Violent Universe 25022 3204 860 (27% of those who started) Course 4: Cosmology 10942 3349 1200 (36% of those who started)  Where did these students come from? Were we predominantly recruiting local students, or international ones? We determined the nationality of the students by their self-reported addresses (entered by 50% of students as part of their initial edX sign-up), and these are shown in Table 2. Note that the students are overwhelmingly from overseas, with about 50% coming from developing countries. This student distribution is typical of edX MOOCs (e.g. Breslow et al. 2013, Rayyan et al. 2013, Despujol et al. 2014). Note that China only contributes 1.3% of the students, which once again is typical of other MOOCs. The “great firewall” makes access for Chinese students hard, as the videos are hosted on YouTube. Alternative download links are provided, and the fraction of students from China is increasing in more recent edX MOOCs (Jo et al. 2015).  Table 2: Top 10 Countries of students enrolled in first course during its first run.  Country % of enrolled students % of students who started % of students who passed Homework 1 % of those who passed course USA  22.7 25.8 22.9 23.2 India  17.1 13.6 10.6 8.0 UK  8.1 8.4 9.6 12.0 Australia  6.2 9.4 11.2 9.6 Spain  3.6 3.0 3.7 3.2 Canada  2.7 2.6 4.3 3.2 Brazil  2.4 2.2 1.6 1.6 Russia  2.1 1.8 1.6 1.6 Pakistan  2.1 1.8 1.6 1.6 Mexico  1.9 2.6 3.2 2.4 Other 33 31 33 36  The MOOCs are thus predominantly reaching students in countries which are not traditional sources of students for Australian universities. Indeed, a large proportion of the students came from a long tail of 175 countries, most of which only contributed a small number of students. Some of the students commented that they felt as if they were the only person in their country interested in astrophysics.   Were these people already familiar with Australian universities? At the end of the 4th course, we surveyed the students about their previous knowledge of the Australian National University. This was an anonymous voluntary survey link to from the feedback section of the course: it was completed by 22% of the students who passed the course: results are in Table 3.    Table 3: Student responses to the question “How much did you know about the Australian National University before you took this course?” Response Number of students 5. I know it very well 14 4. 9 3. 27 2. 56 1. Nothing – I’d never heard of it 148   71% of the students were male, which once again is typical of MOOCs in the physical sciences (Breslow et al. 2013, Ho et al. 2015). The student body was split almost evenly between those with only school qualifications, those with a bachelor’s degree and those with a graduate qualification, which once again is typical. Students ranged in age from 10 to over 90 (Figure 2): the youngest person to pass the course was an 11 years old while the oldest was 91. Not in particular the substantial number of retirees doing the course.  Figure 2: Age distribution of people enrolled in the first course at the end of its first run. Note that the ages are based on self-reported birth dates.  3. Student Progression As can be seen from Table 1, over 80% of the students who enrolled did not start the course (starting being defined as having attempted any piece of assessment). This figure is slightly lower than is typical for edX MOOCS (Jordan 2014, Ho et al. 2015). The odds of starting the course were strongly dependent on age (Figure 3).   Figure 3: Probability of an enrolled student starting the first course, as a function of age, as measured at the start of its first run.  Even if a student started the course, most only watched the first 1-3 videos and did not attempt any assessment (Figure 4). Once again this is typical (Breslow 2015).   Figure 4: Number of times each video less in the first course was viewed (measured as the end of its first run)  Once a student had completed the first homework assignment, however, they had a 75% chance of going on to pass the first course, which is somewhat higher than is typical (Ho et al. 2015). This is comparable to the completion rate of more traditional on-line courses (e.g. Herbert 2006). Due to the very rapid decline in student numbers during the first few minutes of involvement in the course, however, this type of statistic will probably be highly sensitive to the exact nature and placement of the first homework assignment.  In the later courses in our series, the completion rates were complicated by the number of returning students who were working their way through the whole series. In course 4, for example, 10% of the students had previously passed one or more of the preceding courses in the series, while another 35% had enrolled in but not passed one of the preceding courses. Students who had passed a previous course had a 75% chance of passing the fourth course, while those who had not enrolled in any of the other courses only had a 5% pass rate. Interestingly, those students who had enrolled in but not completed any of the previous courses had an even lower completion rate – only 4%. Presumably these students were systematically auditing the series of courses.   It is thus clear that traditional completion rates (number of passing students divided by the number of enrolling students) are very low, but that this statistic does not mean the same thing in MOOCs as it does in on-campus courses. Enrolling in a course costs a student nothing, and many students just do it as a way of bookmarking a course for future reference (Ho et al. 2015). Even students who start the course do so with many different intentions: at the start of course 1, 44% of the students said they had no intention of doing any assessment. Once the initial rush is out of the way, however, we are left with a core of students who have a very respectable completion rate.  One surprise was the number of students who completed the first course after it finished its first run and was re-opened as a self-paced course. 1177 students passed in during its first run, and a very respectable 454 have passed it (as of April 2015) in self-paced mode.  4. The effectiveness of the MOOCs The courses were very popular with the students, achieving student satisfaction ratings that put them in the top 5% of ANU courses. At the end of the third course, students were asked how much they felt they had gained compared to typical face-to-face university courses they had done (over 80% of the students either already have a degree or are studying for one, so they can make a direct comparison). This was an anonymous voluntary survey link to from the feedback section of the course: it was completed by 18% of the students who passed the course: results would undoubtedly have been different had we been able to sample the students who dropped out earlier. Results are in Table 4. 87% said that they gained as much or more from this MOOC.   Table 4: Survey responses to the question “How much do you think you gained from this online course, compared to the typical face-to-face courses you have done?” Students were asked to answer this question only if they had experience in tertiary on-campus courses. . Response Number of Students 1. I gained much more from the face-to-face course 5 2. 15 3. 50 4.  52 5. I gained much more from the MOOC 34   The students were asked to explain their reasoning. The main reason cited for the superiority of MOOCs was the replayability of the videos. Here are some representative comments: ‘In the MOOC, I could really pay attention in the lectures without having to do so much rapid note taking that all I was doing was copying down the blackboard into my notebook. We get it all, and can go back and attend to the lecture as many times as needed to get the concepts through my thick head.’ ‘Doing lessons in a MOOC allows for pausing, rewinding, re-watching, custom pacing, and flexible scheduling. All of that contributes to better understanding of the material.’ Of course, these benefits can also accrue to flipped on-campus classes.  5. Conclusions The first conclusion is that these Australian MOOCs are reaching a student population that is very similar to that reached by overseas MOOCs. The courses attracted tens of thousands of students who were overwhelmingly from overseas: students who mostly knew little or nothing about Australian universities prior to enrolling. This suggests that MOOCs may be a cost-effective way to publicise Australian universities in places where currently their profile is very low.  A second conclusion is that the metrics normally used to assess Australian university courses, such a completion rates, are not really meaningful for MOOCs. Partially this is because enrolling in a MOOC is cost-free for the students, so they can engage in the courses in many more ways than in a traditional high-cost course. In addition, it is hard even to compute completion rates, because the courses were re-opened to self-paced study after their first run. For a self-paced course, you cannot even define an end-point at which to measure the number of completing students, though if they were left open for long enough you could compute some sort of student flux.  And finally, these courses taught tens of thousands of students from all parts of the world at a cost per student hundreds of times lower than traditional on-campus courses. And the students rated them as a better learning experience than traditional courses. Like them or loath them, that is surely telling us that courses like these will have an important role going into the future.  REFERENCES Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research & Practice in Assessment, 8(1), 13-25. http://mooc.pku.edu.cn/source/resource/01.pdf  Colvin, K. F., Champaign, J., Liu, A., Zhou, Q., Fredericks, C., & Pritchard, D. E. (2014). Learning in an introductory physics MOOC: All cohorts learn equally, including an on-campus class. The International Review of Research in Open and Distributed Learning, 15(4). http://www.irrodl.org/index.php/irrodl/article/view/1902  Despujol, I. M., Turro, C., Busqueis, J., & Canero, A. (2014, October). Analysis of demographics and results of student's opinion survey of a large scale mooc deployment for the spanish speaking community. In Frontiers in Education Conference (FIE), 2014 IEEE (pp. 1-8). IEEE. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7044102  Dillahunt, T. R., Wang, B. Z., & Teasley, S. (2014). Democratizing higher education: Exploring MOOC use among those who cannot afford a formal education. The International Review of Research in Open and Distributed Learning, 15(5). http://www.irrodl.org/index.php/irrodl/article/view/1841  Ebben, M., & Murphy, J. S. (2014). Unpacking MOOC scholarly discourse: a review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328-345. http://www.tandfonline.com/doi/abs/10.1080/17439884.2013.878352  Francis, P. (2006). Using role-playing games to teach astronomy: An evaluation. Astronomy Education Review, 4(2), 1-9. http://portico.org/Portico/#!journalAUSimpleView/tab=PDF?cs=ISSN_15391515?ct=E-Journal%20Content?auId=ark:/27927/pgg3ztf84s1  Francis, P. J. (2013, September). Examplecasts: The unreasonable effectiveness of webcast worked examples in introductory university physics. In Proceedings of The Australian Conference on Science and Mathematics Education (formerly UniServe Science Conference). http://openjournals.library.usyd.edu.au/index.php/IISME/article/view/7042  Francis, P. J. (2015). Fantasy Universes: Inquiry Learning in Astrophysics On-Campus and Massive Open Online Courses. Accepted for publication in International Journal of Innovation in Science and Mathematics Education.   Guo, P. J., Kim, J., & Rubin, R. (2014, March). How video production affects student engagement: An empirical study of mooc videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 41-50). ACM. http://dl.acm.org/citation.cfm?id=2566239  Herbert, M. (2006). Staying the course: A study in online student satisfaction and retention. Online Journal of Distance Learning Administration, 9(4). http://www.westga.edu/~distance/ojdla/winter94/herbert94.htm  Ho, A. D., Chuang, I., Reich, J., Coleman, C., Whitehill, J., Northcutt, C., Williams, J. J., Hansen, J., Lopez, G., & Petersen, R. (2015). HarvardX and MITx: Two years of open online courses (HarvardX Working Paper No. 10). doi:10.2139/ssrn.2586847 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2586847  Jordan, K. (2014), Initial trends in enrolment and completion of Massive Open Online Courses, The International Review of Research in Open and Distributed Learning, Vol 15, No 1. http://www.irrodl.org/index.php/irrodl/article/view/1651  Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distributed Learning, 14(3), 202-227. http://www.irrodl.org/index.php/irrodl/article/view/1455  Rayyan, S., Seaton, D. T., Belcher, J., Pritchard, D. E., & Chuang, I. (2013). Participation and performance in 8.02 x electricity and magnetism: The first physics MOOC from MITx. arXiv preprint arXiv:1310.3173. http://arxiv.org/abs/1310.3173   

Students’ demographics and behaviour in a series of Australian massive open online courses

https://openresearch-repository.anu.edu.au/bitstream/1885/147244/1/ACSME2015.pdf

Students’ demographics and behaviour in a series of Australian massive open online courses

Abstract

Similar works

Full text

Available Versions

The Australian National University