Online self‐assessment materials: Do these make a difference to student learning?

During the last decade the teaching of students in first‐year biology has changed from a teacher‐centred focus to a student‐centred focus. The change was designed to encourage students to take responsibility for their learning, develop team and communication skills and put in practice those skills required for lifelong learning. Students are introduced to small learning communities (in large classes) that give the students a sense of belonging and a peer support group. Activities have been devised and implemented to support student‐centred learning, which in more recent years has included using computers. All these activities are integrated into the course design so that the students are offered an array of learning opportunities relevant to the course(s) learning outcomes. An important requirement for the development of student‐centred learning is suitable and timely feedback that gives students guidance about their learning outcomes. Giving feedback to very large classes is an expensive commodity and one that is vulnerable in the current climate of reducing resources and increasing student numbers. First‐year biology students receive online feedback from a weekly quiz (with both formative and summative components), from a mid‐semester mock exam (formative only) and from a series of self‐assessment modules (formative only). This paper will examine the use of such online self‐assessment in a large first‐year biology class, discuss current evaluations of the materials and propose further research into how students use these integrated learning opportunities

Investigating Different Types of Assessment in Massive Open Online Courses

The current technological era has largely influenced the development of learning environments. As a result, there are new opportunities for teaching, learning and assessment. The emergence of Massive Open Online Courses (MOOCs) in particular, has attracted the attention of higher education institutions and course designers. MOOCs may provide the opportunity to thousands of students to learn from anywhere and at their convenience. Assessment is a component of the learning environment that drives student learning. However, only a small proportion of existing literature on assessment investigates its use for the enhancement of educational growth as most of the literature is concerned with how to use assessment for purposes of grading and ranking (Rowntree, 1987). Assessment has a double role in learning by both motivating students to study in order to undertake it, but also providing the necessary feedback on their performance so that students can track their learning progress (Rowntree, 1987). Research in MOOCs is currently growing, focusing on different aspects such as the “questionable course quality, high dropout rate, unavailable course credits, complex copyright, limited hardware and ineffective assessments” (Chen, 2014). Assessment in MOOCs has been mostly investigated from a perspective that is looking at: how the grading load can be diminished by adopting automated techniques, the aims of each technique, and finally new potential approaches that will be able to assess high-level cognition. Summing up, researchers are currently testing tools that will be automatically scoring essays and giving feedback to learners in an effective way (see Balfour, 2013). However, the learners’ voice and standpoint about the different assessment types in the MOOCs context is inconclusive in the current literature and there is need for more research. This study explores learners’ views on assessment types in Massive Open Online Courses, whether any of these has an impact on their enrolment and completion of a course and in what aspects each type of assessment is effective in supporting their learning experience. Auto-assessment, peer-assessment and self-assessment are the types under investigation as they are frequently used in MOOCs and therefore are the most commonly discussed in literature (see Balfour, 2013, Suen, 2013, Wilkowski et al, 2014). The study draws upon literature on assessment in general and on assessment in MOOCs in particular. The concept of online communities, i.e. the learners that appear in MOOCs will also be discussed in detail. Online ethnographic approaches are employed to explore the issue in question by using online interviewing and observation methods. Thematic analysis is carried out using a sample of 12 MOOCs participants from online interviews and 13 posts of online observations. The outcome of this qualitative research study reveals that even though participants identify benefits in peer assessment, there is a preference for automated assessment since it is an already known, clear type of assessment for them. Moreover, self-assessment is not popular by participants. Learners’ comments also reveal that a clear guidance for assessment helps them to carry out peer assessment more effectively. Some learners also consider that the combination of assessment types may also have a positive effect on students’ learning as each of them serves a different purpose

Generating Feedback from Self and Peer Review: Developing Evaluative Judgement

Peer review not only results in students receiving additional feedback from peers but it also encourages self-assessment and develops feedback literacy (e.g. Carless & Boud, 2018; Morrow, 2006; Nicol, Thomson & Breslin, 2014). In this presentation we will examine how the quality of the work reviewed and stage of the review process (i.e. peer review, self-review or receiving reviews) influences students’ evaluative judgments about their own work. Students anonymously reviewed 3 pieces of work via an established online peer review tool. Two of these online submissions were from their peers, and one was an exemplar written by the teacher. After each review, students completed a “self-review” of their own work using a rubric, and after all three reviews, they then received feedback comments from their peers. Participants were psychology students, and the contribution of each aspect of the peer review process to their learning was evaluated using a quantitative questionnaire, open ended questions and focus groups. When asked how the peer review process contributed to their learning, students indicated that reviewing the work of their peers contributed most. Typical comments were: “I felt this helped me to see if I was on the right track with my own work”; “I already had a template in my own head of what I had written so could see how theirs differed to mine”; “Just from reviewing their work I could identify areas in which mine could be improved”. Our findings suggest that while students reported the process of ‘reviewing and commenting’ as challenging, students learned more from comparing, reflecting and generating feedback for themselves than from receiving feedback comments. This presentation will discuss how use of peer review as a platform to develop evaluative judgement can provide an authentic method of assessment which prepares students for a final summative assessment

An activity theory investigation of tool-use in undergraduate mathematics

This mixed methods study investigates a number of aspects related to tool-use in undergraduate mathematics as seen from an Activity Theory perspective. The aims of this study include: identifying the tools that undergraduates use; seeking for an empirically-based typology of these tools; examining how undergraduates themselves can be profiled according to their tool-use; and finally identifying the factors influencing students tool preferences. By combining results from survey, interview and diary data analyses, it was found that undergraduates in the sample preferred using mostly tools related to their institution s practice (notes, textbooks, VLE), other students and online videos. All the tools students reported using were classified into five categories: peers; teachers; external online tools; the official textbook; and notes. Students in the sample were also classified into five distinct groups: those preferring interacting with peers when studying mathematics (peer-learning group); those favouring using online tools (online-learning group); those using all the tools available to them (blended-learning group); those using only textbooks (predominantly textbooks-learning group); and students using some of the tools available to them (selective-learning group). The main factor shaping students tool choices was found to be their exam-driven goals when examined from an individual s perspective or their institution s assessment related rules when adopting a wider perspective. Results of this study suggest that students blend their learning of mathematics by using a variety of tools and underlines that although undergraduates were found to be driven by exam-related goals, this is a result of the rules regulating how Higher Education Institutions (HEI) function and should not be attributed entirely as stemming from individuals practices. Assigning undergraduates exam- driven goals to their university s sociocultural environment, was made possible by combining two versions of Activity Theory (Leontiev and Engestrøm s) and analysing data at two different levels (individual and collective respectively)

Teaching Information Assurance Online

Cyber security threats, systems vulnerabilities, privacy concerns, and other security developments are progressing and proliferating rapidly.   This article describes experience accumulated in teaching an online Information Assurance course at a 4-year college for graduate and undergraduate students. The course content is based on the security curriculum structured by ten security domains of knowledge, as defined by the (ISC)² for professional certification programs CISSP of (ISC)². Only eight out of ten domains are included in the 10-week IA-Study course. The courses uses several basic e-learning activities - online studies supported with Via Voice-narrated slides presentation, video presentation, online reference book and supplemental readings; collaborative and individual coursework supported by Online WorkBook (WB) in Google Docs, assessment quizzes administered online via Blackboard/WebCT, and team/class discussions and networking supported with a dedicated Google Blog.   WB facilitates collaborative students’ work using Wiki technology; it provides both work instructions and the place to submit the students work. Using a log in the Google Docs, the instructor can see individual contributions of team members to a collaborative project.  In addition, each team is required to submit members’ peer evaluations.   WikiDigest project – a value-added work-in-progress - opens opportunities for the instructor and the students to work together on collecting Tips and Recommendations for an Information Security Awareness knowledge base that can be used on campus by students and faculty.   In conclusion, the article reviews students’ perceptions of online learning such as acceptance of online learning in general, time needed and actually spent to study the course, and effectiveness of team collaboration

Metacognitive Assessment Model for Student Project-Based Learning through the Blended Learning Practice MOOCs

Online or blended learning assessments through LMS-MOOCs carried out in the world of education today tend to be multiple-choice assessments that are only based on low-level cognitive. In fact, to measure the metacognitive of students is quite difficult, if only using the form of multiple choice questions. Therefore, it takes the form of questions and assessments that allow students to explore their reflective and metacognitive thinking according to the characteristics of the education they are attending. Vocational education tends to apply a project-based learning (PjBL) model that requires authentic and performance-based learning assessment methods. Therefore, it is necessary to develop an appropriate blended metacognitive skill assessment rubric instrument for vocational education. Metacognitive assessment was developed using research and development procedures, with students as subjects in vocational education in Makassar, Indonesia. The integration between elements of metacognitive skills: planning, monitoring, and evaluation with self-peer-teacher assessment can be an assessment method to measure students’ metacognitive thinking skills in PjBL. Especially metacognitive assessment through blended learning practice MOOCs that are in accordance with the characteristics of vocational education and can be adopted by general education

An investigation into novel software tools for enhancing students' higher cognitive skills in computer programming

Active learning is considered by many academics as an important and effective learning strategy. Students can improve the quality of their work by developing their higher cognitive skills through reflection on their own ideas, and through practice of analytic and evaluative skills. Assessment is a tool for learning, but traditional assessment methods often encourage surface learning, rather than deep learning which is an approach to developing higher cognitive skills. Peer assessment is one of the successful approaches, which can be used to enhance deep learning. It is a method of motivating students, involving students discussing, marking and providing feedback on other students' work. Although it is often used in the context of essays, it has seldom been applied to computer programming courses. The skill of writing good software includes understanding different approaches to the task, and stylistic and related considerations - these can be developed by evaluation of other programmers' solutions. As part of a study investigating the extent that peer assessment can promote deep learning to develop the higher cognitive skills in a programming course, a novel web-based peer assessment tool has been developed. - The process used is novel, since students are engaged not only in marking each other's work, but also in evaluating the quality of marking of their peers. - This system is designed to provide anonymity for the whole process, in order to ensure that the process is fair, and to encourage students to discuss without embarrassment by using an anonymous communication device (ACD) in a variety of roles (script authors, marker, and feedback marker). In this thesis, we describe and compare the learning theory and tools, which are relevant in learning computer programming. Deep learning, which can be described using the six categories of learning in Bloom's taxonomy, is discussed. Other peer assessment software tools are compared and discussed. The design and implementation of a novel web-based peer assessment system (with anonymous communication device) are described, and set in the context of the learning theories. The results of evaluating the tools through several experiments involving large programming classes and an essay writing module are reported. In this thesis, we also propose a new variation of Bloom's taxonomy, which is appropriate to describe the skills required for tasks such as programming. The results indicate that this approach to web-based peer assessment has successfully helped students to develop their higher cognitive skills in learning computer programming, and peer assessment is an accurate assessment method in a programming course.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Building Better Teams at Work using Self and Peer Assessment Practices

There is a reported competency gap between the teamwork skills required by employers and those developed by engineering students during their undergraduate courses. The University of Technology, Sydney is addressing this issue by combining project-based learning with self and peer assessment to determine an individualâs team performance. A confidential online tool is used to collect and collate the student self and peer assessment ratings used both for formative feedback as well as assessment purposes. We found this approach improved student teamwork, engagement and satisfaction, while requiring only a small commitment of academic resources. We propose that using self and peer assessment would also be beneficial in industry to change workplace cultures, promote teamwork, individual skill development, engagement and productivity. Our implementation produces two assessment factors. The performance factor may be used by managers to coach staff to improve their performance, while the formative feedback factor will assist individuals to identify their strengths and weaknesses as perceived by their peers, facilitating ongoing skill development

Visual Analytics: Design Study for Exploratory Analytics on Peer Profiles, Activity and Learning Performance for MOOC Forum Activity Assessment

The massively open online course (MOOC) has become an increasingly popular alternative platform for education due to its open concept and free features. Due to its features that allow enrolment on a massive scale and participation across the globe, it presented new analytic challenges. The vast amount and variety of data generated pose challenges for the learning analytics community to analyse especially concerning peer presence and peer learning. Forum activity data offers the opportunity to assess the relationship between forum activities and user backgrounds with the learner’s progression and retention rate. Furthermore, there are several challenges in implementing data visualization in real-world scenarios such as different task characterisation compared to the existing analytics, along with varied factors on the usability of visualization among the domain analysts. Despite many research on learning analytics, most of the approaches were data-driven and there were only a handful of studies that were focused on interactive visualization design to facilitate MOOC forum user activity assessment using real-world scenarios and educational theories-driven. Our design study aims to investigate and formulate a visual analytic design to facilitate enriched visual analysis towards assessing forum activity in Malaysian MOOC, particularly in pattern and relationship exploration on the user diverse background and activities with the learning performance. This paper presents our review on visual learning analytics and current MOOC practice in Malaysia, our design study methodology and proposed conceptual visual analytics design on visualizing forum activity data

Choosing the best from blended and online e-learning

Blended learning should incorporate the best of contact and online learning, allowing flexibility while retaining connectedness. Therefore, designing effective instruction requires research-informed choices. The Community of Inquiry (CoI) survey is a wellvalidated instrument measuring social-, teaching- and cognitive presences in e-learning that reflect the quality of e-learning courses (Garrison, Anderson and Archer 2000). We compared the CoI results of two blended postgraduate courses: one predominantly online and the other mostly in contact mode, taught by the same lecturer. The endof- course deliverable for both courses in research methodology was a research proposal. Both courses utilised the learning management system (LMS), while students with insufficient Internet access communicated via email or telephone. Both courses included the two-tiered double-blind electronic peer review of assignments. One group had weekly contact sessions, and the other only an initial welcoming session. We discuss using peer review for formative feedback as a particularly beneficial strategy to facilitate teaching effectively in such large classes, and the limitations thereof. The CoI survey showed the strengths of the online environment, with very strong teaching presences due to good organisation, comprehensive online supportive documentation, and automated feedback. High cognitive presence was due to peer review; strong constructive alignment between objectives, activities and assessment; and in the online class due to the constructivist teaching practice of fostering student ownership of outcomes. In both classes social presence was the weakest, although the contact class scored significantly higher on this presence. Low social presence did not compromise course completion in any group.