Scott Bentsen, Niclas

English

Tidsskrift.dk (Det Kongelige Bibliotek)

11Assessment tasks in the course ‘Energy Systemsand Climate Change Mitigation’: ensuringconstructive alignmentNiclas Scott BentsenDepartment of Geosciences and Natural Resource Management, University ofCopenhagenIntroductionIn this project, where I focus on exams and assessment tasks in the course‘Energy systems and climate change mitigation’ I wish to, based on thetheories on constructive alignment, particularly the learning outcome-assessmenttask dimension (ILO-AT), explore how student case work as a teaching andlearning activity (TLA) can be integrated in the final assessment/exam tofurther increase course alignment.Constructive Alignment (CA) is an outcome based didactic theory de-veloped by John Biggs. The fundamental assumption is that deep learningis supported by focusing on what students do rather than on what studentsare, or what teachers do (Biggs & Tang 2007, Mørcke & Rump 2013). CAbuilds on constructivism (hence the first part of the name) as formalizedby Jean Piaget (Dolin 2013) suggesting that learners construct new know-ledge out of their present experience through an interaction between thatexperience and the experience of the surrounding world, e.g. a teacher.‘Alignment’ refers to the process of stimulating deep learning by en-suring that the learner experiences coherence between what the learner isexpected to become capable of (Intended Learning Outcomes, ILOs), thelearning environment (Teaching and Learning Activities, TLAs) and theassessment of the learners learning (Assessment Tasks, ATs).With respect to assessments Biggs & Tang (2007) argue that whileteachers often consider the ILOs as the guiding element in a teaching-learning event students might consider differently. To the student passing132 Niclas Scott Bentsenthe exam is the element that guides learning activities (Fig. 11.1). This so-called (negative) backwash effect leads to surface learning.Fig. 11.1. Teacher’s and student’s conflicting view on assessments. Adopted fromBiggs & Tang (2007), fig. 9.2, page 169.Backwash can be used positively. If the final summative assessment isaligned with the learning outcomes then teaching activities and learningactivities focus on the same goal, the ILOs. When students prepare for theassessment they automatically train for the learning outcomes (Biggs &Tang 2007, Frankland 2007).Material and methodsThis report is based on own experience teaching the course in 2014 and15, and on oral feedback from the students. Written evaluations on Absalonwere few and did not provide specific input to the question treated here.It hasn’t been possible to carry out the suggested changes of exam formswithin the timeframe of the Teaching and Learning in Higher Educationprogramme.CourseThe course ‘Energy Systems and Climate Change Mitigation’ is a 7.5 ECTSrestricted elective course in the MSc programme in Climate Change at Uni-versity of Copenhagen. The course covers one block (8 weeks) in the springsemester. Apart from climate change students the course is open to studentsfrom other programmes at UCHP and from external students. The coursewas executed first time in 2014 (academic year 13/14) with 30 students.11 Assessment in ‘Energy Systems and Climate Change Mitigation’ 133In 2015 40 students attended. The final assessment and grading of studentsis based on four hours written test at the end of the course. The test takesplace at the university (corresponds to exam type IV in table 11.2). Thetest is a mix of calculation exercises, discussion items and multiple choicequestions. Multiple choice questions are used to test basic facts like energyconversion efficiency, energy units and forms, and scale relations (ILO 2).Discussion items focus on evaluation of strategies and interrelations be-tween energy and climate systems (ILO 1 and 4). Calculation exercisestarget ILO 1, 2, 4 and to limited extent 3.Course alignmentCourse alignment The level of course alignment is presented in table 11.1below. By comparing the lecture plan, learning outcomes in the course de-scription and the options provided by the exam form I find misalignment be-tween some assessment tasks and learning outcomes. As can be seen fromtable 11.1 misalignment particularly occurs for the elements emphasizingthe highest level of understanding (level 4 and 5 in the SOLO taxonomy).I evaluate the current exam form inadequate of supporting ILO 3, 5and 6 because these requires more time that four hours for the students toidentify relevant problems, find and analyse data, and reflect over strengthsand weaknesses of potential solutions. Moreover ILO 6, work in inter-disciplinary groups, is incompatible with the current exam form.Student feedback and own experienceA core element in the teacher’s and course responsible’s approach to teach-ing the course is to provide the students with hand-on experience in de-veloping, designing and evaluating renewable energy systems at all scales(ILO 4-6, table 11.1). A chosen activity to support that outcome is groupwork on a self-elected case. From my perspective the case work is veryvaluable in supporting the learning outcomes and this is seconded by stu-dent feedback.Suggested changesIn the following I present my suggestions for changed exam forms to en-sure further course alignment in support of deeper learning. However exam134 Niclas Scott BentsenTable 11.1. Evaluated levels of constructive alignment between intended learningoutcomes (ILO), teaching and learning activities (TLA) and assessment tasks (TA).Central verbs in the ILOs are set in italics. The ‘SOLO’ column refers to levels ofunderstanding following the SOLO taxonomy (Biggs & Tang 2007), table 5.1 page80. Filled arrows indicate alignment, open arrows indicate misalignment.forms cannot be changed at will and all forms have their benefits and weak-nesses, which are discussed.Regulatory frameworkExam forms are particularly regulated by ministerial order 670 of 19 June2014 on exam and censorship at university studies [Bekendtgørelse om ek-samen og censur ved universitetsuddannelser] (Uddannelses- og Forskn-ingsministeriet 2014), and the joint curriculum for BSc and MSc pro-grammes at the Faculty of Science (UCPH 2015). The ministerial ordersection 4 states that exams can be organized as individual or group testsbut in any case an individual assessment must be performed followed byindividual grading of the students.The curriculum section 6.1 states that exams can be organized as oneor more tests leading to either a weighted assessment and grading of the11 Assessment in ‘Energy Systems and Climate Change Mitigation’ 135individual tests or an overall assessment and grading of the test portfo-lio. Furthermore section 6.4.1 states that written tests can be organized asindividual or group tests. However, group tests must allow for individualassessment and grading of the students requiring either a declaration of theindividual contributions by each student to the written work or a subsequentoral examination.The core constraint is that no matter how the final assessment and grad-ing is organized it must provide the examiner and censor the means to assessand grade the students individually.Exam formsIt is important that the choice of exam form ensures reliability as well asvalidity (Biggs & Tang 2007, Andersen et al. 2013). Reliability meaningthat the assessment and grading is unequivocal, transparent and not depen-dent on external factors like the student, examiner or censor them-selves,the place and surroundings but only on the performance. Validity requiresthat the exam tests the learners’ learning with reference to the purpose ofthe learning (ILOs). An example of low validity is presented by Vennervald(2010) in her project on a course in human parasitology, where a multiplechoice test was found inadequate to test “the students’ abilities to analyseproblems in relation to human parasite infections in a broader context inrelation to human health”.Andersen et al. (2013) distinguish exam forms by product type andwhere the exam takes place. Exam forms commonly encountered in Den-mark are illustrated in table 11.2 below. An exam form more commonlyused abroad, course participation, is in a Danish context and regulatoryframework considered a prerequisite for exam participation rather than anexam form in it-self (Andersen et al. 2013). Within each form the exam canbe executed in numerous different ways.Oral exams are often considered having relatively low reliability be-cause they depend too much on e.g. chance, personality, and the student’seloquence and language proficiency (Andersen et al. 2013). On the otherhand the oral exam allows the (good) examiner to test the student’s concep-tual understanding and not only her/his ability to reproduce facts. Higherreliability is associated with some forms of written exams, e.g. multiplechoice tests, calculation exercises as assessment and grading relies on theproduct (result) rather than on the process of reaching the result. As suchthe subjectivity is removed from the assessment.136 Niclas Scott BentsenTable 11.2. Exam forms distinguished by product type and place. An off-site examtakes place where the student determines. An on-site exam takes place at the educa-tional institution or a place the institution determines. Adopted from Andersen et al.(2013), figure 1, page 372.New formA new exam form must emphasise alignment on the ILO-AT axis with focuson learning outcome 3, 5 and 6 without changing the well-established ILO-AT alignment for outcome 1, 2 and 4. A shared teaching activity targetingoutcome 3, 5 and 6 (and 4) is the self-elected case work, and I find it obviousto include, in some form, the case work in the final assessment. The outputof the case work is a poster and an oral plenum presentation of the poster.As organised currently neither the joint poster nor the plenum presentationmeets the regulatory requirements for individual assessment (curriculumsections 6.4.1 and 6.4.2), and based on the positive feedback on the casework and own experience I would be hesitant to change the format.Execution of oral exams is generally perceived as more resource de-manding than written exams (Andersen et al. 2013). This may particularlybe true to courses with many students. With the number and growth of stu-dents in the ‘Energy Systems and Climate Change Mitigation’ course oralexams is not considered feasible. Consequently an individual oral exambased on the case work (type III, table 11.2) is not considered.My suggestion for a new exam form is a weighted two part exam. Onepart is an individual essay based on the case work weighing 40 % (typeI, table 11.2) and the other part is a shorter, e.g. 2.5-3 hours, written examcomparable to the current (type IV, table 11.2) weighing 60 %.11 Assessment in ‘Energy Systems and Climate Change Mitigation’ 137ReflectionsMy suggestion for a new exam form for the ‘Energy and Climate ChangeMitigation’ course could achieve the goal of course constructive alignmentand ensure a reliable and valid assessment of the students learning.The essay part of the exam allows the students to analyse, characterise,compare, contrast, generalise and reflect on their own experience and learn-ing. All verbs associated with the relational and extended abstract level ofunderstanding in the SOLO taxonomy (Biggs & Tang 2007). To achievethis, the essay should not be a summary of the poster and case work buta personal reflection over the implications and perspectives of the casework findings. If planned and organised well the essay could target learn-ing outcomes 3-6. A risk with the essay format suggested here, producedun-invigilated is plagiarism (Andersen et al. 2013). However, as it will bebased on supervised case work, I believe the risk is small, but for that reasonI suggest it to account for less than 50 % of the final grade.To minimise the risk of plagiarism and other fraud it is important to setup a very clear framework of guidelines and expectation for the students(Andersen et al. 2013, Biggs & Tang 2007), and finally software exist todetect internet plagiarism.The validity of the suggested format is high as it builds on a real lifecase. Reliability in the assessment and grading is challenging for the exam-iner and censor and requires not only that they are experienced in assessingwritten work, but also a comprehensive documentation of the assessment(Andersen et al. 2013).The written on-site test could be used more specifically to target learn-ing outcome 1-4 with calculation exercises and multiple choice tests. Thesetest forms are considered highly reliable (Andersen et al. 2013). Validity ofthe test must be ensured in the formulation of test items as topical, timely,and targeted conceptual understanding.Misalignment on the ILO-AT axis is experienced in other courses andtreated in earlier projects in the Teaching and Learning in Higher Educa-tion programme. Vennervald (2010) suggested a change from an on-sitemultiple choice test to an on-site, heavily guided, short essay based on ascientific paper to e.g. ensure alignment, encourage deeper learning, assesshigher levels of the SOLO taxonomy, and activate students. When Dams-gaard (2013) found ILO-AT misalignment in a course in human nutritionshe did not change exam form (on-site oral test partly based on writtenproduct) but revised exam questions to ensure alignment and to demand138 Niclas Scott Bentsenanswers on a higher taxonomic level. The outcome of the changes waspositively received by students as well as other teachers on the course.Hougaard (2013) experienced ILO-AT misalignment because the exam, inpractice, only aligned with parts of the learning outcomes. 75 % of the grad-ing was attributed to a project report and the oral exam (25 %) should targetthe rest of the curriculum, but did not. Her solution was to shift weights be-tween the written and oral part of the exam to equal weights. Furthermoreshe discusses the possibilities of introducing more formative assessment orfeed-back, and hence ILO-AT alignment in a course through student re-flection papers for each module (6 modules in the course), but fears thatan excessive workload be put on the teachers. Hansen (2013) experiencesa similar form of misalignment, where practice doesn’t follow the formalcourse description, and too much weight is put on a report and too littleon the oral examination of the curriculum. His observations did not lead tochanged exam forms but to changed behaviour for the teachers not only toincrease alignment but also to prevent administrative problems caused bythe disagreement between course description and student assessment prac-tice. As demonstrated in the above cases ILO-AT misalignment is not rare,but several actions can be taken to improve it. Changed exam form is onlyone approach. Also shifted weighting between exam elements and changedpractice has been applied to improve alignment.SummaryIn summary this project finds ILO-AT misalignment in the course ‘EnergySystems and Climate Change Mitigation’ and discusses options and lim-itations to improve alignment to support deeper learning at higher taxo-nomic levels. The report discusses the validity and reliability of differentoral and written exam forms, and the risk of plagiarism and fraud. Limita-tions imposed by regulation requiring individual assessment together withcost expectations narrow the field of applicable options to improve align-ment. Finally the project suggests a new exam form for the course madeup of two written parts: An individual un-invigilated essay based on su-pervised group work and a ‘traditional’ invigilated exam with calculationexercises and multiple purpose tests.All contributions to this volume can be found at: http://www.ind.ku.dk/publikationer/up_projekter/2015-8/The bibliography can be found at:http://www.ind.ku.dk/publikationer/up_projekter/kapitler/2015_vol8_nr1-2_bibliography.pdf/

Assessment tasks in the course ‘Energy Systems and Climate Change Mitigation’: ensuring constructive alignment

https://tidsskrift.dk/IUSTL/article/download/137871/182151

Assessment tasks in the course ‘Energy Systems and Climate Change Mitigation’: ensuring constructive alignment

Abstract

Similar works

Full text

Available Versions

Tidsskrift.dk (Det Kongelige Bibliotek)