Opening up multiple-choice: assessing with confidence

Multiple choice questions (MCQ) are the basic fare of e-assessment. MCQs are robust and easy to implement, but are pedagogically not ideal; open questions are preferable but automated marking of free text answers is problematic (although see Butcher & Jordan 2010). A possible squaring of this circle is to appropriate the technique of confidence-based marking (CBM). In CBM, a student both selects an answer and also their level of confidence: they score full marks for knowing that they know the correct answer, some credit for a tentative correct answer but are penalised if they believe they know the answer but get it wrong (Gardner-Medwin 1995, 2006). There are several motivations for CBM: it rewards care and effort so engendering greater engagement, it encourages reflective learning (Gardner-Medwin & Curtin 2007; Nix & Wyllie 2011). This project will take CBM and, with one simple change, enrol it for a different end. Here the MCQ is presented in two stages. Initially, the question is presented with no answer options visible; instead the student must set their confidence level that they know the answer. Only then are the possible answers are revealed and the student answers as a normal MCQ. The marking scheme follows standard CBM practice. Mechanically the question remains a simple MCQ: answer matching is trivial and robust, questions are easy to implement, and existing question banks can be reused. However, to the student, the question is effectively transformed from closed MCQ to an open question. They need to formulate an answer first before they can decide their confidence in their answer, so they must decide their answer in the absence of any positive or negative clues, reducing the chance of misconceptions, or working backwards. The project will trial certainty-first CBM questions in an Open University distance learning course under a controlled experimental design to probe whether students using CBM will engage better with questions, improve their learning, and become more reflective learners (Nicol 2007). Measures of assignment scores and time on task will be collected, together with a survey and/or interview to probe attitudinal aspects. References Butcher, PG & Jordan, SE (2010). A comparison of human and computer marking of short free-text student responses. Computers and Education, 55, 489-499. Gardner-Medwin, AR (1995). Confidence assessment in the teaching of basic science. Association for Learning Technology Journal, 3, 80-85. Gardner-Medwin, AR (2006). Confidence-based marking - towards deeper learning and better exams. Chapter 12 in C. Bryan, & K. Clegg (Eds.), Innovative assessment in higher education Routledge; London. Gardner-Medwin, T & Curtin, N (2007). Certainty-based marking (CBM) for reflective learning and proper knowledge assessment. Online conference: REAP07: Assessment Design for Learner Responsibility, Re-Engineering Assessment Practices in Higher Education: Univs. of Strathclyde, Glasgow, Glasgow Caledonian. [Online: http://www.ucl.ac.uk/lapt/REAP_cbm.pdf] Nicol, D. (2007). E-assessment by design: Using multiple-choice tests to good effect. Journal of further and Higher Education, 31, 53-64. Nix, I & Wyllie, A (2011). Exploring design features to enhance computer-based assessment: Learners' views on using a confidence indicator tool and computer-based feedback. British Journal of Educational Technology, 42, 101-112

Cracking the Code on Stem: A People Strategy for Nevada\u27s Economy

Nevada has in place a plausible economic diversification strategy—and it’s beginning to work. Now, the state and its regions need to craft a people strategy. Specifically, the state needs to boost the number of Nevadans who possess at least some postsecondary training in the fields of science, technology, engineering, or math—the so-called “STEM” disciplines (to which some leaders add arts and design to make it “STEAM”). The moment is urgent—and only heightened by the projected worker needs of Tesla Motors’ planned “gigafactory” for lithium-ion batteries in Storey County. Even before the recent Tesla commitment, a number of the more high-tech industry sectors targeted by the state’s new economic diversification strategy had begun to deliver significant growth. Most notable in fast-growing sectors like Business IT Ecosystems (as defined by the Governor’s Office for Economic Development) and large sectors like Health and Medical Services, this growth has begun to increase the demand in Nevada for workers with at least a modicum of postsecondary training in one or more STE M discipline. However, there is a problem. Even though many available opportunities require no more than the right community college certificate, insufficient numbers of Nevadans have pursued even a little STEM training. As a result, too few Nevadans are ready to participate in the state’s emerging STEM economy. The upshot: Without concerted action to prepare more Nevadans for jobs in STEM-intensive fields, skills shortages could limit growth in the state’s most promising target industries and Nevadans could miss out on employment that offers superior paths to opportunity and advancement. Which is the challenge this report addresses: Aimed at focusing the state at a critical moment, this analysis speaks to Nevada’s STEM challenge by providing a new assessment of Nevada’s STEM economy and labor market as well as a review of actions that leaders throughout the state—whether in the public, private, civic, or philanthropic sectors—can take to develop a workforce capable of supporting continued growth through economic diversification

Developing Online Team Skills

This paper discusses the development and delivery of a new course at the UK Open University (OU). The course makes a virtue of the fact that OU students study in a predominantly distance learning environment, by providing a structure within which team working activities are carried out with no face-to-face contact whatsoever. Issues that were considered in designing this course and decisions about tutoring and assessing the students' team working experience are discussed. Preliminary results from the first delivery of the course, including analysis of archived team conferences, are presented. Directions for future development and enhancement are indicated

Teaching Electronics and Programming in Norwegian Schools Using the air:bit Sensor Kit

We describe lessons learned from using the air:bit project to introduce more than 150 students in the Norwegian upper secondary school to computer programming, engineering and environmental sciences. In the air:bit project, students build and code a portable air quality sensor kits, and use their air:bit to collect data to investigate patterns in air quality in their local environment. When the project ended students had collected more than 400,000 measurements with their air:bit kits, and could describe local patterns in air quality. Students participate in all parts of the project, from soldering components and programming the sensors, to analyzing the air quality measurements. We conducted a survey after the project and describe our lessons learned from the project. The results show that the project successfully taught the students fundamental concepts in computer programming, electronics, and the scientific method. In addition, all the participating teachers reported that their students had showed good learning outcomes

Support of the collaborative inquiry learning process: influence of support on task and team regulation

Regulation of the learning process is an important condition for efficient and effective learning. In collaborative learning, students have to regulate their collaborative activities (team regulation) next to the regulation of their own learning process focused on the task at hand (task regulation). In this study, we investigate how support of collaborative inquiry learning can influence the use of regulative activities of students. Furthermore, we explore the possible relations between task regulation, team regulation and learning results. This study involves tenth-grade students who worked in pairs in a collaborative inquiry learning environment that was based on a computer simulation, Collisions, developed in the program SimQuest. Students of the same team worked on two different computers and communicated through chat. Chat logs of students from three different conditions are compared. Students in the first condition did not receive any support at all (Control condition). In the second condition, students received an instruction in effective communication, the RIDE rules (RIDE condition). In the third condition, students were, in addition to receiving the RIDE rules instruction, supported by the Collaborative Hypothesis Tool (CHT), which helped the students with formulating hypotheses together (CHT condition). The results show that students overall used more team regulation than task regulation. In the RIDE condition and the CHT condition, students regulated their team activities most often. Moreover, in the CHT condition the regulation of team activities was positively related to the learning results. We can conclude that different measures of support can enhance the use of team regulative activities, which in turn can lead to better learning results

Innovating Pedagogy 2015: Open University Innovation Report 4

This series of reports explores new forms of teaching, learning and assessment for an interactive world, to guide teachers and policy makers in productive innovation. This fourth report proposes ten innovations that are already in currency but have not yet had a profound influence on education. To produce it, a group of academics at the Institute of Educational Technology in The Open University collaborated with researchers from the Center for Technology in Learning at SRI International. We proposed a long list of new educational terms, theories, and practices. We then pared these down to ten that have the potential to provoke major shifts in educational practice, particularly in post-school education. Lastly, we drew on published and unpublished writings to compile the ten sketches of new pedagogies that might transform education. These are summarised below in an approximate order of immediacy and timescale to widespread implementation

Pre-service teachers use e-learning technologies to enhance their learning

The purpose of this study was twofold. The primary purpose was to improve pre-service teacher education by using technology to help pre-service teachers bridge the gap between academic preparation and practice. The secondary, but still important, objective was to familiarize pre-service teachers in the use of technology to support their future pedagogical activities. Therefore, this research sought to develop a method for training undergraduate students in designing, implementing, and evaluating lesson plans to solidify the relationship between research, pedagogy, and teaching practice. Specifically, this study investigated the implementation of e-learning as a method of instruction to help pre-service teachers evaluate and improve upon the implementation of their lesson plans during their real world practicum experiences. The study was guided by the following research questions: 1) What successes, challenges, and benefits do university instructors and pre-service teachers experience in using and analyzing video in teacher education methods coursework? 2) In what ways did the use of e-learning help the pre-service teachers improve their teaching during the practicum experience? Results showed that participants reported improved lesson planning, improved lesson implementation, visual interpretations of best practices, modeling, and peer and university instructor feedback as successes of the e-learning project. Challenges included participants’ frustrations of being overworked and overwhelmed with the technical problems associated with e-learning. Overall participants judged the e-learning project as a very positive aspect of their teacher training

Cleveland Schools That Are Making a Difference

Profiles thirteen Cleveland schools -- a cross section of traditional public, private, parochial, and charter schools, where the majority of students are economically disadvantaged -- that have demonstrated progress in student achievement gains