Formative assessment strategies for students' conceptions—The potential of learning analytics

Formative assessment is considered to be helpful in students' learning support and teaching design. Following Aufschnaiter's and Alonzo's framework, formative assessment practices of teachers can be subdivided into three practices: eliciting evidence, interpreting evidence and responding. Since students' conceptions are judged to be important for meaningful learning across disciplines, teachers are required to assess their students' conceptions. The focus of this article lies on the discussion of learning analytics for supporting the assessment of students' conceptions in class. The existing and potential contributions of learning analytics are discussed related to the named formative assessment framework in order to enhance the teachers' options to consider individual students' conceptions. We refer to findings from biology and computer science education on existing assessment tools and identify limitations and potentials with respect to the assessment of students' conceptions. Practitioner notes What is already known about this topic Students' conceptions are considered to be important for learning processes, but interpreting evidence for learning with respect to students' conceptions is challenging for teachers. Assessment tools have been developed in different educational domains for teaching practice. Techniques from artificial intelligence and machine learning have been applied for automated assessment of specific aspects of learning. What does the paper add Findings on existing assessment tools from two educational domains are summarised and limitations with respect to assessment of students' conceptions are identified. Relevent data that needs to be analysed for insights into students' conceptions is identified from an educational perspective. Potential contributions of learning analytics to support the challenging task to elicit students' conceptions are discussed. Implications for practice and/or policy Learning analytics can enhance the eliciting of students' conceptions. Based on the analysis of existing works, further exploration and developments of analysis techniques for unstructured text and multimodal data are desirable to support the eliciting of students' conceptions

Written notes and listening comprehension: A correlation study

Notetaking is a crucial aspect of learning in academic contexts, but as a relatively casual form of academic writing, it seldom receives pedagogic or research attention in the literature. Therefore, as more students study academic content through English as a second language (L2), research on student notetaking as a form of academic writing deserves attention. What students write in their notes and how they do so can play important roles in comprehension and learning. To address this gap, the present study examines 102 sets of notes and corresponding listening comprehension test scores to determine the relationships between four factors of quantity and quality in students’ hand-written notes; namely, notations, words, information units, and efficiency ratio. Results indicate that total notations and total words written in notes do not impact overall test scores, while information units and higher efficiency ratios positively correlate to test scores. The paper closes with pedagogic advice for teachers and students operating in L2 academic contexts with a focus on how best to conceptualise and write notes

Experiences of Middle School Students With Visual Impairments Accessing Technologies In Inclusive Classrooms

As the educational environment is moving more towards a technology-rich system, students with visual impairments (VI) educated in general education classrooms must be guaranteed equitable access to content curricula. The purpose of this study was to understand the experiences of middle school students with VI when accessing and using technologies in general education classrooms. In this multiple case study, three middle school students with VI were observed in general education settings for two school days. In addition to the students, general education teachers and teachers of students with VI (TVI) also participated in the study to understand how best they support access to technologies for students with VI in their classrooms. The theoretical framework that guided this study was Piaget’s cognitive development theory, and the learning model was Universal Design for Learning. Data were collected through multiple instruments: observations, interviews, and educational documents. Students, their general education teachers, and TVIs were interviewed about their experiences with the use of technologies in classrooms. After data collection, the analysis was completed using within-case and cross-case analysis. The within-case analysis revealed the experiences of using technologies in general education classrooms for each student in the form of a narrative story. Each student’s story included the components: (a) how did they see their world?, (b) how did they experience their school day?, and (c) how did their ideal world compare to their real world? The cross-case analysis was conducted by comparing participants’ experiences with technologies in general education classrooms. Four broad themes emerged from the cross-case analysis: (a) technology is imperative in general education classrooms; (b) frustrations with accessibility issues in general education classrooms; (c) for general education teachers, it has been a learning curve; and (d) for TVIs, the buck stops with them when it comes to access technology. Within the above four broad themes, some emerged findings were intriguing. General education teachers were open to training on technologies that are more engaging for students, as opposed to technologies that were universally accessible. Inaccessible technologies used in classrooms were not only the ones adopted by the school or district, but they included programs that were created and shared by other teachers through learning communities. While the students, general education teachers, and TVIs in this study understood the legal mandates of IDEA and an IEP, they did not know any other accessibility laws related to technologies that Kindergarten-Grade 12 schools should abide by. Conceptually, some sub-themes found in this study were: (a) the majority of educators were differentiating the curricula to meet the needs of students through constant adaptation as opposed to using tools that account for learner variability at the outset, and (b) student choice and advocacy played a big role in the experiences of students with VI in general education classrooms. Based on the findings, implications for practice and future research directions are discussed in this study

Assessing Student Learning with Technology: A Descriptive Study of Technology-Using Teacher Practice and Technological Pedagogical Content Knowledge (TPACK)

In 2013, a majority of states in the US had adopted Common Core State Standards under the Race to the Top initiative. With this adoption came the opportunity to utilize computer-delivered and computer-adaptive testing. Although the computer-based assessments were intended to assist teachers in designing classroom assessments and using student data to inform instructional practice, teacher-reported data indicated that the areas in which teachers are most unprepared, lack confidence, or are in need of development were assessment (DeLuca, 2012; Wayman et al., 2007) and technology (Brush & Saye, 2009; Kramarski & Michalsky, 2010). The Technology Assessment Practices Survey (TAPS) study was developed based on research in assessment literacy and in the technological pedagogical content knowledge framework. The purpose for developing this mixed-method study was the need to understand better how technology-using teachers assess student learning with technology. Two primary research questions facilitated a description of the assessment literacy and use of technology by 84 technology-using teachers. Participants in the study represented a diverse population of self-identified technology-using teachers. Quantitative and qualitative data were analyzed to provide insight into how technology-using teachers use technology to assess student learning. These data were analyzed for fitness with the TPACK theoretical model of teacher knowledge in order to fill an identified gap in the TPACK research (Cox & Graham, 2010). The TAPS study shows that technology-using teachers who belong to professional-education organizations have higher levels of confidence in both assessment and technology. Quantitative and qualitative data collected in the study also provides insight into the ways in which technology-using teachers think about, design, implement, and use the results of assessments in the classroom. Technology-using teachers exemplify TPACK, including attention to context at the macro, meso, and micro levels (Abbitt, 2011; Doering et al., 2009; Koehler & Mishra, 2009; Mishra & Koehler, 2005, 2006; Porras-Hernandez & Salinas-Amescua, 2013; Voogt et al., 2012). Future qualitative and quantitative research is needed into how preservice and inservice teachers use technology to assess student learning. Stakeholders in national, state, and local educational institutions need to consider how they are supporting the successful use of technology to assess student learning

The virtual classroom: building the foundations

This is a report on the first year of a three-year project concerned with the development and assessment of new types of software capabilities designed to support university level courses. A virtual classroom or university without walls is being created within a computerized conferencing system. During the first year of the project, students in twelve courses at three universities completed part or all of their coursework online. Pre and post-course questionnaires and automatic monitoring of their computer-mediated communications are the main sources of data. Independent variables include the expectations and attributes of the individual students; characteristics of the particular hardware and software which they use; and variations among classes in the nature of the assignments and activities required or facilitated by the instructor. Intervening variables include the amount and type of use of the system by the students, and the extent to which group learning takes place. Dependent variables are course outcomes and judgments by the students about the relative value of traditional and virtual classrooms. There is considerable variance in outcomes, particularly in student assessments of whether the virtual classroom is a better learning experience and whether they learned more or learned less. There was also extreme variation in measures of activity levels by students. For instance, the mean number of student sessions online was 41, but the standard deviation was 61; and the mean number of comments (contributions per student to the class discussion) was six, while the standard deviation was eight. Variations in measures of online activity and outcomes were significantly related to course, pre-use expectations of the students, sex, and system access variables including workstation hardware and response time. However, the strongest relationships are for measures of process vs. outcome. Those students who actively participated (by making comments rather than just reading the comments of others, and by engaging in private communication online with a number of other students as well as the professor) and those students who experienced group learning (learning from peer-group activity rather than one-way transmission of knowledge from professor to student) reported the most positive outcomes

Applying science of learning in education: Infusing psychological science into the curriculum

The field of specialization known as the science of learning is not, in fact, one field. Science of learning is a term that serves as an umbrella for many lines of research, theory, and application. A term with an even wider reach is Learning Sciences (Sawyer, 2006). The present book represents a sliver, albeit a substantial one, of the scholarship on the science of learning and its application in educational settings (Science of Instruction, Mayer 2011). Although much, but not all, of what is presented in this book is focused on learning in college and university settings, teachers of all academic levels may find the recommendations made by chapter authors of service. The overarching theme of this book is on the interplay between the science of learning, the science of instruction, and the science of assessment (Mayer, 2011). The science of learning is a systematic and empirical approach to understanding how people learn. More formally, Mayer (2011) defined the science of learning as the “scientific study of how people learn” (p. 3). The science of instruction (Mayer 2011), informed in part by the science of learning, is also on display throughout the book. Mayer defined the science of instruction as the “scientific study of how to help people learn” (p. 3). Finally, the assessment of student learning (e.g., learning, remembering, transferring knowledge) during and after instruction helps us determine the effectiveness of our instructional methods. Mayer defined the science of assessment as the “scientific study of how to determine what people know” (p.3). Most of the research and applications presented in this book are completed within a science of learning framework. Researchers first conducted research to understand how people learn in certain controlled contexts (i.e., in the laboratory) and then they, or others, began to consider how these understandings could be applied in educational settings. Work on the cognitive load theory of learning, which is discussed in depth in several chapters of this book (e.g., Chew; Lee and Kalyuga; Mayer; Renkl), provides an excellent example that documents how science of learning has led to valuable work on the science of instruction. Most of the work described in this book is based on theory and research in cognitive psychology. We might have selected other topics (and, thus, other authors) that have their research base in behavior analysis, computational modeling and computer science, neuroscience, etc. We made the selections we did because the work of our authors ties together nicely and seemed to us to have direct applicability in academic settings

A study of the effects on achievement of supplemental computer-aided instruction versus supplemental reading in the instruction of micro-economics

The purpose of this investigation was to assess the effectiveness of computer aided instruction in the achievement of students in undergraduate microeconomic classes. The procedures were to use a quasi-experimental pretest-posttest, non-equivalent control group design. Two forms of the Test of Understanding of College Economics and a questionnaire were administered to the students in order to test three hypotheses dealing with differences in achievement among the groups based on type of instruction, keyboard familiarity, GPA, socioeconomic background and teacher effects. The analysis of data confirmed the hypotheses that there would be a significant difference between the posttest scores of the control groups and the treatment groups. The hypothesis that the CAI group would score significantly higher than the reading group was not confirmed. The chosen variables to explain the differences in achievement were not significant although GPA, family income, and number of prior economic classes displayed a trend towards significance. The trend of the data analysis appeared to confirm the beneficial effects of CAI and the theory of operant conditioning

Digital connection in a physical classroom: clickers and the student-teacher relationship

Education is fundamentally relational, and the student-teacher relationship is central to student learning. However, high-enrollment classrooms, now common on college campuses, limit student-faculty interaction and opportunities for relationship building. “Clickers” facilitate communication in large classes, but there is a lack of research on the potential relational functions of this technology. This study addresses this gap in the literature by asking: How might the use of clickers in the classroom contribute to the student-teacher relationship? Employing a mixed-method descriptive research design, I created and analyzed three data sets to respond to this question: I observed 3 large clicker-based classes, surveyed students to explore their perceptions of clicker use and student-teacher relational dimensions, and I interviewed a subset of students for assistance interpreting the results. Data analyses resulted in four general findings: clickers can be used for multiple purposes and ends; clickers facilitate aspects of the student-teacher pedagogical relationship; clicker communication is not perceived as comprising a student-teacher relationship; and clickers are viewed as a tool for collective rather than individual communication and dialogue. Clickers may have value as relational tools, as they facilitate some aspects of the student-teacher relationship. The frame of the technology may explain why only some relational dimensions are facilitated, and not others. More research is needed to explore how clickers and other educational technologies may facilitate the student-teacher relationship