Data-wary, Value-driven: Teacher Attitudes, Efficacy, and Online Access for Data-Based Decision Making

How do teachers use online student assessment data? School districts invest increasing resources in online systems for reporting and analyzing student assessment data, yet few studies describe the direct use of such applications or explore how these systems relate to teachers’ professional roles, data use attitudes, or data use efficacies. This dissertation applies learning analytics methods for log file analysis and visual data analytics to explore the extensive variation in teachers’ online data use behaviors and attitudes over six months in one urban secondary school. Descriptive statistics and visualizations of online usage over time demonstrate strong connections between teachers’ online behavior and common organizational factors, such as school level (middle vs. high school), content area, and required training. Correlational evidence suggests that data use self-efficacies have stronger relationships to online use than general data use attitudes. Hierarchical cluster analysis heatmaps are used to identify novel subgroups of teacher online data use behaviors and attitudes. These exploratory findings are used to generate data use dashboards for school-based leadership and an expanded determinant framework for the adoption of online assessment systems. Combining data-intensive methods with theoretical frameworks for self-efficacy, technology acceptance, and use diffusion, this dissertation aims to describe the rich variation in teachers’ online data use and attitudes, as well as productively inform the practice and study of data-based decision making in schools

Mitigating Negative Behaviors in Student Project Teams: An Information Technology Solution

The evolution of the information systems (IS) profession as well as the IS educational environment has resulted in student project teams becoming an increasingly important component of IS education. Negative behaviors within these project teams result in less than optimal project outcomes and confound attempts to fairly evaluate individual efforts within the team. These negative behaviors can also result in unfavorable perceptions of the educational experience for many students. An information technology-based application consisting of an online interactive activity log and an online peer review system was developed to mitigate the common negative behaviors of social loafing and free riding and provide more accurate and useful information for evaluation. Its successful implementation in a specific classroom environment is discussed and evidence of its efficacy is supplied in the form of subjective assessment by the instructors and the results of a survey measuring student perceptions of the system

Assessment of Professional Behavior in Students: Thoughts and Opinions of Occupational Therapy Faculty

Professionalism and professional behavior of students has been a growing concern in the health professions for the past two decades (Aguilar et al., 2013; Davis, 2009; Swick, 2000). As a result multiple professions have identified core values and the professional behaviors that are a reflection of these in practice; and have begun to integrate assessment of student professional behaviors into educational standards. Occupational therapy has a history of publication dating back to 1995 regarding assessment of the professional behaviors of students but no profession wide expectations have been established.;This study attempted to describe the thoughts and opinions of faculty in Master\u27s level entry Occupational Therapy educational programs nationwide regarding assessment of students\u27 professional behaviors. This study also looked for differences in thoughts and opinions between faculty from different types of Carnegie Classification institutions and full- versus part-time faculty. An invitation to complete an online survey was sent to all Occupational Therapy faculty identified through reviews of the websites of all 154 accredited occupational therapy education programs. The survey gathered demographic data (N = 294) and then asked the faculty to indicate their level of agreement with behaviors to be included in professional behavior assessment and asked them to assign value to the various methods used in assessment.;A total of 292 responses (22%) were included in the final statistical analysis. Participants assigned high importance to all behaviors listed in the survey and identified an additional 52 behaviors for inclusion in professional behavior assessment. Regarding assessment techniques participants preferred the inclusion of direct skilled observations of student behavior in classroom and clinical environments. Participants also indicated a preference for regular assessment intervals.;Statistical analysis of differences in responses of faculty from different Carnegie classification type institutions only revealed significant differences in the value of including observations of students\u27 behavior at conferences and non-institutional sponsored continuing education events with faculty from Baccalaureate institutions assigning less value to these observations. Faculty from Baccalaureate institutions also assigned less importance to compiling assessment results to reflect performance of the group as a whole than faculty from other institution types.;When comparing responses from full-time and part-time faculty some statistically significant results were identified. Full-time faculty assigned more importance to the assessment of initiative and responsibility for own learning. In the area of assessment practices full-time faculty assigned less value to observations of student behaviors in lab sessions while part-time faculty assigned less value to the observations of student behavior in group work and assigned less importance to the use of assessment results in program evaluation and to inform professional development opportunities.;Occupational therapy is a broad profession that practices in many settings with a variety of clients with a wide range of disabilities. This diversity of practice expectations is reflected in the importance assigned to all of the behaviors included in the survey as well as the large number of additionally written in behaviors to be included in assessment. The profession of occupational therapy will be strengthened by identifying the essential behaviors that students should develop in academic programs to reflect the professional values of occupational therapy in their professional practice. Occupational therapy faculty express that they value assessment that occurs at regular intervals and includes direct skilled observations and student self-assessment of student\u27s behavior in the classroom, clinic, and professional activities. Further research is needed to identify key behaviors, best assessment practices, and to establish a clear relationship between behavior exhibited in the academic and clinical environment

Thinking Systemically: a Study of Course Communication and Social Processes in Face-to-Face and Online Courses

Traditionally, research that has examined online courses compared course modes, online and face-to-face (f2f). Studies tend to examine the two modes to determine whether online courses are as effective as online courses by comparing student outcomes, such as student learning and satisfaction. Seldom has research examined how the course communication in online and f2f courses impact student outcomes. Moreover, there is little examination of the relationship between the design of the course and the relationship with social processes, in particular, communication. In this study, t-tests indicated that there were no significant differences between antecedents (technological familiarity and instructional characteristics) and outcomes variables (learning, performance, and satisfaction) between online or face-to-face courses. However, there were significant differences in course communication constructs including richness, social presence, learning community, and active learning behaviors. Multiple regression analyses indicated assessment and evaluation in instructional characteristics explained 36% of the variance in social presence, 42% of the variance in richness, and 27% of the variance in a learning community. Two components in instructional characteristics, organization and instructional design and course support, did not contribute to the model predicting these communication variables. However, they did predict 55% of the variance in engagement. Assessment and evaluation did not contribute to the model for predicting engagement. Assessment and evaluation are key factors in predicting communication variables where organization and instructional design and course support are a key factor in predicting engagement. Finally, multiple regression analyses indicated that 67% of the variance of learning can be predicted by communication variables of social presence, richness, engagement, and learning community, 52% of the variance of performance can be predicted by richness and engagement, 72% of the variance of satisfaction can be predicted by richness, engagement, and presence. Self-reported active learning behaviors did not predict learning, performance, or satisfaction

An Investigation of Using Wikis as a Collaborative Tool for Teaching in a Non-Western Tertiary Education Classroom

Wikis are innovative Web 2.0 tools that allow users to create, expand, and edit content collaboratively. This makes wikis promising for online collaborative learning, but further exploration is required to determine if using wikis can achieve learning goals efficiently and appropriately. With increasing globalization, it is useful to determine how students from non-Western cultures respond to using wikis in the learning environment. In this study, the author compares two Taiwanese undergraduate classes with different instructional design and peer assessment strategies to understand Taiwanese student online behaviors and learning preferences in a learning environment using wikis. The results indicate that students prefer to be assigned responsibility in group projects rather than free writing assignments, and student collaboration is limited in the assignment. The author also discusses the efficiency of in-group peer assessment and inter-group peer assessment used in group projects. The author suggests using wikis as a knowledge management system may be better than using wikis as a collaborative tool in tertiary education classroom

Online peer assessment: Effects of cognitive and affective feedback

This study reports the effects of online peer assessment, in the form of peer grading and peer feedback, on students' learning. One hundred and eighty one high school students engaged in peer assessment via an online system-iLap. The number of grade-giving and grade-receiving experiences was examined and the peer feedback was coded according to different cognitive and affective dimensions. The effects, on both assessors and assessees, were analyzed using multiple regression. The results indicate that the provision by student assessors of feedback that identified problems and gave suggestions was a significant predictor of the performance of the assessors themselves, and that positive affective feedback was related to the performance of assessees. However, peer grading behaviors were not a significant predictor of project performance. This study explains the benefits of online peer assessment in general and highlights the importance of specific types of feedback. Moreover, it expands our understanding of how peer assessment affects the different parties involved. © 2011 The Author(s).published_or_final_versionSpringer Open Choice, 21 Feb 201

ANALYZING AND MODELING STUDENTS¿ BEHAVIORAL DYNAMICS IN CONFIDENCE-BASED ASSESSMENT

Confidence-based assessment is a two-dimensional assessment paradigm which considers the confidence or expectancy level a student has about the answer, to ascertain his/her actual knowledge. Several researchers have discussed the usefulness of this model over the traditional one-dimensional assessment approach, which takes the number of correctly answered questions as a sole parameter to calculate the test scores of a student. Additionally, some educational psychologists and theorists have found that confidence-based assessment has a positive impact on students\u2019 academic performance, knowledge retention, and metacognitive abilities of self-regulation and engagement depicted during a learning process. However, to the best of our knowledge, these findings are not exploited by the educational data mining community, aiming to exploit students (logged) data to investigate their performance and behavioral characteristics in order to enhance their performance outcomes and/or learning experiences. Engagement reflects a student\u2019s active participation in an ongoing task or process, that becomes even more important when students are interacting with a computer-based learning or assessment system. There is some evidence that students\u2019 online engagement (which is estimated through their behaviors while interacting with a learning/assessment environment) is also positively correlated with good performance scores. However, no data mining method to date has measured students engagement behaviors during confidence-based assessment. This Ph.D. research work aimed to identify, analyze, model and predict students\u2019 dynamic behaviors triggered by their progression in a computer-based assessment system, offering confidence-driven questions. The data was collected from two experimental studies conducted with undergraduate students who solved a number of problems during confidence-based assessment. In this thesis, we first addressed the challenge of identifying different parameters representing students\u2019 problem-solving behaviors that are positively correlated with confidence-based assessment. Next, we developed a novel scheme to classify students\u2019 problem-solving activities into engaged or disengaged behaviors using the three previously identified parameters namely: students\u2019 response correctness, confidence level, feedback seeking/no-seeking behavior. Our next challenge was to exploit the students\u2019 interactions recorded at the micro-level, i.e. event by event, by the computer-based assessment tools, to estimate their intended engagement behaviors during the assessment. We also observed that traditional non-mixture, first-order Markov chain is inadequate to capture students\u2019 evolving behaviors revealed from their interactions with a computer-based learning/assessment system. We, therefore, investigated mixture Markov models to map students trails of performed activities. However, the quality of the resultant Markov chains is critically dependent on the initialization of the algorithm, which is usually performed randomly. We proposed a new approach for initializing the Expectation-Maximization algorithm for multivariate categorical data we called K-EM. Our method achieved better prediction accuracy and convergence rate in contrast to two pre-existing algorithms when applied on two real datasets. This doctoral research work contributes to elevate the existing states of the educational research (i.e. theoretical aspect) and the educational data mining area (i.e. empirical aspect). The outcomes of this work pave the way to a framework for an adaptive confidence-based assessment system, contributing to one of the central components of Adaptive Learning, that is, personalized student models. The adaptive system can exploit data generated in a confidence-based assessment system, to model students\u2019 behavioral profiles and provide personalized feedback to improve students\u2019 confidence accuracy and knowledge by considering their behavioral dynamics

UNDERSTANDING STUDENT BEHAVIORS USING IMMEDIATE FEEDBACK FEATURES IN A BLENDED LEARNING ENVIRONMENT

Feedback serves to close the gap between learners’ current understanding and the desired understanding. Informative feedback can keep students from holding onto misconceptions, actively engage learners in knowledge acquisition, and increase confidence and motivation to learn. Yet, in the context of higher education, it is usually not possible for instructors to provide timely feedback to every individual student. This is especially difficult in first-year foundational courses due to the large number of students. Online learning platforms offer a solution by providing students immediate feedback during the course of their interactions with formative assessment tools (e.g., online homework, quizzes, embedded questions in lecture videos). However, how students choose to interact with these features and how these features influence students’ learning experiences have not been well understood. Even less is known about student behaviors with these immediate feedback features in a blended learning class

The impact of student facilitation on student engagement in asynchronous online discussion

Lack of engagement in asynchronous online discussion is a common issue due to insufficient participation, disinterest in discussion, superficial discussion, and the influence of peer behaviors. The purpose of this study was to enhance student engagement in asynchronous online discussion by using student-led discussion and peer assessment. A mixed methods design was utilized in this study to offset quantitative and qualitative research weaknesses and to draw on the strengths of both (Plano Clark & Creswell, 2008). The control group participated in the discussion managed by the instructor, who posted weekly discussion questions to a class discussion board and facilitated the discussion. The experimental group worked in smaller discussion groups, led by a different student each week who posed a discussion question, facilitated the group discussion, and conducted a peer review. Interviews and discussion post analysis showed that students were behaviorally engaged and the way in which they demonstrated cognitive and emotional engagement varied. Survey results indicated that there was no statistically significant difference in the control group and experimental groups’ cognitive and emotional engagement, and academic performance. Students liked the student-led format, however, they preferred participating rather than leading. Directions for future research on student engagement in online discussion were identified