Research to Practice: Leveraging Concept Inventories in Statics Instruction

There are many common challenges with classroom assessment, especially in first-year large enrollment courses, including managing high quality assessment within time constraints, and promoting effective study strategies. This paper presents two studies: 1) using the CATS instrument to validate multiple-choice format exams for classroom assessment, and 2) using the CATS instrument as a measure of metacognitive growth over time. The first study focused on validation of instructor generated multiple choice exams because they are easier to administer, grade, and return for timely feedback, especially for large enrollment classes. The limitation of multiple choice exams, however, is that it is very difficult to construct questions to measure higher order content knowledge beyond recalling facts. A correlational study was used to compare multiple choice exam scores with relevant portions of the CATS assessment (taken within a week of one another). The results indicated a strong relationship between student performance on the CATS assessment and instructor generated exams, which infers that both assessments were measuring similar content areas. The second study focused on a metacognition, more specifically, on students’ ability to self-assess the extent of their own knowledge. In this study students were asked to rank their confidence for each CATS item on a 1 (not at all confident) to 4 (very confident) Likert-type scale. With the 4-point scale, there was no neutral option provided; students were forced to identify some degree of confident or not confident. A regression analysis was used to compare the relationship between performance and confidence for pre, post, and delayed-post assessments. Results suggested that the students’ self-knowledge of their performance improved over time

Recruiting VR Troopers: Bringing Introductory Programming Projects to Life in Virtual Reality

Classes in introductory programming often focus on solving small, succinct problems that can typically be completed in few lines of source code. While useful for learning the basics of algorithm implementation and language syntax, this method suggests to learners that all programming problems exist in isolation and are self-contained. In contrast, most programming assignments faced by fresh graduates are large in scope and require use of many pre-built libraries and extensions. As a result, students are not entirely prepared to write code that will function within a larger system. To address this problem, an introductory C programming course at Valparaiso University has explored the use of virtual reality as a means to motivate students to have fun while practicing coding skills and showcase the power of working within constraints of a complex system. Students are provided a brief introduction to the OpenGL 3D graphics framework and then asked to design a small, optionally animated, scene using their current knowledge of the C programming language. Later in the semester, these same students are brought into a VisCube Virtual Reality system to experience their scenes in a fully immersive environment. The VisCube uses eight rendering paths and stereo displays to generate a 3D scene in a 10’x8’x6’ cube. This exercise serves to show students that even a simple scene can then easily expanded to display in a virtual reality environment. We discuss the project assignment and student impacts using assessment and provide a brief discussion of how this can be adapted to facilities with other viualization capabilities

Crowdsourcing Classroom Observations to Identify Misconceptions in Data Science

Web-browsing histories, online newspapers, streaming music, and stock prices all show that we live in an age of data. Extracting meaning from data is necessary in many fields to comprehend the information flow. This need has fueled rapid growth in data science education aiming to serve the next generation of policy makers, data science researchers, and global citizens. Initially, teaching practices have been drawn from data science\u27s parent disciplines (e.g., computer science and mathematics). This project addresses the early stages of developing a concept inventory of student difficulty within the newly emerging field of data science. In particular this project will address three primary research objectives: (1) identify student misconceptions in data science courses; (2) document students’ prior knowledge and identify courses that teach early data science concepts; and (3) confirm expert identification of data science concepts, and their importance for introductory-level data science curricula. During the first year of this grant, we have collected approximately 200 responses for a survey to confirm concepts from an existing body of knowledge presented by the Edison Project. Survey respondents are comprised of faculty and industry practitioners within data science and closely related fields. Preliminary analysis of these results will be presented with respect to our third research objective. In addition, we developed and launched a pilot assessment for identifying student difficulties within data science courses. The protocol includes regular responses to reflective questions by faculty, teaching assistants, and students from selected data science courses offered at the three participating institutions. Preliminary analyses will be presented along with implications for future data collection in year two of the project. In addition to the anticipated results, we expect that the data collection and analysis methodologies will be of interest to many scholars who have or will engage in discipline-based educational research

Evaluation of EDISON\u27s Data Science Competency Framework Through a Comparative Literature Analysis

During the emergence of Data Science as a distinct discipline, discussions of what exactly constitutes Data Science have been a source of contention, with no clear resolution. These disagreements have been exacerbated by the lack of a clear single disciplinary \u27parent.\u27 Many early efforts at defining curricula and courses exist, with the EDISON Project\u27s Data Science Framework (EDISON-DSF) from the European Union being the most complete. The EDISON-DSF includes both a Data Science Body of Knowledge (DS-BoK) and Competency Framework (CF-DS). This paper takes a critical look at how EDISON\u27s CF-DS compares to recent work and other published curricular or course materials. We identify areas of strong agreement and disagreement with the framework. Results from the literature analysis provide strong insights into what topics the broader community see as belonging in (or not in) Data Science, both at curricular and course levels. This analysis can provide important guidance for groups working to formalize the discipline and any college or university looking to build their own undergraduate Data Science degree or programs

A Mixed-Method Approach to Investigating Difficulty in Data Science Education

The purpose of this study was to define a methodology to identify any disconnect between students and instructors in data science classrooms through analyzing qualitative data. A combined qualitative and quantitative approach was used for analysis of survey data from students, faculty/instructors, and teaching assistants across three institutions. Using a manual content analysis paired with a TF-IDF analysis, researchers were able to pull out frequently used terms within responses and encode them into categories and subcategories. Trends were identified from these categories and subcategories to examine general areas of disconnect within the data science classroom. Additionally, a quality analysis was run to determine the effectiveness of the phrasing of the questions posed during the survey. As a whole, the methods used throughout this research process provide direction for researchers in interpretation and analysis of the survey data in an efficient and time-sensitive manner. Furthermore, it allows researchers to analyze the quality of responses to give insight towards rephrasing of survey questions in future analyses. Although the research was applied to data science classrooms, this method has the potential to be applied into other fields and areas of study when performed with coordination between a field expert and a data scientist

Dr. Ruth Wertz - Teaching Engineering Online

Teaching engineering online is a uniquely challenging task. Conceptually difficult knowledge, hands-on experience, problem-solving and teamwork are at the core of many engineering courses and when taught remotely, require careful planning. Dr. Wertz talks to us about her practical and theoretical experience in online engineering education. She discusses topics of social bonding, team’s cohesion, assessment approaches, and generally, what is important to remember when designing (or redesigning) an engineering course for online instruction

Exploring the design of online instruction to support and engage learners with Dr. Ruth Wertz

How do you design online instruction to support and engage a community of learners? To explore this question, Dr. Ruth Streveler interviews Dr. Ruth Wertz (ENE PhD \u2714), Assistant Professor of General Engineering at Valparaiso University in Valparaiso Indiana. This episode was hosted by Dr. Ruth Streveler, produced by the School of Engineering Education at Purdue, and features music composed by Patrick Vogt

Toward a new model within the community of inquiry framework: Multivariate linear regression analyses based on graduate student perceptions of learning online

Online education has seen dramatic changes in recent years and been recognized as a significant learning platform. However, teaching and learning in these environments is not yet well understood. This study evaluated operational models of the Community of Inquiry (COI) framework that explain student perceptions of learning online. The study participants ( n = 256) were graduate students from multiple institutions who had taken at least one online course as part of their degree requirements. Survey data were collected using the WebTALK survey, a Likert-scale instrument. A two-phase, sequential quantitative research program composed of a series of multivariate linear regression analyses was conducted. The first phase was confirmatory and examined the measurement of four COI constructs: cognitive presence (CP), which represents students\u27 interaction with the course content; teaching presence (TP), which represents students\u27 interaction with instructional tools and learning activities; and social presence (SP), which represents students\u27 interaction with other learners and cultural aspects of the learning environment, and learning presence (LP), which represents students\u27 self-regulation and learning strategies. The first phase included a series of confirmatory factor analyses (CFAs) to evaluate the measurement models of the TP, SP, LP, and CP constructs individually, followed by the WebTALK measurement model, which modeled all four constructs simultaneously. The second phase focused on testing several hypotheses that explore how the COI constructs relate and interact with one another. These hypotheses were evaluated using SEM path analyses and hierarchical linear regression analyses. The findings indicate that the WebTALK instrument provided a reliable measure of all four COI constructs with Cronbach\u27s alpha values ranging from 0.63 to 0.92 and measurement models with very good model fit. While SP acted as a mediator between the other COI constructs when analyzed individually, mediation criteria were not met when all four constructs are included in the model simultaneously. In addition, LP did not have significant moderation effect in this study, which contradicts findings from independent prior research. Together, these findings indicate that LP significantly relates to the other COI constructs, but in a way that is not well-explained by the existing models. A post-hoc hypothesis was generated based on the analysis of the three hypotheses tested in the second phase of this study. Findings suggest that the post-hoc hypothesis, where both SP and LP act as mediators between TP and CP and SP has a direct effect on LP, should be accepted. This dissertation makes unique contributions to the study of online learning environments through the COI framework by introducing a comprehensive survey that includes learning presence and producing evidence on the multi-dimensionality of the COI constructs and strong relationship between learning presence and cognitive presence