BeSocratic: An Intelligent Tutoring System for the Recognition, Evaluation, and Analysis of Free-form Student Input

This dissertation describes a novel intelligent tutoring system, BeSocratic, which aims to help fill the gap between simple multiple-choice systems and free-response systems. BeSocratic focuses on targeting questions that are free-form in nature yet defined to the point which allows for automatic evaluation and analysis. The system includes a set of modules which provide instructors with tools to assess student performance. Beyond text boxes and multiple-choice questions, BeSocratic contains several modules that recognize, evaluate, provide feedback, and analyze student-drawn structures, including Euclidean graphs, chemistry molecules, computer science graphs, and simple drawings. Our system uses a visual, rule-based authoring system which enables the creation of activities for use within science, technology, engineering, and mathematics classrooms. BeSocratic records each action that students make within the system. Using a set of post-analysis tools, teachers have the ability to examine both individual and group performances. We accomplish this using hidden Markov model-based clustering techniques and visualizations. These visualizations can help teachers quickly identify common strategies and errors for large groups of students. Furthermore, analysis results can be used directly to improve activities through advanced detection of student errors and refined feedback. BeSocratic activities have been created and tested at several universities. We report specific results from several activities, and discuss how BeSocratic\u27s analysis tools are being used with data from other systems. We specifically detail two chemistry activities and one computer science activity: (1) an activity focused on improving mechanism use, (2) an activity which assesses student understanding of Gibbs energy, and (3) an activity which teaches students the fundamentals of splay trees. In addition to analyzing data collected from students within BeSocratic, we share our visualizations and results from analyzing data gathered with another educational system, PhET

Building Chemistry One Atom at a Time: An Investigation of the Effects of Two Curricula in Students\u27 Understanding of Covalent Bonding and Atomic Size

Chemists have to rely on models to aid in the explanation of phenomena they experience. Instruction of atomic theory has been used as the introduction and primary model for many concepts in chemistry. Therefore, it is important for students to have a robust understanding of the different atomic models, their relationships and their limitations. Previous research has shown that students have alternative conceptions concerning their interpretation of atomic models, but there is less exploration into how students apply their understanding of atomic structure to other chemical concepts. Therefore, this research concentrated on the development of three Model Eliciting Activities to investigate the most fundamental topic of the atom and how students applied their atomic model to covalent bonding and atomic size. Along with the investigation into students’ use of their atomic models, a comparison was included between a traditional chemistry curriculum using an Atoms First approach and Chemistry, Life, the Universe and Everything (CLUE), a NSF–funded general chemistry curriculum. Treatment and Control groups were employed to determine the effectiveness of the curricula in conveying the relationship between atoms, covalent bonds and atomic size. The CLUE students developed a Cloud representation on the Atomic Model Eliciting Activity and maintained this depiction through the Covalent Bonding Model Eliciting Activity. The traditional students more often illustrated the atom using a Bohr representation and continued to apply the same model to their portrayal of covalent bonding. During the analysis of the Atomic Size Model Eliciting Activity, students had difficulty fully supporting their explanation of the atomic size trend. Utilizing the beSocratic platform, an activity was designed to aid students\u27 construction of explanations using Toulmin’s Argumentation Pattern. In order to study the effectiveness of the activity, the students were asked questions relating to a four-week long investigation into the identity of an inorganic salt during their laboratory class. Students who completed the activity exhibited an improvement in their explanation of the identity of their salt’s cation. After completing the activity, another question was posed about the identity of their anion. Both groups saw a decrease in the percentage of students who included reasoning in their answer; however, the activity group maintained a significantly higher percentage of responses with a reasoning than the control group