Advancement Auto-Assessment of Students Knowledge States from Natural Language Input

Knowledge Assessment is a key element in adaptive instructional systems and in particular in Intelligent Tutoring Systems because fully adaptive tutoring presupposes accurate assessment. However, this is a challenging research problem as numerous factors affect students’ knowledge state estimation such as the difficulty level of the problem, time spent in solving the problem, etc. In this research work, we tackle this research problem from three perspectives: assessing the prior knowledge of students, assessing the natural language short and long students’ responses, and knowledge tracing.Prior knowledge assessment is an important component of knowledge assessment as it facilitates the adaptation of the instruction from the very beginning, i.e., when the student starts interacting with the (computer) tutor. Grouping students into groups with similar mental models and patterns of prior level of knowledge allows the system to select the right level of scaffolding for each group of students. While not adapting instruction to each individual learner, the advantage of adapting to groups of students based on a limited number of prior knowledge levels has the advantage of decreasing the authoring costs of the tutoring system. To achieve this goal of identifying or clustering students based on their prior knowledge, we have employed effective clustering algorithms. Automatically assessing open-ended student responses is another challenging aspect of knowledge assessment in ITSs. In dialogue-based ITSs, the main interaction between the learner and the system is natural language dialogue in which students freely respond to various system prompts or initiate dialogue moves in mixed-initiative dialogue systems. Assessing freely generated student responses in such contexts is challenging as students can express the same idea in different ways owing to different individual style preferences and varied individual cognitive abilities. To address this challenging task, we have proposed several novel deep learning models as they are capable to capture rich high-level semantic features of text. Knowledge tracing (KT) is an important type of knowledge assessment which consists of tracking students’ mastery of knowledge over time and predicting their future performances. Despite the state-of-the-art results of deep learning in this task, it has many limitations. For instance, most of the proposed methods ignore pertinent information (e.g., Prior knowledge) that can enhance the knowledge tracing capability and performance. Working toward this objective, we have proposed a generic deep learning framework that accounts for the engagement level of students, the difficulty of questions and the semantics of the questions and uses a novel times series model called Temporal Convolutional Network for future performance prediction. The advanced auto-assessment methods presented in this dissertation should enable better ways to estimate learner’s knowledge states and in turn the adaptive scaffolding those systems can provide which in turn should lead to more effective tutoring and better learning gains for students. Furthermore, the proposed method should enable more scalable development and deployment of ITSs across topics and domains for the benefit of all learners of all ages and backgrounds

Learning by Augmented Reality : Cluster Analysis Approach

Because the use of augmented reality (AR) is increasing, it is important to study its possibilities within both formal and informal learning contexts. We clustered 146 sixth graders using AR at a science center based on their reasoning, motivation, and science learning results using the self-organizing maps method (SOM) to identify AR-using subgroups. The aim was to consider reasons why the AR method could be of more beneficial for some students than others. The clustering results complemented earlier findings on AR gains in learning, as an unexpected response to intervention was discovered using this nonlinear analysis. The previous results had indicated that after the AR experience, science test results generally improved and particularly among students with the lowest achievement. The SOM-clustering results showed a majority group of boys, especially those interested in science learning both at school and at the science center using AR. Despite low school achievement, their high motivation led to good science learning results. The prior results, according to which girls closed the science knowledge gap between boys after using AR, became more relative, as two girldominated subgroups were identified. The reasons for the results were considered on the basis of motivation, multimedia learning theory, and concept formation theories. Keywords: science learning, augmented reality, informal learning environment, SOM-clustering, self-determination theoryPeer reviewe

Individual differences in the preference for worked examples: lessons from an application of dispositional learning analytics

Worked-examples have been established as an effective instructional format in problem-solving practices. However, less is known about variations in the use of worked examples across individuals at different stages in their learning process in student-centred learning contexts. This study investigates different profiles of students’ learning behaviours based on clustering learning dispositions, prior knowledge, and the choice of feedback strategies in a naturalistic setting. The study was conducted on 1,072 students over an eight-week long introductory mathematics course in a blended instructional format. While practising exercises in a digital learning environment, students can opt for tutored problem-solving, untutored problem-solving, or call worked examples. The results indicated six distinct profiles of learners regarding their feedback preferences in different learning phases. Finally, we investigated antecedents and consequences of these profiles and investigated the adequacy of used feedback strategies concerning ‘help-abuse’. This research indicates that the use of instructional scaffolds as worked-examples or hints and the efficiency of that use differs from student to student, making the attempt to find patterns at an overall level a hazardous endeavour

The Content of Their Coursework: Understanding Course-Taking Patterns at Community Colleges by Clustering Student Transcripts

Community college students typically have access to a large selection of courses and programs, and therefore the student transcripts at any one college or college system tend to be very diverse. As a result, it is difficult for faculty, administrators, and researchers to understand the course-taking patterns of students in order to determine what programs of study they appear to be pursuing. Attempting to examine these patterns and then comparing them with listed program requirements would be a very time-consuming activity; clustering can be a useful way to make sense of the relevant data. Clustering allows researchers to group similar items into clusters, relying only on a measure of the similarity of those items. In this paper, we apply a clustering algorithm to the problem of understanding college transcripts, which serve as the items to be clustered. To our knowledge, this is the first effort to organize transcripts based on their course content using clustering. We base the measure of similarity on the proportion of curricular subjects that each transcript has in common with every other one. Our data are community and technical college transcripts for a cohort of students who first entered the Washington State system during the fall of the 2005-06 academic year and who had no prior postsecondary experience. We used our clustering algorithm to separately cluster liberal arts and career-technical students. We found that the algorithm did a good job of separately clustering each of these groups. The clusters roughly corresponded to programs of study, so we were able to estimate how many students were undertaking each program and what subjects students were studying within each cluster. We were also able to examine the demographics and the completion and transfer rates of the students within each cluster, in order to get an idea of what types of students were in each program of study and how successful they seemed to be in college. We found substantial variation on these dimensions as well as on the extent to which students' programs were either concentrated in a single subject or spread across several subjects. We conclude that this method would be useful to researchers throughout education who are trying to understand student course-taking patterns and programs of study, and who need to organize large amounts of transcript data

Investigating attributes affecting the performance of WBI users

This is the post-print version of the final paper published in Computers and Education. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2013 Elsevier B.V.Numerous research studies have explored the effect of hypermedia on learners' performance using Web Based Instruction (WBI). A learner's performance is determined by their varying skills and abilities as well as various differences such as gender, cognitive style and prior knowledge. In this paper, we investigate how differences between individuals influenced learner's performance using a hypermedia system to accommodate an individual's preferences. The effect of learning performance is investigated to explore relationships between measurement attributes including gain scores (post-test minus pre-test), number of pages visited in a WBI program, and time spent on such pages. A data mining approach was used to analyze the results by comparing two clustering algorithms (K-Means and Hierarchical) with two different numbers of clusters. Individual differences had a significant impact on learner behavior in our WBI program. Additionally, we found that the relationship between attributes that measure performance played an influential role in exploring performance level; the relationship between such attributes induced rules in measuring level of a learners' performance

Investigation of the use of navigation tools in web-based learning: A data mining approach

Web-based learning is widespread in educational settings. The popularity of Web-based learning is in great measure because of its flexibility. Multiple navigation tools provided some of this flexibility. Different navigation tools offer different functions. Therefore, it is important to understand how the navigation tools are used by learners with different backgrounds, knowledge, and skills. This article presents two empirical studies in which data-mining approaches were used to analyze learners' navigation behavior. The results indicate that prior knowledge and subject content are two potential factors influencing the use of navigation tools. In addition, the lack of appropriate use of navigation tools may adversely influence learning performance. The results have been integrated into a model that can help designers develop Web-based learning programs and other Web-based applications that can be tailored to learners' needs

Sequence Modelling For Analysing Student Interaction with Educational Systems

The analysis of log data generated by online educational systems is an important task for improving the systems, and furthering our knowledge of how students learn. This paper uses previously unseen log data from Edulab, the largest provider of digital learning for mathematics in Denmark, to analyse the sessions of its users, where 1.08 million student sessions are extracted from a subset of their data. We propose to model students as a distribution of different underlying student behaviours, where the sequence of actions from each session belongs to an underlying student behaviour. We model student behaviour as Markov chains, such that a student is modelled as a distribution of Markov chains, which are estimated using a modified k-means clustering algorithm. The resulting Markov chains are readily interpretable, and in a qualitative analysis around 125,000 student sessions are identified as exhibiting unproductive student behaviour. Based on our results this student representation is promising, especially for educational systems offering many different learning usages, and offers an alternative to common approaches like modelling student behaviour as a single Markov chain often done in the literature.Comment: The 10th International Conference on Educational Data Mining 201