ANALYZING AND MODELING STUDENTS¿ BEHAVIORAL DYNAMICS IN CONFIDENCE-BASED ASSESSMENT

Confidence-based assessment is a two-dimensional assessment paradigm which considers the confidence or expectancy level a student has about the answer, to ascertain his/her actual knowledge. Several researchers have discussed the usefulness of this model over the traditional one-dimensional assessment approach, which takes the number of correctly answered questions as a sole parameter to calculate the test scores of a student. Additionally, some educational psychologists and theorists have found that confidence-based assessment has a positive impact on students\u2019 academic performance, knowledge retention, and metacognitive abilities of self-regulation and engagement depicted during a learning process. However, to the best of our knowledge, these findings are not exploited by the educational data mining community, aiming to exploit students (logged) data to investigate their performance and behavioral characteristics in order to enhance their performance outcomes and/or learning experiences. Engagement reflects a student\u2019s active participation in an ongoing task or process, that becomes even more important when students are interacting with a computer-based learning or assessment system. There is some evidence that students\u2019 online engagement (which is estimated through their behaviors while interacting with a learning/assessment environment) is also positively correlated with good performance scores. However, no data mining method to date has measured students engagement behaviors during confidence-based assessment. This Ph.D. research work aimed to identify, analyze, model and predict students\u2019 dynamic behaviors triggered by their progression in a computer-based assessment system, offering confidence-driven questions. The data was collected from two experimental studies conducted with undergraduate students who solved a number of problems during confidence-based assessment. In this thesis, we first addressed the challenge of identifying different parameters representing students\u2019 problem-solving behaviors that are positively correlated with confidence-based assessment. Next, we developed a novel scheme to classify students\u2019 problem-solving activities into engaged or disengaged behaviors using the three previously identified parameters namely: students\u2019 response correctness, confidence level, feedback seeking/no-seeking behavior. Our next challenge was to exploit the students\u2019 interactions recorded at the micro-level, i.e. event by event, by the computer-based assessment tools, to estimate their intended engagement behaviors during the assessment. We also observed that traditional non-mixture, first-order Markov chain is inadequate to capture students\u2019 evolving behaviors revealed from their interactions with a computer-based learning/assessment system. We, therefore, investigated mixture Markov models to map students trails of performed activities. However, the quality of the resultant Markov chains is critically dependent on the initialization of the algorithm, which is usually performed randomly. We proposed a new approach for initializing the Expectation-Maximization algorithm for multivariate categorical data we called K-EM. Our method achieved better prediction accuracy and convergence rate in contrast to two pre-existing algorithms when applied on two real datasets. This doctoral research work contributes to elevate the existing states of the educational research (i.e. theoretical aspect) and the educational data mining area (i.e. empirical aspect). The outcomes of this work pave the way to a framework for an adaptive confidence-based assessment system, contributing to one of the central components of Adaptive Learning, that is, personalized student models. The adaptive system can exploit data generated in a confidence-based assessment system, to model students\u2019 behavioral profiles and provide personalized feedback to improve students\u2019 confidence accuracy and knowledge by considering their behavioral dynamics

A Human-Centric System for Symbolic Reasoning About Code

While testing and tracing on specific input values are useful starting points for students to understand program behavior, ultimately students need to be able to reason rigorously and logically about the correctness of their code on all inputs without having to run the code. Symbolic reasoning is reasoning abstractly about code using arbitrary symbolic input values, as opposed to specific concrete inputs. The overarching goal of this research is to help students learn symbolic reasoning, beginning with code containing simple assertions as a foundation and proceeding to code involving data abstractions and loop invariants. Toward achieving this goal, this research has employed multiple experiments across five years at three institutions: a large, public university, an HBCU (Historically Black Colleges and Universities), and an HSI (Hispanic Serving Institution). A total of 862 students participated across all variations of the study. Interactive, online tools can enhance student learning because they can provide targeted help that would be prohibitively expensive without automation. The research experiments employ two such symbolic reasoning tools that had been developed earlier and a newly designed human-centric reasoning system (HCRS). The HCRS is a first step in building a generalized tutor that achieves a level of resolution necessary to identify difficulties and suggest appropriate interventions. The experiments show the value of tools in pinpointing and classifying difficulties in learning symbolic reasoning, as well as in learning design-by-contract assertions and applying them to develop loop invariants for code involving objects. Statistically significant results include the following. Students are able to learn symbolic reasoning with the aid of instruction and an online tool. Motivation improves student perception and attitude towards symbolic reasoning. Tool usage improves student performance on symbolic reasoning, their explanations of the larger purpose of code segments, and self-efficacy for all subpopulations

Analysis of Student Behavior and Score Prediction in Assistments Online Learning

Understanding and analyzing student behavior is paramount in enhancing online learning, and this thesis delves into the subject by presenting an in-depth analysis of student behavior and score prediction in the ASSISTments online learning platform. We used data from the EDM Cup 2023 Kaggle Competition to answer four key questions. First, we explored how students seeking hints and explanations affect their performance in assignments, shedding light on the role of guidance in learning. Second, we looked at the connection between students mastering specific skills and their performance in related assignments, giving insights into the effectiveness of curriculum alignment. Third, we identified important features from student activity data to improve grade prediction, helping identify at-risk students early and monitor their progress. Lastly, we used graph representation learning to understand complex relationships in the data, leading to more accurate predictive models. This research enhances our understanding of data mining in online learning, with implications for personalized learning and support mechanisms

From Social Data Mining to Forecasting Socio-Economic Crisis

Socio-economic data mining has a great potential in terms of gaining a better understanding of problems that our economy and society are facing, such as financial instability, shortages of resources, or conflicts. Without large-scale data mining, progress in these areas seems hard or impossible. Therefore, a suitable, distributed data mining infrastructure and research centers should be built in Europe. It also appears appropriate to build a network of Crisis Observatories. They can be imagined as laboratories devoted to the gathering and processing of enormous volumes of data on both natural systems such as the Earth and its ecosystem, as well as on human techno-socio-economic systems, so as to gain early warnings of impending events. Reality mining provides the chance to adapt more quickly and more accurately to changing situations. Further opportunities arise by individually customized services, which however should be provided in a privacy-respecting way. This requires the development of novel ICT (such as a self- organizing Web), but most likely new legal regulations and suitable institutions as well. As long as such regulations are lacking on a world-wide scale, it is in the public interest that scientists explore what can be done with the huge data available. Big data do have the potential to change or even threaten democratic societies. The same applies to sudden and large-scale failures of ICT systems. Therefore, dealing with data must be done with a large degree of responsibility and care. Self-interests of individuals, companies or institutions have limits, where the public interest is affected, and public interest is not a sufficient justification to violate human rights of individuals. Privacy is a high good, as confidentiality is, and damaging it would have serious side effects for society.Comment: 65 pages, 1 figure, Visioneer White Paper, see http://www.visioneer.ethz.c

Jazz Improvisation and the Law: Constrained Choice, Sequence, and Strategic Movement Within Rules

This Article argues that a richer understanding of the nature of law is possible through comparative, analogical examination of legal work and the art of jazz improvisation. This exploration illuminates a middle ground between rule of law aspirations emphasizing stability and determinate meanings and contrasting claims that the untenable alternative is pervasive discretionary or politicized law. In both the law and jazz improvisation settings, the work involves constraining rules, others’ unpredictable actions, and strategic choosing with attention to where a collective creation is going. One expects change and creativity in improvisation, but the many analogous characteristics of law illuminate why change and choice are the norm in law too. Rarely is law just about ferreting out some isolated, clear, but abstruse legal command. In jazz and legal settings, relative assessments of strength are more commonly apt than are expectations of a single correct answer or simple binary right-versus-wrong determinations. There is a world of difference between claims that law simply provides determinate answers, versus claims that law constrains and guides what remain choices. Much as jazz improvisers must be highly sensitive to the surrounding constrained choices of others, legal analysis of context and consequences of legal choices, with substantial attention to others’ roles and competence, should always be part of legal actions. This different way of thinking about law’s nature helps illuminate and critique both major methodological legal divides, enduring jurisprudential debates, and several cutting-edge case studies. Those case studies include standing law’s transformation, including the 2021 TransUnion standing decision, ongoing battles over what waters are protected by the Clean Water Act, debates over textualist methodology’s claims of constraint, and increasing judicial reliance on the “major questions doctrine” with shifts away from the familiar deferential Chevron framework. Improvising musicians must ensure their choices musically fit with governing forms, practices, and others’ choices. Similarly, the Article closes by illuminating why, to further rule of law values and check power abuses, legal actors should always assess the consequential congruence of their tenable choices with surrounding law, giving substantial weight to statutory policies and linked effects analysis by agencies

Building Intelligent Tutoring Systems

This project\u27s goal was to improve the ASSISTments intelligent tutoring system\u27s algebraic capabilities. We worked towards three main objectives. First, we built support for parsing expressions and comparing them for algebraic equality. Second, we implemented an interactive grapher capable of plotting a variety of expressions. Third, we added support for rendering expressions to well formatted images. Finally, we implemented a basic tutoring system including sample problems that demonstrate our work, establishing our tools\u27 usability and integrability

Development and Application of a Rasch Model Measure of Student Competency in University Introductory Computer Programming

University computer programming instruction nomenclature commonly uses the term Computer Science 1 (CS1) to describe introductory units of study. Success in CS1 is important as a pre-requisite for further study in programming and related disciplines. It is important to measure student progress and the antecedent influences. This study applied the Rasch Model and Messick’s Unified Theory of Validity to construct an interval level measure of CS1 competency with demonstrable suitability for this purpose

Characterizing Productive Perseverance Using Sensor-Free Detectors of Student Knowledge, Behavior, and Affect

Failure is a necessary step in the process of learning. For this reason, there has been a myriad of research dedicated to the study of student perseverance in the presence of failure, leading to several commonly-cited theories and frameworks to characterize productive and unproductive representations of the construct of persistence. While researchers are in agreement that it is important for students to persist when struggling to learn new material, there can be both positive and negative aspects of persistence. What is it, then, that separates productive from unproductive persistence? The purpose of this work is to address this question through the development, extension, and study of data-driven models of student affect, behavior, and knowledge. The increased adoption of computer-based learning platforms in real classrooms has led to unique opportunities to study student learning at both fine levels of granularity and longitudinally at scale. Prior work has leveraged machine learning methods, existing learning theory, and previous education research to explore various aspects of student learning. These include the development of sensor-free detectors that utilize only the student interaction data collected through such learning platforms. Building off of the considerable amount of prior research, this work employs state-of-the-art machine learning methods in conjunction with the large scale granular data collected by computer-based learning platforms in alignment with three goals. First, this work focuses on the development of student models that study learning through the use of advancements in student modeling and deep learning methodologies. Second, this dissertation explores the development of tools that incorporate such models to support teachers in taking action in real classrooms to promote productive approaches to learning. Finally, this work aims to complete the loop in utilizing these detector models to better understand the underlying constructs that are being measured through their application and their connection to productive perseverance and commonly-observed learning outcomes