Towards an Intelligent Tutor for Mathematical Proofs

Computer-supported learning is an increasingly important form of study since it allows for independent learning and individualized instruction. In this paper, we discuss a novel approach to developing an intelligent tutoring system for teaching textbook-style mathematical proofs. We characterize the particularities of the domain and discuss common ITS design models. Our approach is motivated by phenomena found in a corpus of tutorial dialogs that were collected in a Wizard-of-Oz experiment. We show how an intelligent tutor for textbook-style mathematical proofs can be built on top of an adapted assertion-level proof assistant by reusing representations and proof search strategies originally developed for automated and interactive theorem proving. The resulting prototype was successfully evaluated on a corpus of tutorial dialogs and yields good results.Comment: In Proceedings THedu'11, arXiv:1202.453

Students´ language in computer-assisted tutoring of mathematical proofs

Truth and proof are central to mathematics. Proving (or disproving) seemingly simple statements often turns out to be one of the hardest mathematical tasks. Yet, doing proofs is rarely taught in the classroom. Studies on cognitive difficulties in learning to do proofs have shown that pupils and students not only often do not understand or cannot apply basic formal reasoning techniques and do not know how to use formal mathematical language, but, at a far more fundamental level, they also do not understand what it means to prove a statement or even do not see the purpose of proof at all. Since insight into the importance of proof and doing proofs as such cannot be learnt other than by practice, learning support through individualised tutoring is in demand. This volume presents a part of an interdisciplinary project, set at the intersection of pedagogical science, artificial intelligence, and (computational) linguistics, which investigated issues involved in provisioning computer-based tutoring of mathematical proofs through dialogue in natural language. The ultimate goal in this context, addressing the above-mentioned need for learning support, is to build intelligent automated tutoring systems for mathematical proofs. The research presented here has been focused on the language that students use while interacting with such a system: its linguistic propeties and computational modelling. Contribution is made at three levels: first, an analysis of language phenomena found in students´ input to a (simulated) proof tutoring system is conducted and the variety of students´ verbalisations is quantitatively assessed, second, a general computational processing strategy for informal mathematical language and methods of modelling prominent language phenomena are proposed, and third, the prospects for natural language as an input modality for proof tutoring systems is evaluated based on collected corpora

Computer supported mathematics with Ωmega

AbstractClassical automated theorem proving of today is based on ingenious search techniques to find a proof for a given theorem in very large search spaces—often in the range of several billion clauses. But in spite of many successful attempts to prove even open mathematical problems automatically, their use in everyday mathematical practice is still limited.The shift from search based methods to more abstract planning techniques however opened up a paradigm for mathematical reasoning on a computer and several systems of that kind now employ a mix of interactive, search based as well as proof planning techniques.The Ωmega system is at the core of several related and well-integrated research projects of the Ωmega research group, whose aim is to develop system support for a working mathematician as well as a software engineer when employing formal methods for quality assurance. In particular, Ωmega supports proof development at a human-oriented abstract level of proof granularity. It is a modular system with a central proof data structure and several supplementary subsystems including automated deduction and computer algebra systems. Ωmega has many characteristics in common with systems like NuPrL, CoQ, Hol, Pvs, and Isabelle. However, it differs from these systems with respect to its focus on proof planning and in that respect it is more similar to the proof planning systems Clam and λClam at Edinburgh

An Approach to Assertion Application via Generalised Resolution

In this paper we address assertion retrieval and application in theorem proving systems or proof planning systems for classical first-order logic. Due to Huang the notion of assertion comprises mathematical knowledge such as definitions, theorems, and axioms. We propose a distributed mediator module between a mathematical knowledge base KB and a theorem proving system TP which is independent of the particular proof representation format of TP and which applies generalised resolution in order to analyze the logical consequences of arbitrary assertions for a proof context at hand. Our approach is applicable also to the assumptions which are dynamically created during a proof search process. It therefore realises a crucial first step towards full automation of assertion level reasoning. We discuss the benefits and connection of our approach to proof planning and motivate an application in a project aiming at a tutorial dialogue system for mathematics

MENON : automating a Socratic teaching model for mathematical proofs

This thesis presents an approach to adaptive pedagogical feedback for arbitrary domains as an alternative to resource-intensive pre-compiled feedback, which represents the state-of-the-art in intelligent tutoring systems today. A consequence of automatic adaptive feedback is that the number of tasks with pedagogical feedback that can be offered to the student increases, and with it the opportunity for practice. We focus on automating different aspects of teaching that together are primarily responsible for learning and can be integrated in a unified natural-language output. The automatic production and natural-language generation of feedback enables its personalisation both at the pedagogical and the natural-language dialogue level. We propose a method for automating the production of domain-independent adaptive feedback. The proof- of-concept implementation of the tutorial manager Menon is carried out for the domain of set-theory proofs. More specifically, we define a pedagogical model that abides by schema and cognitive load theory, and by the synergistic approach to learning. We implement this model in a Socratic teaching strategy whose basic units of feedback are dialogue moves. We use empirical data from two domains to derive a taxonomy of tutorial-dialogue moves, and define the most central and sophisticated move hint. The formalisation of the cognitive content of hints is inspired by schema theory and is facilitated by a domain ontology.Die vorliegende Arbeit präsentiert eine Annäherung an adaptives pädagogisches Feedback für beliebige Domäne. Diese Herangehensweise bietet eine Alternative zu ressource-intensivem, vorübersetztem Feedback, dass das heutige "state-of-the-art'; in intelligenten tutoriellen Systemen ist. Als Folge können zahlreiche Aufgaben mit pädagogischem Feedback für die Praxis angeboten werden. Der Schwerpunkt der Arbeit liegt auf der Automatisierung verschiedener Aspekte des Lehrprozesses, die in ihrer Gesamtheit wesentlich den Lernprozess beeinflussen, und in einer einheitlichen Systemausgabe Natürlicher Sprache integriert werden können. Die automatische Produktion und die Systemgenerierung von Feedback in Natürlicher Sprache ermöglichen eine Individualisierung des Feedback auf zwei Ebenen: einer pädagogischen und einer dialogischen Ebene. Dazu schlagen wir eine Methode vor, durch die adaptives Feedback automatisiert werden kann, und implementieren den tutoriellen Manager Menon als "proof-of-concept'; beispielhaft für die Domäne von Beweisen in der Mengentheorie. Konkret definieren wir ein pädagogisches Modell, das sich auf Schema- und Kognitionstheorie sowie auf die synergetische Herangehensweise an Lernen stützt. Dieses Modell wird in einer Sokratischen Lehrmethode implementiert, deren basale Feedback-Elemente aus Dialogakten bestehen. Zur Bestimmung einer Taxonomie Tutorielle-Dialogakte sowie des zentralen und komplexen Dialogakts hint (Hinweis) wenden wir empirische Daten aus zwei Domänen an. Die Formalisierung des kognitiven Inhaltes von Hinweisen folgt der Schematheorie und basiert auf einer Domänenontologie

Assertion level proof planning with compiled strategies

This book presents new techniques that allow the automatic verification and generation of abstract human-style proofs. The core of this approach builds an efficient calculus that works directly by applying definitions, theorems, and axioms, which reduces the size of the underlying proof object by a factor of ten. The calculus is extended by the deep inference paradigm which allows the application of inference rules at arbitrary depth inside logical expressions and provides new proofs that are exponentially shorter and not available in the sequent calculus without cut. In addition, a strategy language for abstract underspecified declarative proof patterns is developed. Together, the complementary methods provide a framework to automate declarative proofs. The benefits of the techniques are illustrated by practical applications.Die vorliegende Arbeit beschäftigt sich damit, das Formalisieren von Beweisen zu vereinfachen, indem Methoden entwickelt werden, um informale Beweise formal zu verifizieren und erzeugen zu können. Dazu wird ein abstrakter Kalkül entwickelt, der direkt auf der Faktenebene arbeitet, welche von Menschen geführten Beweisen relativ nahe kommt. Anhand einer Fallstudie wird gezeigt, dass die abstrakte Beweisführung auf der Fakteneben vorteilhaft für automatische Suchverfahren ist. Zusätzlich wird eine Strategiesprache entwickelt, die es erlaubt, unterspezifizierte Beweismuster innerhalb des Beweisdokumentes zu spezifizieren und Beweisskizzen automatisch zu verfeinern. Fallstudien zeigen, dass komplexe Beweismuster kompakt in der entwickelten Strategiesprache spezifiziert werden können. Zusammen bilden die einander ergänzenden Methoden den Rahmen zur Automatisierung von deklarativen Beweisen auf der Faktenebene, die bisher überwiegend manuell entwickelt werden mussten

Assertion level proof planning with compiled strategies

This book presents new techniques that allow the automatic verification and generation of abstract human-style proofs. The core of this approach builds an efficient calculus that works directly by applying definitions, theorems, and axioms, which reduces the size of the underlying proof object by a factor of ten. The calculus is extended by the deep inference paradigm which allows the application of inference rules at arbitrary depth inside logical expressions and provides new proofs that are exponentially shorter and not available in the sequent calculus without cut. In addition, a strategy language for abstract underspecified declarative proof patterns is developed. Together, the complementary methods provide a framework to automate declarative proofs. The benefits of the techniques are illustrated by practical applications.Die vorliegende Arbeit beschäftigt sich damit, das Formalisieren von Beweisen zu vereinfachen, indem Methoden entwickelt werden, um informale Beweise formal zu verifizieren und erzeugen zu können. Dazu wird ein abstrakter Kalkül entwickelt, der direkt auf der Faktenebene arbeitet, welche von Menschen geführten Beweisen relativ nahe kommt. Anhand einer Fallstudie wird gezeigt, dass die abstrakte Beweisführung auf der Fakteneben vorteilhaft für automatische Suchverfahren ist. Zusätzlich wird eine Strategiesprache entwickelt, die es erlaubt, unterspezifizierte Beweismuster innerhalb des Beweisdokumentes zu spezifizieren und Beweisskizzen automatisch zu verfeinern. Fallstudien zeigen, dass komplexe Beweismuster kompakt in der entwickelten Strategiesprache spezifiziert werden können. Zusammen bilden die einander ergänzenden Methoden den Rahmen zur Automatisierung von deklarativen Beweisen auf der Faktenebene, die bisher überwiegend manuell entwickelt werden mussten

Tools for Tutoring Theoretical Computer Science Topics

This thesis introduces COMPLEXITY TUTOR, a tutoring system to assist in learning abstract proof-based topics, which has been specifically targeted towards the population of computer science students studying theoretical computer science. Existing literature has shown tremendous educational benefits produced by active learning techniques, student-centered pedagogy, gamification and intelligent tutoring systems. However, previously, there had been almost no research on adapting these ideas to the domain of theoretical computer science. As a population, computer science students receive immediate feedback from compilers and debuggers, but receive no similar level of guidance for theoretical coursework. One hypothesis of this thesis is that immediate feedback while working on theoretical problems would be particularly well-received by students, and this hypothesis has been supported by the feedback of students who used the system. This thesis makes several contributions to the field. It provides assistance for teaching proof construction in theoretical computer science. A second contribution is a framework that can be readily adapted to many other domains with abstract mathematical content. Exercises can be constructed in natural language and instructors with limited programming knowledge can quickly develop new subject material for COMPLEXITY TUTOR. A third contribution is a platform for writing algorithms in Python code that has been integrated into this framework, for constructive proofs in computer science. A fourth contribution is development of an interactive environment that uses a novel graphical puzzle-like platform and gamification ideas to teach proof concepts. The learning curve for students is reduced, in comparison to other systems that use a formal language or complex interface. A multi-semester evaluation of 101 computer science students using COMPLEXITY TUTOR was conducted. An additional 98 students participated in the study as part of control groups. COMPLEXITY TUTOR was used to help students learn the topics of NP-completeness in algorithms classes and prepositional logic proofs in discrete math classes. Since this is the first significant study of using a computerized tutoring system in theoretical computer science, results from the study not only provide evidence to support the suitability of using tutoring systems in theoretical computer science, but also provide insights for future research directions

Use of proofs-as-programs to build an anology-based functional program editor

This thesis presents a novel application of the technique known as proofs-as-programs. Proofs-as-programs defines a correspondence between proofs in a constructive logic and functional programs. By using this correspondence, a functional program may be represented directly as the proof of a specification and so the program may be analysed within this proof framework. CʸNTHIA is a program editor for the functional language ML which uses proofs-as-programs to analyse users' programs as they are written. So that the user requires no knowledge of proof theory, the underlying proof representation is completely hidden. The proof framework allows programs written in CʸNTHIA to be checked to be syntactically correct, well-typed, well-defined and terminating. CʸNTHIA also embodies the idea of programming by analogy — rather than starting from scratch, users always begin with an existing function definition. They then apply a sequence of high-level editing commands which transform this starting definition into the one required. These commands preserve correctness and also increase programming efficiency by automating commonly occurring steps. The design and implementation of CʸNTHIA is described and its role as a novice programming environment is investigated. Use by experts is possible but only a sub-set of ML is currently supported. Two major trials of CʸNTHIA have shown that CʸNTHIA is well-suited as a teaching tool. Users of CʸNTHIA make fewer programming errors and the feedback facilities of CʸNTHIA mean that it is easier to track down the source of errors when they do occur