User-friendly Support for Common Concepts in a Lightweight Verifier

Machine verification of formal arguments can only increase our confidence in the correctness of those arguments, but the costs of employing machine verification still outweigh the benefits for some common kinds of formal reasoning activities. As a result, usability is becoming increasingly important in the design of formal verification tools. We describe the "aartifact" lightweight verification system, designed for processing formal arguments involving basic, ubiquitous mathematical concepts. The system is a prototype for investigating potential techniques for improving the usability of formal verification systems. It leverages techniques drawn both from existing work and from our own efforts. In addition to a parser for a familiar concrete syntax and a mechanism for automated syntax lookup, the system integrates (1) a basic logical inference algorithm, (2) a database of propositions governing common mathematical concepts, and (3) a data structure that computes congruence closures of expressions involving relations found in this database. Together, these components allow the system to better accommodate the expectations of users interested in verifying formal arguments involving algebraic and logical manipulations of numbers, sets, vectors, and related operators and predicates. We demonstrate the reasonable performance of this system on typical formal arguments and briefly discuss how the system's design contributed to its usability in two case studies

Lightweight Formal Verification in Classroom Instruction of Reasoning about Functional Code

In college courses dealing with material that requires mathematical rigor, the adoption of a machine-readable representation for formal arguments can be advantageous. Students can focus on a specific collection of constructs that are represented consistently. Examples and counterexamples can be evaluated. Assignments can be assembled and checked with the help of an automated formal reasoning system. However, usability and accessibility do not have a high priority and are not addressed sufficiently well in the design of many existing machine-readable representations and corresponding formal reasoning systems. In earlier work [Lap09], we attempt to address this broad problem by proposing several specific design criteria organized around the notion of a natural context: the sphere of awareness a working human user maintains of the relevant constructs, arguments, experiences, and background materials necessary to accomplish the task at hand. We report on our attempt to evaluate our proposed design criteria by deploying within the classroom a lightweight formal verification system designed according to these criteria. The lightweight formal verification system was used within the instruction of a common application of formal reasoning: proving by induction formal propositions about functional code. We present all of the formal reasoning examples and assignments considered during this deployment, most of which are drawn directly from an introductory text on functional programming. We demonstrate how the design of the system improves the effectiveness and understandability of the examples, and how it aids in the instruction of basic formal reasoning techniques. We make brief remarks about the practical and administrative implications of the system’s design from the perspectives of the student, the instructor, and the grader

A User-friendly Interface for a Lightweight Verification System

User-friendly interfaces can play an important role in bringing the benefits of a machine-readable representation of formal arguments to a wider audience. The "aartifact" system is an easy-to-use lightweight verifier for formal arguments that involve logical and algebraic manipulations of common mathematical concepts. The system provides validation capabilities by utilizing a database of propositions governing common mathematical concepts. The "aartifact" system's multi-faceted interactive user interface combines several approaches to user-friendly interface design: (1) a familiar and natural syntax based on existing conventions in mathematical practice, (2) a real-time keyword-based lookup mechanism for interactive, context-sensitive discovery of the syntactic idioms and semantic concepts found in the system's database of propositions, and (3) immediate validation feedback in the form of reformatted raw input. The system's natural syntax and database of propositions allow it to meet a user's expectations in the formal reasoning scenarios for which it is intended. The real-time keyword-based lookup mechanism and validation feedback allow the system to teach the user about its capabilities and limitations in an immediate, interactive, and context-aware manner

A User-friendly Interface for a Lightweight Verification System

User-friendly interfaces can play an important role in bringing the benefits of a machine-readable representation of formal arguments to a wider audience. The "aartifact" system is an easy-to-use lightweight verifier for formal arguments that involve logical and algebraic manipulations of common mathematical concepts. The system provides validation capabilities by utilizing a database of propositions governing common mathematical concepts. The "aartifact" system's multi-faceted interactive user interface combines several approaches to user-friendly interface design: (1) a familiar and natural syntax based on existing conventions in mathematical practice, (2) a real-time keyword-based lookup mechanism for interactive, context-sensitive discovery of the syntactic idioms and semantic concepts found in the system's database of propositions, and (3) immediate validation feedback in the form of reformatted raw input. The system's natural syntax and database of propositions allow it to meet a user's expectations in the formal reasoning scenarios for which it is intended. The real-time keyword-based lookup mechanism and validation feedback allow the system to teach the user about its capabilities and limitations in an immediate, interactive, and context-aware manner

Ontology Support for a Lightweight Formal Verification System

The usability of verification systems is becoming increasingly important, and the effective integration of ontologies of formal facts (definitions, propositions, and syntactic idioms) into machine verification systems will likely play a role in improving the usability of such systems. The "aartifact" lightweight verification system utilizes an ontology of formal propositions in order to support lightweight verification of formal arguments that involve common mathematical concepts. The ontology is stored within a relational database, and can be assembled and extended using a simple web interface by contributors who are domain experts. The database can be compiled into two separate components of the "aartifact" system: a verifier component that computes congruence closures of expressions containing relations and predicates found in the ontology, and a JavaScript application that interactively presents to users information about the constants, operators, relations, predicates, syntactic constructs, and idioms found in the ontology (and, thus, supported by the verifier). In this way, the database serves to improve both the verification system's capacity to infer implicit applications of logical propositions within a user's formal argument, and to inform users in a context-aware and structured manner of the verification system's capabilities and limitations

Safe Compositional Network Sketches

NetSketch is a tool for the specification of constrained-flow networks (CFNs) and the certification of desirable safety properties imposed thereon, conceived to assist system integrators in modeling and design. It provides compositional analysis capabilities based on a strongly-typed domain-specific language (DSL) for describing and reasoning about CFNs and relevant invariants. Users can model or design individual network components and perform manual or automated whole-system analysis of the properties thereof. Users can also assemble many instances of these components into larger networks, relying on NetSketch's less precise but more tractable compositional analysis capabilities. This ability to trade "precision of analysis" for "feasibility of analysis" according to available resources is among the novel features of NetSketch. In earlier work we illustrated how NetSketch is applied to actual domains, and provided a formal definition of its underlying formalism. While the NetSketch DSL provides automatic compositional analysis capabilities for modeling and designing entire networks, users may need to employ a wider variety of tools and techniques when modeling and designing individual network components. These can include common tools for reasoning about systems of constraints of various classes (such as linear constraints, quadratic constraints, and so on), as well as logical systems and ontologies that deal with concepts relevant to the application domain. We integrate the "aartifact" lightweight automated assistant for formal reasoning (which has also been applied in proving the soundness of the NetSketch formalism) as a tool for modeling and designing individual network components. We present several use cases within an example application of the NetSketch DSL to demonstrate how the automated assistant provides NetSketch users with both an interface for reasoning formally about constraints, and a straightforward way to implicitly employ a rich domain-specific ontology of logical propositions. This allows users to verify common properties of constraints and constraint sets, and to reason about constraint relationships using automatically verifiable algebraic manipulations

Seamless composition and integration: a perspective on formal methods research

Formal methods are now a central component of computer-science education and research. However, there will always be advances in mathematical logic -- a.k.a. `formal methods' among computer scientists -- leading to advances in reliable, safe and secure computing. There are many research directions that will promote the impact of formal methods on computer science in significant and novel ways. We outline two directions, each associated with its own research challenges, that are complementary to the current state-of-the-art: one of composability and one of integration, each considered in a specific context drawn from our own recent research and teaching experience. We try to clarify why the study and ultimate resolution of these two challenges hold the promise of important breakthroughs in the accessability of formal methods and, ultimately, their applicability.National Science Foundation (CCF-0820138

Students´ language in computer-assisted tutoring of mathematical proofs

Truth and proof are central to mathematics. Proving (or disproving) seemingly simple statements often turns out to be one of the hardest mathematical tasks. Yet, doing proofs is rarely taught in the classroom. Studies on cognitive difficulties in learning to do proofs have shown that pupils and students not only often do not understand or cannot apply basic formal reasoning techniques and do not know how to use formal mathematical language, but, at a far more fundamental level, they also do not understand what it means to prove a statement or even do not see the purpose of proof at all. Since insight into the importance of proof and doing proofs as such cannot be learnt other than by practice, learning support through individualised tutoring is in demand. This volume presents a part of an interdisciplinary project, set at the intersection of pedagogical science, artificial intelligence, and (computational) linguistics, which investigated issues involved in provisioning computer-based tutoring of mathematical proofs through dialogue in natural language. The ultimate goal in this context, addressing the above-mentioned need for learning support, is to build intelligent automated tutoring systems for mathematical proofs. The research presented here has been focused on the language that students use while interacting with such a system: its linguistic propeties and computational modelling. Contribution is made at three levels: first, an analysis of language phenomena found in students´ input to a (simulated) proof tutoring system is conducted and the variety of students´ verbalisations is quantitatively assessed, second, a general computational processing strategy for informal mathematical language and methods of modelling prominent language phenomena are proposed, and third, the prospects for natural language as an input modality for proof tutoring systems is evaluated based on collected corpora

Assertion level proof planning with compiled strategies

This book presents new techniques that allow the automatic verification and generation of abstract human-style proofs. The core of this approach builds an efficient calculus that works directly by applying definitions, theorems, and axioms, which reduces the size of the underlying proof object by a factor of ten. The calculus is extended by the deep inference paradigm which allows the application of inference rules at arbitrary depth inside logical expressions and provides new proofs that are exponentially shorter and not available in the sequent calculus without cut. In addition, a strategy language for abstract underspecified declarative proof patterns is developed. Together, the complementary methods provide a framework to automate declarative proofs. The benefits of the techniques are illustrated by practical applications.Die vorliegende Arbeit beschäftigt sich damit, das Formalisieren von Beweisen zu vereinfachen, indem Methoden entwickelt werden, um informale Beweise formal zu verifizieren und erzeugen zu können. Dazu wird ein abstrakter Kalkül entwickelt, der direkt auf der Faktenebene arbeitet, welche von Menschen geführten Beweisen relativ nahe kommt. Anhand einer Fallstudie wird gezeigt, dass die abstrakte Beweisführung auf der Fakteneben vorteilhaft für automatische Suchverfahren ist. Zusätzlich wird eine Strategiesprache entwickelt, die es erlaubt, unterspezifizierte Beweismuster innerhalb des Beweisdokumentes zu spezifizieren und Beweisskizzen automatisch zu verfeinern. Fallstudien zeigen, dass komplexe Beweismuster kompakt in der entwickelten Strategiesprache spezifiziert werden können. Zusammen bilden die einander ergänzenden Methoden den Rahmen zur Automatisierung von deklarativen Beweisen auf der Faktenebene, die bisher überwiegend manuell entwickelt werden mussten

Assertion level proof planning with compiled strategies

This book presents new techniques that allow the automatic verification and generation of abstract human-style proofs. The core of this approach builds an efficient calculus that works directly by applying definitions, theorems, and axioms, which reduces the size of the underlying proof object by a factor of ten. The calculus is extended by the deep inference paradigm which allows the application of inference rules at arbitrary depth inside logical expressions and provides new proofs that are exponentially shorter and not available in the sequent calculus without cut. In addition, a strategy language for abstract underspecified declarative proof patterns is developed. Together, the complementary methods provide a framework to automate declarative proofs. The benefits of the techniques are illustrated by practical applications.Die vorliegende Arbeit beschäftigt sich damit, das Formalisieren von Beweisen zu vereinfachen, indem Methoden entwickelt werden, um informale Beweise formal zu verifizieren und erzeugen zu können. Dazu wird ein abstrakter Kalkül entwickelt, der direkt auf der Faktenebene arbeitet, welche von Menschen geführten Beweisen relativ nahe kommt. Anhand einer Fallstudie wird gezeigt, dass die abstrakte Beweisführung auf der Fakteneben vorteilhaft für automatische Suchverfahren ist. Zusätzlich wird eine Strategiesprache entwickelt, die es erlaubt, unterspezifizierte Beweismuster innerhalb des Beweisdokumentes zu spezifizieren und Beweisskizzen automatisch zu verfeinern. Fallstudien zeigen, dass komplexe Beweismuster kompakt in der entwickelten Strategiesprache spezifiziert werden können. Zusammen bilden die einander ergänzenden Methoden den Rahmen zur Automatisierung von deklarativen Beweisen auf der Faktenebene, die bisher überwiegend manuell entwickelt werden mussten