How to Put Usability into Focus: Using Focus Groups to Evaluate the Usability of Interactive Theorem Provers

In recent years the effectiveness of interactive theorem provers has increased to an extent that the bottleneck in the interactive process shifted to efficiency: while in principle large and complex theorems are provable (effectiveness), it takes a lot of effort for the user interacting with the system (lack of efficiency). We conducted focus groups to evaluate the usability of Isabelle/HOL and the KeY system with two goals: (a) detect usability issues in the interaction between interactive theorem provers and their user, and (b) analyze how evaluation and survey methods commonly used in the area of human-computer interaction, such as focus groups and co-operative evaluation, are applicable to the specific field of interactive theorem proving (ITP). In this paper, we report on our experience using the evaluation method focus groups and how we adapted this method to ITP. We describe our results and conclusions mainly on the ``meta-level,'' i.e., we focus on the impact that specific characteristics of ITPs have on the setup and the results of focus groups. On the concrete level, we briefly summarise insights into the usability of the ITPs used in our case study

Theoretical models of the role of visualisation in learning formal reasoning

Although there is empirical evidence that visualisation tools can help students to learn formal subjects such as logic, and although particular strategies and conceptual difficulties have been identified, it has so far proved difficult to provide a general model of learning in this context that accounts for these findings in a systematic way. In this paper, four attempts at explaining the relative difficulty of formal concepts and the role of visualisation in this learning process are presented. These explanations draw on several existing theories, including Vygotsky's Zone of Proximal Development, Green's Cognitive Dimensions, the Popper-Campbell model of conjectural learning, and cognitive complexity. The paper concludes with a comparison of the utility and applicability of the different models. It is also accompanied by a reflexive commentary[0] (linked to this paper as a hypertext) that examines the ways in which theory has been used within these arguments, and which attempts to relate these uses to the wider context of learning technology research

Extracting proofs from documents

Often, theorem checkers like PVS are used to check an existing proof, which is part of some document. Since there is a large difference between the notations used in the documents and the notations used in the theorem checkers, it is usually a laborious task to convert an existing proof into a format which can be checked by a machine. In the system that we propose, the author is assisted in the process of converting an existing proof into the PVS language and having it checked by PVS. 1 Introduction The now-classic ALGOL 60 report [5] recognized three different levels of language: a reference language, a publication language and several hardware representations, whereby the publication language was intended to admit variations on the reference language and was to be used for stating and communicating processes. The importance of publication language ---often referred to nowadays as "pseudo-code"--- is difficult to exaggerate since a publication language is the most effective way..

Multiple Viewpoints for Tutoring Systems.

This thesis investigates the issue of how a tutoring system, intelligent or otherwise, may be designed to utilise multiple viewpoints on the domain being tutored, and what benefits may accrue from this. The issue was relevant to earlier systems, such as WHY (Stevens et al. 1979) and STEAMER (Hollan et al. 1984). The relevant literature is reviewed, and criteria which must be met by our implementation of viewpoints are established. Viewpoints are conceptualised as pre-defined structures which can be represented in a tutoring system with the potential to increase its effectiveness and adaptability. A formalism is proposed where inferences are drawn from a model by a range of operators. The application of this combination to problems and goals is to be described heuristically. This formulation is then related to the educational philosophy of Cognitive Apprenticeship. The formalism is tested and refined in a protocol analysis study which leads to the definition of three classes of operators. The viewpoint structure is used to produce a detailed formulation of the domain of Prolog debugging for novices, with the goal that students should learn how different bugs may be localised using different viewpoints. Three models of execution are defined, based on those described by Bundy et al. (1985). These are mapped onto a restricted catalogue of bugs by specifying a number of conventions which produce a simplified and consistent domain suited to the needs of novices. VIPER, a tutoring system which can itself accomplish and explain the relevant domain tasks, is described. VIPER is based on a meta-interpreter which produces detailed execution histories which are then analysed. An evaluation of VIPER is reported, with generally favourable results. VIPER is discussed in relation to the research goals, the usefulness of Cognitive Apprenticeship in supporting such a design, and possible future work. This discussion exemplifies the use of established student modeling techniques, the implementation of other viewpoints on Prolog, and the application of the design strategy to other domains. Claims are made in relation to the formulation of viewpoints, the architecture of VIPER, and the relevance of Cognitive Apprenticeship to the use of multiple viewpoints

Automated Deduction – CADE 28

This open access book constitutes the proceeding of the 28th International Conference on Automated Deduction, CADE 28, held virtually in July 2021. The 29 full papers and 7 system descriptions presented together with 2 invited papers were carefully reviewed and selected from 76 submissions. CADE is the major forum for the presentation of research in all aspects of automated deduction, including foundations, applications, implementations, and practical experience. The papers are organized in the following topics: Logical foundations; theory and principles; implementation and application; ATP and AI; and system descriptions

Mathematical Explanation and Ontology: An Analysis of Applied Mathematics and Mathematical Proofs

The present work aims at providing an account of mathematical explanation in two different areas: scientific explanation and within mathematics. The research is addressed from two different perspectives: the one arising from an ontological concern about mathematical entities, and the other originating from a methodological choice: to study our chosen problems (mathematical explanation in science and in mathematics itself) in mathematical practice, that is to say, looking at the way mathematicians understand and perform their work in these diverse areas, including a case study for the context of intra-mathematical explanation. The central target is the analysis of the role that mathematical explanation plays in science and its relevance to the success or failure of scientific theories. The ontological question of whether the explanatory role of abstract objects, mathematical objects in particular, is enough to postulate their existence will be one of the issues to be addressed. Moreover, the possibility of a unified theory of explanation which can accommodate both external and internal mathematical explanation will also be considered. In order to go deeper into these issues, the research includes: (1) an analysis how the question of what is involved in internal mathematical explanation has been addressed in the literature, an analysis of the role of mathematical proof and the reasons why it makes sense to search for more explanatory proofs of already known results, and (2) an analysis of the relation between the use of mathematics in scientific explanation and the ontological commitment that arises from these explanatory tools in science. Part of the present work consists of an analysis of the explanatory role of mathematics through the study of cases reflecting this role. Case studies is one of the main sources of data in order to clarify the role mathematical entities play, among other methodological resources