Brna, P.

Bundy, Alan

Pain, H.

English

We outline the progress we have made in connection with the Alvey Grant "Evaluating Prolog Environments" (SERCGRJDJ44287and Alvey IKBS 136). This grant runs for three years from 1st November1985. The grant holders are Professor Alan Bundy and Dr Helen Pain and it employs Dr Paul Brna as a Research Fellow

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Evaluating Prolog environmentsCitation for published version:Bundy, A, Brna, P & Pain, H 1988, 'Evaluating Prolog environments' Proceedings of the 1988 AlveyTechnical Conference.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Author final version (often known as postprint)Published In:Proceedings of the 1988 Alvey Technical ConferenceGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 20. Feb. 2015Evaluating Prolog EnvironmentsA Progress ReportAlan Bundy, Helen Pain and Paul BmaDAI Research Paper No. 351December, 1987A short report for the Alvey KBS ClubDepartment of Artificial IntelligenceUniversity of Edinburgh80 South BridgeEdinburgh EHl IHNUnited Kingdom@ Alan Bundy, Helen Pain and Paul BrnaAlan Bundy, Helen Pain and Paul BrnaDepartment of Artificial IntelligenceUniversity of EdinburghDecember 23, 1987AbstractWe outline the progress we have made in connection with the Alvey Grant "Eval-uating Prolog Environments" (SERC GRJDJ44287 and Alvey IKBS 136). This grantruns for three years from 1st November 1985. The grant holders are Professor AlanBundy and Dr Helen Pain and it employs Dr Paul Brna as a Research Fellow.Background1At the start of the grant period, the Alvey Logic Programming Initiative chose to promoteexpertise in Prolog. It was felt that Prolog, being a relatively young computing language,had not accumulated the kind of interactive development environment that was commonlyfound, for instance, in Lisp systems. The provision of such environments was widely seenas essential if the full potential of Prolog was to be realised.As part of the development of new, sophisticated Prolog environments it is vital to evalu-ate tools, to identify problems with them, and to draw attention to omissions and needs.In particular, it is known that many tools do not meet user's needs for debugging largeprograms. Empirical study is seen as vital to discover what kind of bugs occur, how theyare currently fixed, and what tools could improve user productivity.Goals2Our main goal is to inform and influence long term activities in the UK Prolog communityby the evaluation of extant tools and the identification of problems with them. We alsoset out to draw attention to omissions and needs.In particular, our concerns include:1. Do the user interfaces present consistent stories about how Prolog programs arerun and are these stories consistent with available teaching materials?2Evaluating Prolog Environments -A Progress Report2. Are these stories consistent with the Logic Programming phil~phy of Prolog? Isit p~ible to maintain an entirely declarative story for the programmer?3. To what extent do the debugging tools, error messages, etc., help users avoid orcope with the kind of bugs which arise in practice? Do there remain commonlyoccurring bugs for which existing tools are inadequate?4. Do the tools help eliminate misconceptions about the Prolog virtual machine?We had paid attention to the problem of presenting consistent stories (concerning items 1and 2 above) in previous papers [Bundy 84, Pain &. Bundy 87, Bundy et al 85]. This in-cludes an examination of the early Prolog teaching materials to extract the Prolog storiesthey used and the development of a complete story that covers all the aspects of Prolog ina uniform and coherent manner [Bundy et al 85]. This has had some influence on teachingmaterials (e.g. 2nd edition of [Clocksin &. Mellish 84] and [Eisenstadt &. Brayshaw 87])and tracers (e.g.[Eisenstadt &. Brayshaw 86]). Our recent work has mainly focussed onthe issues of bug-avoidance and bug-elimination (item 3 above).In order to confront these issues we set out to evaluate a number of Prolog environmentswith a view to producing the following:.Some classifications of Prolog programming errors (bugs) and lists of commonlyoccurring Prolog bugs..An account of bug-avoiding and debugging strategies used on these bugs..An account of how various environmental tools are used in these strategies..The identification of gaps, i.e. bugs for which no efficient strategies exist and notools are useful..Suggestions about new environmental tools to plug these gaps..Suggestions about language improvements/extensions that would avoid certain bugs..Suggestions about teaching methods that would enable students to avoid certainbugs..An evaluation of how available environments score on the above, e.g. identificationof those offering novel, useful tools.3AchievementsAt this point we have made some progress on all these goals -although much still needsto be done. We have set up a framework for the description of Prolog bugs. This is fun-damental as we want both to motivate the construction of new and improved tools basedon an analysis of bugs (and program debugging) and to evaluate Prolog environmentswith respect to the practical requirements of programmers.3Evaluating Prolog Environments -A Progress ReportThis framework -see figure 1- can be viewed in two distinct ways. The first way(program generation), as a means of describing stages through which a programmer goesfrom the start of coding to the moment that the programmer realises that the programis 'buggy'. The other way (bug detection), as the path by which the programmer mighttrack down the essential cause of the problem.We now briefly 8ummarise the different layers of description. More detail can be foundin [Brna et al 87].Symptom Description If a programmer believes that something has gone wrong dur-ing the execution of some Prolog program then there are a limited number of waysof directly describing such evidence ~.g..(apparent) non-termination.exit with operating system error.generation of Prolog error message.unexpected generation or failure to generate a side-effect.unexpected "no" or "yes".wrong binding of answer variableProgram Misbehaviour Description The explanation offered for a symptom. Thelanguage used is in terms of the flow of control and relates, therefore, to run-timebehaviour --e.g..the unexpected failure of some clause.the unwanted success of some clause -e.g. with a cut then causing a subse-quent clause not to be called.a missing clause.etc.Program Code Error Description The explanation offered in terms of the code itself.Such a description may suggest what fix might cure the program misbehaviour -e.g..there is a missing base case.there is a clause that should have been deleted.a test is needed to detect an unusual caseUnderlying Misconception Description The fundamental misunderstandings and falsebeliefs that the programmer may have to overcome in order to come to terms withspecific features of the language -e.g..recursion is really iteration.if a subgoal fails then the goal failsEvaluating Prolog Environments -A Progress Report 4Program GenerationThat Prolog always tries touse facts before rules in theresolution processUnderlying Misconceptionfactori~(I.in.):-1..1 i. 1-1.factori~(...I in.).in. i. I*...in..factori~(O.l). Program Code Errorfactorial(1.11)factorial(O, 12)factorial(-1,13) Program MisbehaviourLocal Stack Overflow SymptomBug DetectionFigure 1: Four Levels of Description of Prolog Bugs5Evaluating Prolog Environments -A Progress Report.isle is really assignmentTo illustrate that Prolog bugs can be classified at a number of levels consider the symptomthat a query seems to be taking a long time. The immediate cause of this might be thatthe program may be in an infinite loop -described at the level of program misbehaviour.This looping might be because of a programming error -for example, that the body of aclause contains a literal identical to the head. And this might be because the programmerhas an underlying misconception about recursion. This example illustrates the four levels:symptom, program misbehaviour, program code error and underlying misconception.We have examined the Symptom Description and Program Misbehaviour Description lev-els in some detail. The Program Code Error Description level is interesting because itadmits of at least two quite different approaches: a syntactic classification and a technique-based one.An example of such a programming technique is a failure driven loop.PREDICATE(.~.Xl,...X2,...):-NON .DETERMINISTICYREDICATE( ...Xl,...X2,...),SIDE.EFFECT .YREDICATE(...Xl,...X2,...),fail.PREDICATE(...Xl,...X2,...In the above schema, ellipsis is used to indicate missing sub-goals, arguments. Observethat the schema is not sufficient to capture the full range of clauses that exhibit theproperties of a failure driven loop. For example, the side-effecting predicate may notrequire the same list of arguments as the non-deterministic one.A necessary requirement for a failure driven loop is that it has a clause that always fails.It also should contain at least one non-deterministic literal and one that side-effects. Notethat this definition includes some procedural semantics: fail, side-effect, etc. There couldbe other clauses and literals. The non-deterministic and side-effecting one could be thesame or different.The various ways in which a program can violate the definition of a failure driven loopconstitute a technique-oriented bug classification, include:.No failing clause.No side-effecting literal.No non-deterministic literalA catalogue of techniques are being considered in order to build up other technique-oriented bug classifications [Bundy et al 87]. In particular, the techniques of building up6Evaluating Prolog Environments -A Progress ReportAlgorithm'""'"~,IProgramSpecification/TechniquesFigure 2: Techniques and Software Productionrecursive data structures by pattern matching in the clause head, and building them upin the clause body with an accumulator.We contrast techniques with both the notion of algorithm and that of programmingcliches (see the work on the Programmers' Apprentice [Waters 85]). An algorithm is alanguage independent procedure which meets some specification, e.g. sorts the element ofa list. In contrast, our techniques are language dependent (i.e. Prolog), but specificationindependent -e.g. the same technique might be used in sorting a list or in finding themaximum of two numbers. Furthermore, a technique might apply to only part of acomplete procedure, and many techniques may be combined together in a procedure.Figure 2 illustrates a formalisation of some aspects of the software production process.Algorithms are selected according to the nature of the specifications. Techniques are notdirectly related to the specification but they are related to the algorithms selected.The Underlying Misconception Level description has not been developed to any significantextent.4 Current StatusThe next stage in our work requires the evaluation of current Prolog systems and tools.Systems have been selected for a variety of reasons: because they are influential (e.g.DEC-IO Prolog, MicroProlog) , because they offer novel tools relating to bug avoidance ordebugging (e.g. BIM, ESI Prolog II, NU-Prolog, LPA Sigma Prolog), or because they are7Evaluating Prolog Environments -A Progress Reportreadily widely used (e.g. LPA MacProlog, C-Prolog). Not every important Prolog systemis necessarily included. SICSTUS Prolog, for example, is a significant fast, new Prologsystem but appears to have few, if any, new tools. We hope to provide an evaluationbased on about twenty systems.So far, we have concentrated on producing a systems/tools survey. Following on fromthis, our bug description framework will be used as part of the basis of the evaluation ofcurrent Prolog systems that we are undertaking. Various identified debugging strategieswill also be part of our evaluation framework.We expect that the bug description system, the list of bug-avoidance strategies and theknown debugging strategies will be subject to further revision and development. Thisnecessarily requires that the evaluation evolve. The preliminary set of results, however,are expected by the middle of 1988.References[Brna et al 87J P. Brna, A. Bundy, H. Pain, and L. Lynch. Programmingtools for Prolog environments. In J. Hallam and C. Mellish,editors, Advances in Artificial Intelligence, pages 251-264,Society for the Study of Artificial Intelligence and Simula-tion of Behaviour, John Wiley and Sons, 1987. Previously,DAI Research Paper No 302.[Bundy 84] A. Bundy. What stories should we tell Prolog Students?Working Paper 156, Dept. of Artificial Intelligence, Edin-burgh, 1984.[Bundy et al 85] A. Bundy, H. Pain, P. Brna, and L. Lynch. A ProposedProlog Story. Research Paper 283, Dept. of Artificial Intel-ligence, Edinburgh, 1985.[Bundy et al 87] A. Bundy, P. Brna, B. Smith, T. Dodd, M. Eisenstadt, M.van Sorneren, and C.K. Looi. Prolog programming tech-niques. 1987. Paper produced as part of a workshop onProlog at Abingdon.[Clocksin & Mellish 84] W.F. Clocksin and C.S. Mellish. .Springer Verlag, 1984.[Eisenstadt & Brayshaw 86] M. Eisenstadt and M. Brayshaw. The Transparent PrologMachine TPM. Technical Report 21, Human Cognition Re-search Laboratory, The Open University, 1986.Programming in Prolog[Eisenstadt &; Brayshaw 87] M. Eisenstadt and M. Brayshaw. An Integrated Textbook,Video, and Software Environment for Novice and ExpertProlog Programmers. Technical Report 23, Human Cogni-tion Research Laboratory, The Open University, 1987.8Evaluating Prolog Environments -A Progress Report[Pain &,. Bundy 87] H. Pain and A. Bundy. What stories should we tell novicePROLOG programmers. In Hawley R., editor, Artificial In-telligence Programming Environments, Ellis Horwood, 1987.Also available as DAI research paper 269.[Waters 85] R.C. Waters. KBEmacs: A Step Toward the Program-mer's Apprentice. Technical Report 753, MIT Artificial In-teligence Laboratory, 1985.

An Integrated Textbook, Video, and Software Environment for Novice and Expert Prolog Programmers.

KBEmacs: A Step Toward the Programmer's Apprentice.

Programming tools for Prolog environments.

Prolog programming techniques.

The Transparent Prolog Machine TPM.

What stories should we tell novice PROLOG programmers.

What stories should we tell Prolog Students? Working Paper 156, Dept. of Artificial Intelligence,

Evaluating Prolog environments

Evaluating Prolog environments

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer