Hank is a visual programming language devised specically for the use of cognitive psychologists rather than computer programmers. This paper introduces an eighteen month evaluation project on the use of Hank. This project began on November the 1st 1998, the initial findings and planned

evaluation programme will be presented at the workshop in January

Collins, Trevor

Fung, Pat

English

Open Research Online 

Open Research OnlineThe Open University’s repository of research publicationsand other research outputsEvaluating Hank, a cognitive modelling language forpsychologistsConference or Workshop ItemHow to cite:Collins, Trevor and Fung, Pat (1999). Evaluating Hank, a cognitive modelling language for psychologists. In:The Eleventh Annual Workshop of the Psychology of Programming Interest Group (PPIG 11).For guidance on citations see FAQs.c© 1999 The AuthorsVersion: Version of RecordLink(s) to article on publisher’s website:http://ppig.org/papers/11th-collins.pdfCopyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.uk1. Introductionmotherperson1 person2t.d.collins/p.fung @open.ac.ukEvaluating Hank, a Cognitive ModellingLanguage for PsychologistsTrevor Collins and Pat FungInstitute of Educational TechnologyThe Open UniversityMilton Keynes, UKemail:Hank is a visual programming language devised specically for the use of cognitive psychologistsrather than computer programmers. This paper introduces an eighteen month evaluation projecton the use of Hank. This project began on November the 1st 1998, the initial ndings and plannedevaluation programme will be presented at the workshop in January.Hank is a visual cognitive modelling language that was devised by Paul Mulholland and StuartWatt at The Open University's Knowledge Media Institute and rst presented at last year's PPIG[Mulholland and Watt, 1998]. The design of Hank was fueled by the experiences Mulholland andWatt gained through teaching Open University (\OU") psychology students how to build cognitivemodels using the Prolog programming language as well as from their work on source-level steppers[Watt, 1994] and tracing tools [Mulholland, 1995].In order to produce a suitable cognitive modelling language for OU psychology students, Mul-holland and Watt identied four essential user requirements;1. usable by non-programmers,2. usable in groups,3. usable on paper and on computer, and4. capable of showing the processes involved in execution.In response to these requirements Hank was designed as a visual programming language utilizingthe principles of end-user programming, programming by demonstration and software visualization.An example screen image is shown here in Figure 1.In Hank declarative knowledge is expressed in a table known as a \Fact Card." The fact card'stitle indicates the relation being expressed and its column labels indicate the objects being referredto. The individual rows in a fact card indicate the individual items of knowledge. For example,a fact card can be used to express the relation \mother" by using the word as the title ofthe fact card and the words and as the table's column headings (see Figure 1 topwindow, top right table).Knowledge is queried using \Questions" (Figure 1 top window, middle right dashed table). Ques-tions also follow the same table format as fact cards; the title and column labels indicate the relationand objects, and the items in a single data row indicate the items to be found by the question (i.e.Figure 1?someone?: An example screen image taken from Hank. The program window (top) illustratesthe use of \Fact Cards" for describing declarative knowledge (top right), \Questions" for queryingknowledge (middle right), and \Instruction Cards" for describing procedural knowledge (left). Thecontrol panel (middle) allows the user to ask questions using either the \Ask Briey" or \Ask in Full"options, in both cases the control panel also displays the results. The workspace window (bottom)shows the program's execution when a question is asked using the \Ask in Full" option.the symbols to be matched). Variables can also be used in questions in order to identify unknownvalues, variables are dierentiated in Hank by bracketing the variable label with question marks(e.g. ).Thirdly, procedural knowledge is expressed using \Instruction Cards" (Figure 1 top window, lefttable). An instruction card contains a ow chart of questions which are interconnected using either\OK" or \Fail" links. OK links are followed when an item of knowledge has been matched eitherwith an item in a fact card or through the application of an instruction card, Fail links are followedwhen such a match has not been found. An instruction card also includes a title and a numberof column labels, similar to those used in fact cards and questions, to identify the process beingmodelled and the objects used in the process (ie. the \inputs" and \outputs").The above three constructs provide an explicit representation of declarative and proceduralknowledge, as well as a means for querying that knowledge, which can be used either on paper or oncomputer. These form the basic syntax of the Hank language. However, in addition to questions,fact cards and instruction cards, Hank also includes a gallery of predened functional instruction2. Survey Methods3. Current Statuscognitive walkthroughcognitive dimensionseld studiesHank:SayHank:SubtractHank:Samehttp://kmi.open.ac.uk/people/paulm/Hankpage.htmlcards which can also be queried. For example, the primitive is equivalent to a printstatement in C, the primitive is equivalent to the subtract function (i.e. -), and theprimitive is equivalent to the equals comparator (i.e. =).In order to show the processes involved in execution, a \storyboard" can be used in Hank toshow the current focus and the instantiated variables at each step of a program's execution (i.e. theprogram's execution story). These storyboards can either be drawn by hand on paper or by thecomputer on the computer screen. The computer-based version of Hank currently displays thesestoryboards in a workspace view and allows the user to step both forwards and backwards througha program's execution (see Figure 1 bottom window).The above description of Hank is intended to provide some background information on Hank aswell as an introduction to the language itself. Further information on Hank can be found on theHank web site: .The aim of this paper is to introduce a separate project to investigate how well Hank fullls itsdesign requirements and how well those design requirements measure up to their anticipated benetswith regard to support the development of cognitive models by psychologists.Three survey methods are to be used during the course of this evaluation, namely Cogni-tive Walkthroughs; [Polson et al., 1992] and [Lewis and Rieman, 1993], Cognitive Dimensions;[Green, 1990] and [Green and Petre, 1996], and Field Studies, e.g. [Nardi and Miller, 1991] and[Johnson and Nardi, 1996].The approach is a task centered approach used to predict user diculties.The complete task to be carried out by the user is analysed step by step, in an attempt to identifyany possible sources of error in the interface and the supporting materials, such as ambiguities,inconsistencies, and insucient or poorly structured guidance.The framework is a broad brush evaluation technique in which a num-ber of dimensions are identied to investigate the cognitive load associated with a specic task.[Green and Petre, 1996] proposed a set of thirteen cognitive dimensions for evaluating visual pro-gramming environments. Namely; the abstraction gradient, closeness of mapping, consistency, dif-fuseness, error-proneness, hard mental operations, hidden dependencies, premature commitment,progressive evaluation, role-expressiveness, secondary notation, viscosity and visibility. It is intendedthat a similar set of dimensions will be used to analyse Hank.The to be carried out in this study are to record the experiences of Hank users. Thesestudies will focus upon the experiences of OU psychology students using Hank to build cognitivemodels whilst attending a one week residential school. Prelimnary evaluations will start in the newyear with studies of how the OU introduce Hank to their associate lectures, how those associatelecturers introduce Hank to the students in their tutorials, and how the students get on with apaper-based version of Hank prior to attending the residential school.The initial studies carried out during the last month have involved cognitive walkthrough evaluations.These have focused primarily upon the educational use of Hank for cognitive psychology studentsstudying with The Open University, examining;The materials to be used for introducing Hank and for teaching cognitive modelling to distancelearning students with a paper-based version of Hank.Possible solutions to the group modelling projects carried out by OU students with a computer-based version of Hank while attending residential school.4. Scheduled WorkAcknowledgmentsReferencesHuamn-Computer Interaction - INTERACT '90Journal of Visual Languages and ComputingACM Transactions onComputer-Human InteractionTask-Centered User Interface DesignA framework for describing and evaluating SoftwareVisualization Systems: A case-study in PrologThe Proceedings of the 10th Workshop of the Psychology ofProgramming Interest GroupThe interface design and structure of the computer-based version of Hank.The ndings of these three studies will be presented at the conference.As stated in Section 3 the cognitive walkthroughs carried out so far are only one part of a multi-method evaluation approach. The second phase of the project will be to examine the cognitivedimensions associated with the use of Hank. A comparison between the issues highlighted by thecognitive walkthrough and cognitive dimension approaches will then be made.The third phase of the project will involve a series of eld studies carried out at The OpenUniversity's residential schools for cognitive psychology students. These studies will examine theactual use of Hank in an educational setting, in order to identify Hank's pedagogical benets as wellas the value of the cognitive walkthrough and cognitive dimensions analyses.The concluding phase of this project will focus upon the underlying design rationale used toproduce Hank. As detailed in Section 1, Hank started with a clear set of design requirements fromwhich a principled design was produced, as a result the ndings of the above evaluations can beattributed to the design features included in Hank, and the design requirements upon which theywere based. It is hoped that by explicitly identifying the contributions to usability made by eachaspect of the Hank design, the design rationale may be reused and the lessons learnt in this projectwill provide some guidance to the design of future modelling environments.The authors would like to thank Clayton Lewis from The University of Colorado for providing somuch of his time to help with the walkthrough evaluations described in this abstract. During arecent three month sabbatical at the OU's Knowledge Media Institute Clayton took an active rolein this project. His enthusiastic support was greatly appreciated. We would also like to thank theResearch Committee of the Institute of Educational Technology for supporting this research project.[Green, 1990] Green, T. (1990). The cognitive dimension of viscosity: a sticky problem for HCI. In, pages 79{86. Elsevier Science Publishers.[Green and Petre, 1996] Green, T. and Petre, M. (1996). Usability analysis of visual programmingenvironments: a 'cognitive dimensions' framework. ,7(2):131{174.[Johnson and Nardi, 1996] Johnson, J. and Nardi, B. (1996). Creating presentation slides: A studyof user preferences for task-specic versus generic application software., 3(1):38{65.[Lewis and Rieman, 1993] Lewis, C. and Rieman, J. (1993). .ftp://ftp.cs.colorado.edu/pub/distribs/clewis/HCI-Design-Book/.[Mulholland, 1995] Mulholland, P. (1995).. PhD thesis, Knowledge Media Institute, TheOpen University, UK.[Mulholland and Watt, 1998] Mulholland, P. and Watt, S. (1998). Hank: A cognitive modellinglanguage for psychology students. In.International Journal of Man-MachineStudiesInternational Journal ofMan-Machine Studies[Nardi and Miller, 1991] Nardi, B. and Miller, J. (1991). Twinkling lights and nested loops: Dis-tributed problem solving and spreadsheet development., 34:161{184.[Polson et al., 1992] Polson, P., Lewis, C., Rieman, J., and Wharton, C. (1992). Cognitive walk-throughs: A method for theory-based evaluation of user-interfaces., 36:741{773.[Watt, 1994] Watt, S. (1994). Froglet: a source-level stepper for lisp. Technical Re-port KMI-TR-10, Knowledge Media Institute, The Open University, UK. Available fromhttp://kmi.open.ac.uk/kmi-abstracts/kmi-tr-10-abstract.html.

Evaluating Hank, a cognitive modelling language for psychologists

http://oro.open.ac.uk/23763/1/11th-collins.pdf

Evaluating Hank, a cognitive modelling language for psychologists

Abstract

Similar works

Full text

Available Versions

Open Research Online