Basics of man-machine communication for the design ofeducational systems : NATO Advanced Study Institute, August16-26, 1993, Eindhoven, The NetherlandsCitation for published version (APA):Janse, M. D., & Kuijck-Aerts, van, I. (Eds.) (1993). Basics of man-machine communication for the design ofeducational systems : NATO Advanced Study Institute, August 16-26, 1993, Eindhoven, The Netherlands.Technische Universiteit Eindhoven, Institute for Perception Research.Document status and date:Published: 01/01/1993Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)Please check the document version of this publication:• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publicationGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverneTake down policyIf you believe that this document breaches copyright please contact us at:openaccess@tue.nlproviding details and we will investigate your claim.Download date: 05. Oct. 2023M124187 • NATO Advanced Study lnstitute Basicsof Man-Machine Communication for the Design of Educational Systems August 16-26, 1993 Eindhoven, The Netherlands Organized by IPO 0 r g a n i z i n g C o m m i t t e e Maddy D. Brouwer-Janse, ASI - Directer, IPO - lnstitute for Perception Research, Eindhoven, Nl Dominiqua Béroule, LIMSI-CNRS, Orsay, F Robbert-Jan Beun, IPO, Eindhoven, Nl Tom Bösser, WWU, Münster, BRD E d i t o r s Maddy D. Brouwer-Janse, IPO, Eindhoven, Nl llse van Kuijck-Aerts, IPO, Eindhoven, Nl Proceedings Volume IPO !. 1.1 i lil ~.1: !.1!\ll!:ill!i 1!11111 '1 .. ''' 'i'' ,, 11· I :i ! lil I ! : :ll!i! ! 11 *9490023* EINDHOVEN >130Zè!30NO 31ld3:lt!3d t!OOI\ 1nn111SNI 33Hl011818 Contents VOLUME 1 Designing intelligent interface tor an advice-giving system Ivan Alexandrov, Marietanna Milanova, Technica! University of Sofia, Buigaria .................. 1 Electronic books and their potential tor interactive learning Philip Barker, University of Teesside, Cleveland, UK . . . . . . . . . . . . . . . . . . . 5 A framewerk tor speech-act generation in cooperative dialogues Robbert-Jan Beun, lnstitute tor Perception Research, Eindhoven, The Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Cooperative problem solving as a basis tor computer assisted learning Sviatoslav Brainov, lnstitute of Mathematics, Sofia, Buigaria . . . . . . . . . . . . . . . . 25 A change of view the parameter projection problem in threedimensional view setting Eric Brok, Peter van Splunder, Open University, Heerlen, The Netherlands .............. 33 Partners dialogue as a method tor non-tormal problem solving lgor Chmyr, lnstitute of Low Temperature Engineering and Energetics, Odessa, Ukraine ............................... _. . 43 The design of interacting agents tor use in interfaces David Connah, lnstitute tor Perception Research, Eindhoven, The Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Referring in a shared workspace Anita H.M. Cremers, lnstitute for Perception Research, Eindhoven, The Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 User responses to eenstraint management in computer-mediated design activities: a conceptual framewerk Donald L. Day, Syracuse University, New Vork, USA .................. 77 A design for accessing multimedia intermation using cohesion Roger Espinosa, Patricia Baggett, Univarsity of Michigan, Ann Arbor, USA . . . . . . . . . . . . . . . . 95 i "Ciass, you're not making enough noise!" ii The case tor sound-effects in educational software Bill Gaver, Rank Xerox Cambridge EuroPARC, Cambridge, UK . . . . . . 109 Prosody in experimental dialogues Ronald Geluykens, Marc Swerts, lnstitute tor Perception Research, Eindhoven, The Nètherlands ................................ 121 The writing instructien script Hebrew (WISH) system Jon W. McKeeby, Yael M. Moses and Rachelle S. Heller, George Washington University, USA . . . . . . . . . . . . . . . . . . 123 The leamer's partner: foreign language leaming and real world encounters Stacie L. Hibino, Univarsity of Michigan, USA ........................ 133 New human-computer interaction techniques Robert J.K. Jacob, Naval Research Laboratory, Washington, O.C., USA . . . . . . 135 Psychological peculiarities of man-machine communication in the instructional systems Yelena Kommisarova, Psychological lnstitute, Kiev, Ukraine, CIS .............. 143 Some benefits and problems of adaptive action-oriented user-interfaces Pertti Saariluoma, Michael Miettinen, Univarsity of Helsinki, Finland ....................... 147 Active knowiedge-based interfaces tor existing systems Peter Mikulecky, Comenius Univarsity Bratislava, Czechoslovakia Intelligent knowledge based systems in intelligence analysis Konrad Morgan, laura Holland a.o., 161 Univarsity of Portsmouth, UK ....................... 163 learning in neural networks Paul Munro, Univarsity of Pittsburgh, USA . . . . . . . . . . . . . . . . . . . . . . . 179 Laws of visual perception and their consequences tor the user interface Floris L. van Nes, lnstitute tor Perception Research, Eindhoven, The Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 Contents VOLUME 2 Interactiva learning and naturallanguage systems M. Offereins, Univarsity of Twente, The Netherlands . . . . . . . . . . . . . . . . 205 The probabilistic retreat from biases: implications for man machine communication? Mike Oaksford, Univarsity of Wales at Banger, UK . . . . . . . . . . . . . . . . . . . 217 An X window based GUl for the dynamic simuiatien of the chemica! recovery cycle of a paper pulp mill F. Pais, B. Gay and A. Portugal, Univarsity of Coimbra, Portugal . . . . . . . . . . . . . . . . . . . . . 233 A method of evaluating the usability of a prototype user interface tor CBT courseware Garry Patterson, The Univarsity of Ulster, Jordanstown, UK . . . . . . . . . . . . . . 247 Modelling the answering partner of the dialogue process Oleg P. Pilipenko, Odessa University, Ukraine, CIS . . . . . . . . . . . . . . . . . . . . . 261 Intelligent assistance for simulator-based training Eva L. Ragnemalm, Univarsity of Linköping, Sweden . . . . . . . . . . . . . . . . . . . . . 267 Dialogue specificatien language and tools for an information system over the telephone Andrés Santos, José Colás, Juan Lestani, Universidad Politécnica Madrid, Spain . . . . . . . . . . . . . . . . . 269 The acquisition of troubleshooting skill lmplications for tools tor learning Alma Schaafstal, Jan Maarten Schraagen, TNO lnstitute for Human Factors, Soesterberg, The Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 A visual knowledge elicitation language and methodology for acquiring non verbal expertise Benjamin Singer, Jean-Luc Soubi, CNRS, Toulouse, France . . . . . . . . . . . . . . . . . . . . . . . . . . 299 iii lV A visual display systems for the teaching of intonation to deaf persons: a preliminary report Gerard W.G. Spaai 1•2 , Esther S. Derksen 2 , Paul A.P. Kaufholz1.2, 11nstitute for Perception Research, Eindhoven, The Netherlands 21nstitute for the Deaf, Sint-Michielsgestel, The Netherlands ................................ 311 Animated icons promote learning of their functions Alp Tiritoglu, James F. Juola, Univarsity of Kansas, USA . . . . . . . . . . . . . . . . . . . . . . . . . 323 Generated naturallanguage in an immersive language learning system Henry Hamburger1, Dan Tufis2 , Raza Hashim1, 1George Mason University, Fairtax, Usa 2Romanian Academy, Bucharest, Romania ............. 341 lmproving functional programming environments for education J. Ángel Velázquez-lturbide, Universidad Politécnica de Madrid, Spain .............. 347 Cognitive Bases for Communication and Learning Don G. Bouwhuis, lnstitute for Perception Research, Eindhoven The Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 Activity theory: its implications for man machine communication Victor Kaptelinin, Russian Academy of Education, Moscow, Russia, CIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 Motion as a variabie of visual communication Thomas L. Harrington1 and Peter Bidyuk2, 1Fast Motion Perception, Reno, USA 2Kiev Polytechnical lnstitute, Kiev, Ukraine, CIS .......... 369 Cognitive load and the acquisition of a problem-solving skill R. Van Hoe, lnstitute for Perception Research, Eindhoven The Netherlands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 ...... Designing Inteli i gent Interface for an Advice-Giving System Ivan A/exandrov, Marlofanna Milanova Technica/ University of Sofia kv. ''Strelbishte" bi. 25, ap. 20, Sofia-1404, Buigaria 1. Introduetion The development of computer techniques nowadays leads to quickly iocrement of computer users. Most of them are oot computer specialists and they don't have time to learn much about computers. That's why the modern computer systems have to take into consideration the features of human mind and to help people to solve their probieros using minimum efforts. Creators of computer systems should also take into consideration that users have different levels of knowiedges about computers, i.e. the systems can be used by beginners, intermediale specialists and experts. 2. Man-machine communication Man-machine communication means the interactive exchange of messages between a user and a computer system in order to achive a eertaio task. A basic principle of man-machine communication consistsof user's independent choice of input and the system's totally deterministic reaction. Dialog can be considered as a problem solving metbod where the user knows the problem and the system is used to solve subproblems [1]. Man-machine communication can be organized in different ways. But they can be classified by 3 levels of abstraction: basic structure, representation and teehoical realization. Basic structure includes fundamental features which characterize the dialog. These are the probieros about the initiator of the dialog, the way the action influence the reaction, the establishment of the task and so on. Representation includes the describtion of messages by which action and reaction are represented (vocabulary, inner and outer format, etc.). Teehoical realization includes the technica( media (input and output facilities) by which the messages are realized. On the basis of the basic structure 6 types of abstract dialog exist: - simpte question. This is a question by the system with predetermined input interpretation. The user cao only enter objects which will be interpreted. This type is mentioned in conneetion with data collection; - proposal for selection. The user chooses from a set of alternative tasks proposed by the system. Two variants of this dialog type are menu and yes/no question; - request with syntax for respond. This means a system request on which the user bas to react with a syntactically restricted input. The question of the system about the data is a request of such type; - request for free response. The systems demands from the user a statement in quasinatural language; - command. The user specifies bis tasks and objects according to a prescribed syntax; - quasi-natural language statement. The user freely chooses a task using a familiar language. This type of dialog imposes at least restrictions on the user. On the basic of the representation the dialogs cao be classified using the following components: vocabulary, inner and outer format of input, formal input redundancy, output syntax and format, semantic properties and others. 3. Describtion of the system Our team have developed a CAM system which cao be conneeled with any other CAD syster:n (for example, AutoCAO by AutoDESK) and which can generate CNC programs for the following types of detail manufacture: 2D and 3D milling, turning, pocketing, drilling and wire EDM machining [2-3]. The flowchart of the system is given in fig.t. The interface between the user and the computer system is organized using some menus. At first the user bas to select the type of operation he wants to use (milling or turning or pocketing or something else). Then he has to choose the tooi and its diameter. The tooi can be selected from a tooi library where tools are presented as icons. The geometry of the detail has already been given in the CAD system so the user bas to teil the CAM system where this geometry is stored. He has also to determine some regimes of manufacture such as spindel, feedrate, etc. After that the system delermines the tooi path and creates CLDAT A file. Using menu, which contains different types of postprocessors, user can select the suitable postprocessor for its CNC machine and as a result CNC program is generated. determing of detail geometry (using a CAD system) selection of manufacture type (milling, drilling, turning, pocketing, wire EDM machining) computer automa-tically determines the tooi sequence and regimes of manufacture user determines the tooi sequence and regimes of rnanuf acture determing of tooi path CNCprogram generation correct i ons Fig. I For some types of manufacture an advice-giving module is developed. This module cao determine the sequence of tools which can be used for manufacture of a given detail and the regimes of processing. These are the cases of drilling (more precisely, manufacture of holes), milling (more precisely, manufacture of channels) and die blanking in order to reduce its width so it becames approximately equal to the detail width. The sequence of tools and regimes of manufacture given by the advice-giving module cao be used directly by users or cao be corrected. This module asks user about the sizes and the locations of the holes and channels he wants to manufacture. In order to determine the tooi sequence and the regimes of manufacture the system uses algorithms which are developed on the basis of the practice of specialists in this field. The following types of man-machine communication are used in the system: - menu. In this case the dialog step consists of the display of section possibilities (action) and the indication of the chosen alternative (reaction). The action has the following basic characteristics: vocabulary (natural language words) and output syntax (quasi-natural language sentences). The reaction has the following characteristics: vocabulary (natural language words), inner format (a sentence out of those displayed in the action) and outer format (free, with an input terminalion symbol at the end); - yes/no question. The characteristics of the action are the same as in the previous point (menu). The characteristics of the reaction are: vocabulary ( {yes,no} ), inner format (a single word is allowed) and outer format (free); - request with syntax for respond. This type is used when user should determine the sizes of holes and channels; - command. This type is used when user should determine the location of the holes and channels. He initiates the dialog between him and the computer system when he tells the system that there is a hole or a channel somewhere. 4. Conclusions A CAM system is developed. This system gives users the possibility to generale CNC programs for detail manufacture. The man-machine communication in this system is developed on the basis of modern techniques. That's why the system is user-friendly so people cao work easily with it. References 1. Dehning W., Essig H., Maass S.: The adaptation of virtual Man-Computer Interfaces to User Requirements in Dialogs, Lecture Notes in Computer Science, 110, 1981. 2. Pavlidis T.: Algorithms for grapbics and image processing, Computer Science press, 1982. 3. Spur G., Krause F.: CAD- Technik, Lehr- und Arbeitsbuch fur die Rechnerunterstutzung in Koostruktion und Arbeitsplanung, Cart Hanser Verlag, Wien, 1984 Electrooie Books and their Potential for Interactive Learning Philip Barker Interactive Systems Research Group, Human-Computer Interaction Laboratory, School ofComputing and Mathematics, University ofTeesside, Cleveland, UK Abstract. The growing popularity and importance of electrooie books opens up many possibilities for the dissemination of instructional materiaL Electrooie books are made possible because ofthe availability oflow-cost, high-eapacity storage facilities based upon the effective use of digital optical storage media- particularly, compact disc read-only-memory (CD-ROM). Because oftheir portability, robustness and pedagogie potential such books offer a powerfut mechanism by which to imptement distance learning programmes. This paper describes and discusses the basic nature of electrooie books and those special features which make them particularl y suitable for the support ofboth open and distance learning. Two ongoing projectsin which we are currently involved are then briefly described. Keywords. Electrooie books, interactive learning, interface design l.Introduction Within most modem societies virtually everyone is familiar with books. Indeed, from very early ages (about 12-18 months) young children begin to explore simpte picture books thereby leaming more about the world they live in and the faotasy worlds that we create forthem. Subsequently, they teamtoselect the particular books that they wish tolook at and acquire the skilis needed toturn the pages of a book and identify the objects that are portrayed upon these pages. Later, as they grow older and their skilis develop further, children 'move' from picture books to illustrated story books aod subsequently to novels and other forms ofthe 'printed word'. From what bas been said above it is easy to see that books form an important part ofhuman culture. They are used to support a wide range ofhuman activity- such as education, entertainment, business and research. Indeed, for many professions (such as law, engineering and science) books form the basic mechanism by which information is stored and made available to others who may wish to use it. In termsoftheir functionality, the purpose ofbooks is thus twofold. First, they can act as repositories for information which we wish to keep- possibly, forever. Second, they cao act as amechanism by which one person ( or a groupofpeople)- the author(s) -cao communicate with aoother group ofpeople (the readers). Obviously, as was 5 6 hinted above, the material that is stored and/or communicated within the pages of a book may be fact, tietion or fantasy. Overthe centuries, since the introduetion ofthe printing press, conventional books have obviously played a fundamental role in a wide range of in formation dissemination and knowledge transfer activities. Some years ago we analysed the role of conventional books in technica! knowledge dissemination processes [1]. We identified a numberofimportant limitationsofsuch books (see table 1) and suggested that some new form ofbook was neededinorder to overcome the limitations ofthose that are printed on paper. Table 1 Basic Limitations of Conventional Books - difficult to reproduce - expensive to disseminate - difficult to update - single copies cannot easily be shared - easily damaged and vandalised - bulky to transport - embedded material is unreactive and static - cannot utilise sound - cannot utilise animation or moving pictures - unable to monitor reader's activity - cannot assess reader's understanding - unable to adapt material dynamically We used the term' electtonic book' to describe a new form ofbook whose pages were composed, not of static printer's ink, but from dynamic electtonic information. Generally, we now use this new term to describe information delivery systems that are capable of providing their users with access to pages of reactive electtonic information with which they can interact As we shall discuss later, the pages of information which make up an electtonic book are organised conceptually just like the pages of a conventional book. Obviously, the properties of a book will depend critically upon the medium upon which it is published. Nowadays, three basic media are commonly used for this purpose: paper, magnetic disc and optica! disc. Unfortunately, pubHeation on paper renders the information embedded in a book static and unreactive. In order to make its information reactive and dynamic the book concemed must be publisbed on a suitable 'interactive medium' [2]. Most electrooie books will therefore be publisbed on eithermagnetic oroptical storage media. Although books publisbed on magnetic media are useful, they are not as useful as those that are produced using digital optical storage media. The substantial utility of optica! storage media arises from its large capacity, robustness, and relative cheapness. Two types of optical storage media are commonly used for producing electtonic books: magneto-optical re-writable disc storage [3] and compactdisc read-only-memory (CD-ROM) [4]. Although a considerable amount ofwork is undertaken using re-writable optical disc (mainly for testing and prototyping purposes), the optical medium that is most often used for electrooie hook pubHeation is CD-ROM. As well as its durability and stabiHty, the other attractive feature of CD-ROM as a pubHeation medium for electrooie hooks is its immense storage capacity. A single CD-ROM is able to store a total of 650 Mbytes of information. In terms ofthe magnetie storage available of a Iow-capacity 5.25 inch floppy disc, this is equivalenttoabout 1800 floppy discs! The 650 Mbytes of storage available with a CD-ROM disc can be used in a variety of different ways forthe storage of multimedia information such as text, sound, static pictures, animation, computer programs and a very Hmited amount of motion video. Typieal figures that are often quoted to reflect the storage capacity of a CD-ROM di se are: 200,000 pages of A4 text; or 20,000 low-quaHty (PCX) image files; or 2,000 TV quaHty still images; or 30 seconds of video; or 18 hours oflow-quaHty sound. The way in which the available storage is used within a given electrooie hook production will depend critically on the 'media mix' needed by the particular pubHeation concemed. Obviously, the actual amount of material that can be storedon a CD-ROM di se will depend upon whether ornot any form of data compression technique is appHed to the in formation before it is committed to storage. Normally, in order to store full-motion video pictures on a CD-ROM various types of compression (and decompression) techniques must be applied. From the point of view of electrooie hook publication, the most commonly used CD-ROM format is that spe ei fied in the ISO 9660 standard. This is an intemationally recognised way of storing in formation on a CD-ROM so that it can be read by any computer system that bas a suitable disc drive attached to it. Unfortunately, the ISO 9660 standard doesimpose some restrictions on what can be done with a CD-ROM di se. In order to overcome these restrictions, several other approaches to using CD-ROM are rapidly emerging. The four most important of these are: CD-ROM XA (extended arhitecture); CD-I (compact disc interactive); CDTV (Commodore's Dynamic Total Vision); and DVI (Digital Video Interactive). Each ofthese offers many exciting possibilities for the design of electrooie hooks. For a variety of reasons, electrooie hooks are rapidly becoming an important pedagogie resource. Because of their importance in the context of leaming and training activity, the remaioder ofthis paper discusses a number of important issues re latingtotheir production and use. We shall first discuss their basic architecture and some of the important factors that need to be considered during their design and fabrication. A taxonomie framework (which enables hooks of this sort to be classified) will then be presented. Finally, an outline will be given ofthe nature of telemedia hooks and how they can be used to support distance leaming. 2 Basic Architecture Unlike conventional hooks, those that are publisbed on electrooie mediarequire some form of'delivery platform' to facilitate access to them. Unfortunately, although a large number of electrooie hooks exist, there seems to be no 'standard' delivery 7 8 platfonn that will enable all the currently available electrooie books to be accessed. Indeed, each particular type of electTonic book that is presently available seems to need its own specific tailor-made delivery platfonn. Three examples will be used to illustrate this point. Consider first the 'expanded books' produced by the Voyager Company for delivery platfonns based upon suitably contigured Apple Macintosh computers. 'Jurassic Park' [5] isjust one example ofthe many electTonic book titles that have recently been released by this company. Each one consistsof a single 3.5 inch magnetic disc containing compressed multimedia infonnation that must be loaded onto a hard di se before it can be accessed. Once the contents of the di se have been installed the electTonic text and diagrams provide a 'dynamic medium' that enables a user to 'become a more active reader'. As a second example consider the electTonic books that have been produced for delivery on the Sony Data Discman electTonic book player [ 6]. Hutchinson 's 'Guide to the World' [7] is a typical example of a pubHeation that bas been produced for delivery using this platfonn. Each ofthe available publications consistsof a single 8 cm digital optica) storage disc (ofthe read-only variety) embedded within a caddy that makes its appearance very similar to the expanded books that were described above. Although similar in appearance, these books are totally incompatible with those intended for the Macintosh or any other currently available delivery platfonn. The third example is taken from an electrooie book series being produced by the Elsevier publishing company - known as the 'Active Library' series. A typical example of a pubHeation taken from this series ofbooks is the 'Active Library on Corrosion' [8]. Each of these books is publisbed on a 12 cm CD-ROM disc for delivery using an IBM PC platfonn that is equipped with the Microsoft Windows 3.1 graphical user interface. Again, this electrooie pubHeation (and its host delivery platfonn) is totally incompatible witheither of the other two products that have been previously mentioned. The above three examplesare just a few ofthemany othersthat could be cited. Each of the electrooie books is excellent in its own right but, unfortunately, each one requires a different kind of delivery platfonn. There is therefore no compatibility between products. Despite the gloomy incompatibility problems outlined above some rationale is starting to emerge with respect to electrooie book architectures. This architectural rationale is emerging along two different basic directions. First, in tenns of the fundamental structure and composition of delivery platfonns; and second, in tenns of how users are meant to perceive the conceptual structure of an electTonic book publication. The basic architecture and composition of a typical electTonic book delivery platfonn is illustrated schematically in figure 1. Four basic components are needed. First, either an embedded or an explicit computer facility (this is required for control purposes and in order to achieve overall system integration). Second, a multimedia infonnation storage device (such as a CD-ROM unit). Third, a display facility that enables users to see (and/or hear) the infonnation contained in an electTonic book. Electronic Book Facility Multimedia lnformation Starage Unit i _.. Display l .. Facility COMPUTER i r;:. Access Control BB Mechanism Fig. 1. Basic structure ofan electronic book delivery station Fourth, a suitable access control mechanism. This latter component will usually consist of a suitably designed hardware interface ( such as a keyboard of a mouse) and a software environment to control the retrieval and presentation of information. The nature of electrooie book software varles considerably and can be used in many different ways to influence how users will perceive a particular publication. One useful conceptual model that now onderlies the development of many electrooie book publications is illustrated schematically in tigure 2. This depiets an electrooie book in termsof a coneetion ofreactive and dynamic pages of multimedia information (that can embed text, pietures and sound). The information contained within these pages is of three basic types: aesthetic (which is used both to help reinforce the underlying book metaphor and also to provide an ergonomically 'pleasing' appearance); informative (which is intended to instructor inform those who use a particular book); and either implicit or explicit control functions. The control options that are available are important because they enable users to specify the nature ofthe information that they wish to retrieve from a given book and how this retrieved material is to be displayed within the contines of the host delivery 9 10 platfonn. Some examples of simpte primitive control options for electrooie hooks are illustrated in tigure 2. The most commonly used functions are: next page; previous page; goto page N; exit book; and so on. exit book w....... I c pog next page c T page up t I I e lelt o4 I 1--I 1--Front Cover 0. .. 1--age ... .. -page rlght ~ 1 page down Fig. 2. Conceptual model for an electronic hook The basic control options that are used within the pages of an electrooie book may take the fonn of icons, dynamic menus, dialogue box es, scroll bars and/or hot spots. The reacti ve areas that make up hotspots may be embedded within chunks of screen-based text and/orpictures (these fonn the basis ofthe 'hypennedia' electrooie hooks that we discuss later). Obviously, the particular combination of control options made available within any given pubHeation will depend on a variety of factors- such as: the type of book involved; the purpose for which it is intended; the logical and physical sizes ofits pages; whether it has a linear or non-linear structure; and so on. When designing and producing an electrooie book it is important to give ample consideration to all those design factors that are likelyto inttuenee the overall impact, effectiveness and efficiency ofthe final publication. Some ofthe factors that we have found to be important are discussed in the following section. 3 Electrooie Book Production The design, fabrication and dissemination of electTonic books is rapidly becoming a major area of activity both within education and within the publishing industry. Because of the importance of this topic our 'electronic book' project (which commenced in 1990) [9] was intended to explore the use of CD-ROM for the pubHeation of electTonic books. Two particular objectives that we had in mind were: (a) to try and formulate a set of design guidelines to facilitate electTonic book production (paying particular attention to the role of end-user interfaces); and (b) to assess the potential of electTonic books as a mechanism for the distribution of interactive training and learning resources for use in distance education and flexible learning environments. This section of the paper concentrates on the first of these objectives; the second objective is described in more detail later. During our project two basic research and development strands were pursued. The first of these involved an evaluative study of a range of commercially available electTonic book publications (such as the Grolier Encyclopedia and Compton's Multimedia Encyclopedia). The second strand involved the design, production and controlled evaluation of a number of in-house productions. Three different in-house electTonic book demonstrators were produced [ 1 0]. These provided examples of: ( 1) a hypermedia book (entitled 'An ElectTonic Book for Early Learners'); (2) a multimedia book ( entitled 'Screen Design for Computer-based Training'); and (3) an intelligent electTonic book ( called 'a Static Picture Book with Audio Narrations'). Each demonstrator was designed to explore a different aspect of electTonic book production. In our research project we discovered that two basic types of design tooi were needed to facilitate the creation of electTonic books. The first of these was a set of high-level design models while the second constitutes a collection of more pragmatic low-level 'tips' ( or guidelines) re lating to: (I) end-user interface design; (2) the way in formation is organised on CD-ROM; and (3) the effective creation ofaccess stations for use with electronic books. Undoubtedly, the most important of our findings was the set ofthree high-level development models that we formulated. We referto these as the 'conceptual' model (shown in figure 2), the 'design' modeland the 'fabrication' model. The first of these is intended forend-users of electTonic books (as an orientation tooi). The second and third models are intended for designers and producers of electTonic books as they describe architectural and procedural issues, respectively. When designing electTonic books we found that the basic design model shown in tigure 3 was an extremely useful asset [I 0-12]. A majorobjective ofthe initia! design phase ofan electronic book is the formulation ofthe end-user interfaces that will be used to enable users to access the information that is held within it [13]. Book and page structures then have to be decided upon [ 14]. The content ofthe book must then be specified. Finally, the nature of the 'reader services' (browsers, bookmarks, glossari es, etc) must be agreed upon. As can beseen from tigure 3, theuse of a suitably designed knowledge corpus is fundamental to the creation of our electronic books. 11 12 Users Interfaces Text ~i"' i Page Structure Pictures ~ '""' """~ ""- I,~·:.::":.::,., ~ ,, V Strategles Animalions -----~ • ~ Browsers --. Multi-media ,.--Electronic Book .,_-- lndexing Facllities ___..... System Learnlng ~~ _ .__ Strategles Skill Development '"----~-~ Exercises .~:::~ /_ __ ?_:_,:_:~.:.~-~=-~--H-;{e_~·_,;!.:~n-mc-~g_:_""" Simulations Hyper-media Knowledge Corpus Fig. 3. Development model for electrooie books Obviously, the attractive feature of CD-ROM as a pubHeation medium is the large capacity that it offers for the starage of multimedia and hypermedia knowledge corpora. The creation of electrooie hooks requires a number of different development stages. Theseneed to be rigorously adhered to i f efficient production is to be achieved. The fabrication model is therefore used to describe the relationships between the various stages that are involved in transferring materials from the initia! development phase (based upon the use of hard disc and re-writable optica! disc for CD-ROM emulation) through prototyping through tothe final production stage (usingread-only discs). Obviously, it is these latter discs (in ISO 9660 format) that are distributed to users of our products. The fabrication model that we use to support our electrooie book productions is illustrated schematically in tigure 4. lntegratlon, Interlinklog and Synchronlsatlon Fig. 4. Fabrication model for electTonic books As can be seen from this tigure the initial development phase of an electTonic book project involves specifying the content and structure of the book. These are then brought together by means of the scripting process. The script gives a detailed specification ofthe material that is to be embedded within the book (on a page by page basis) and the relationship between the pages, control mechanisms and end-user tools that are to be made available. lt will also embed details of all the textual, sonic and pictorial information that is needed for the electronic book pages. The creation of the script will therefore involve the simultaneous production of all the multimedia resources needed for the book. Once these have been created and individually tested the next phase of electronic book production can commence. This will involve integrating and synchronising the multimedia resources and (if a hypermedia book is being produced) interlinking them in appropriate ways. When the integrating, interlinking and synchronising phase of electTonic book production is complete the evaluation and testing phases can commence. This may 13 14 involve the generation ofvarious kinds of'prototype book'. Once the results ofthe evaluation and testing have been obtained any final amendations can be incorporated before the final CD~ ROM product is mastered, replicated and distributed to users. During the course of our research into electrooie book production we were able to formulate a range ofuseful design guidelines. Wetook some trouble to document these as we anticipated that they would be of help to other people who might wish to become involved in electrooie book production. The series of guidelines that we produced feil naturally into the following six basic categories: knowledge engineer-ing; page design; interaction styles; end-user tools and services; use of multimedia; and use of hypermedia. Further detailed descriptions of the actual guidelines themselves are given elsewhere [10,13,t5]. 4 Types of Electronic Book Electrooie books can be classified in a variety of different ways depending upon the medium that they are publisbed on, the functions they perform and the types of facilities and services that they provide. One very simpte taxonomy proposed by Barker and Giller [t6] categorises electrooie books into four basic classes: (t) archival; (2) informational; (3) instructional; and (4) interrogational. The first category ofbook offers a metbod of storing large volumes ofinformation re latingtosome particular subject area. Within such books the end-userinterface will normally be designed in such a way that it will permit a variety of different methods of in formation retrieval. Examples of such books include large catalogue systems and databases of records and data. The Grolier Encyclopedia [t7] and Compton's Multimedia Encyclopedia [18] are two such examples of this category. In many ways, electrooie books that fall into the informational category overlap with those in category one. However, the stored information is usually tess comprehensive and more specific- relating toa particular topic area. An example ofthis category of electrooie book is the Oxford Textbook ofMedicine on CompactDisc [t9]. The third category of electrooie book listed above (instructional) is intended to provide a means of achieving highty efficient and effective skill and knowledge transfer mechanisms for the support oflearning and training activities. Users of such books are given the opportunity to team and progress at their own pace using their own particular style oflearning. Some electrooie books in this category will actually assess and adapt to its user' s personallearning style. Such books automatically re-con tigure the material that is presented so as to accommodate its user' s preferred approach to learning. Our 'Screen Design for Computer -based Training' [ t 0] is an example of an instructional electronic book. The intention ofthe last category of electrooie book (interrogational) is to support testing, quizzing and assessment activities which will enable readers to gauge their depth ofknowledge about a particular topic. This type ofbook contains three essential components: a question ( or exercise) bank; a testing and assessment package; and an expert system. The latter is used to analyse a reader' s responses and deduce an appropriate grade or level of competence based on these responses. Although the taxonomy described above is a useful one, it is often advantageous to use one which is more 'fine-grained'. In view ofthis requirement, we nowpropose a taxonomythat containsten basic classes of electrooie book [2,14]. Depending upon the type of information that they embed and the kinds of facility that they make available, we now suggest that electrooie books can be classified into the following basic categories: text books; static picture books; moving picture books; talking books; multimedia books; polymedia books; hypermedia books; intelligent elec-trooie books; telemedia electrooie books; and cyberspace books. Each of these categoties of electrooie book is briefly discussed below. Astheir name suggests, text books are composed of pages oftextual material that have been organised into suitably sized 'chunks' ofinformation. The chunk size that is employed will depend upon the screen size that is used for in formation display and the number of chunks/page that it is required to present simultaneousl y. Static picture books consist of a colleefion ofpictures that are organised into some partienlar theme; the pictures may be ofvarious 'qualities' with respecttotheir resolution and range of colours that they embed. Moving picture books are constructed from either animation clips or motion video segments - or combinations of each of these; the 'mix' used will depend upon a variety of factors such as the purpose ofthe electrooie book and the 'message' that it is to • convey'. Talking books depend fortheir success upon recorded sound (both high- and low-quality) that is used in conjunction with a variety of 'interactive audio' techniques to facilitate end-user control of in formation and knowledge transfer. Multimedia books use various combinations of two or more communication channels (either in sequence or simultaneously) in order to eneode a partienlar message. Such books use text, sound, pictures and moving images that are basically organised in a 'linear' fashion. The materials are delivered by means of a single delivery medium (such as magnetic disc orCD-ROM). Polymedia books, in contrast to multimedia books, use a combination of several different media (CD-ROM, magnetic disc, paper, and so on) in order to deliver their information to end-users. Hypermedia electrooie books have much in common with multimedia books in that they depend upon the use of multiple communication channels. However, unlike multimedia books, hypermedia books employ 'non-linear' organisations ofinforma-tion based upon the use of web-like structures [20]. Because of the embedded intelligence that they contain, intelligent books are in many ways similar to the 'interrogational' books described by Barker and Giller [ 16]. These books are capable of dynamic adaptation as a consequence of interaction with end-users. Undoubtedly, two ofthe most exciting types ofbook that we are currently developing are telemedia books and cyberspace books. The first of these uses telecommunication facilities to augment the capabilities of a CD-ROM pubHeation in order to support highly interactive distributed distance leaming activities [21-23]. Cyberspace books are used as a means of providing their readers with access to various types of virtual reality facility; such books employ different kinds ofinteractive simulation environ-ment in order to provide end-users with participative, 'real-life' en counters that they would not normally be able to experience. 15 16 Because oftheir importance in the context of distance and independent leaming activities, telemedia hooks will he discussed in more detail in the following section ofthe paper. 5 Telemedia Books and Distance Learning As we have suggested above, electTonic hookscan he used to support a wide range ofleaming and training applications. They are particulary important in the context of supporting distance leaming, individualised self-supported leaming and co-operative group leaming at a distance [21,23]. The way in which we envisage this happening is illustrated conceptually in tigure 5. ~ Electronlc ~ Leamlng ~1/ Electronlc Vlrtual Classrooms Note-Book Computers Classrooms ~,....---..., Portable lnteracllve Learning Envlropnments Fig. 5. The role of electronic hooks in distance education Emhedded within this tigure is the idea of students leaming by electTonic means through the use of electTonic classrooms. Such classrooms may exist in two basic forms. First, they may exist physically in a particular geographicallocation- heing composed of a relatively small numher of interactive workstations contained within a given room or building. Second, they may ex i st in the form of a 'virtual classroom' that is composed ofan almost Iimitless numher ofleaming stations that are physically distributed anywhere in the world- for example, in home environments, in people's places ofwork, in public places such as Iibraries, and so on. We envisage that a very large proportion ofthe delivery platforms for interactive learning will be of a highly portable nature - based upon the use of various types of portable computersystem (suchas lap-tops, hand-held computers, notebook comput-ers and various sorts of consumer products based upon technologies such as CD-I and CDTV). We refer to such environments as 'portable interactive learning environ-ments' [22]. The use of such technology is attractive because it means that many aspects of learning and training can transeend institutional boundaries. Of course, it is anticipated that all the workstations used to support this type of learn~ng will incorporate the type of architecture that was previously illustrated in figure I. They will therefore be capable ofmaking large amounts of multimedia andlor hypermedia information available through the medium of electrooie books that are publisbed on optical media such as CD-ROM. Obviously, the use of stand-alone workstations similar to those described above (and illustrated schematically in figures I and 5) would only go part ofthe way to realising the overall needs oflearners and trainees in a virtual classroom situation. lt is therefore important to consider what other functions a portable learning environment needs to provide. Naturally, an important aspect of conventional classrooms that must be provided within virtual classrooms is 'class contact' with other students and tutors. In order to meet this requirement it is necessary to provide various forms of'person to person • communication facilities (for example, by means of electrooie mail, bulletin boards, telephone, video-phone andlor conferencing facilities). Such facilities not only allow fellow students to communieate with each other but also support, if necessary, communication with tutors and subject matter expertswhomake themselves available for use as 'human learning resources'. The type of computer-based learning environments needed to support this approach to learning and training is illustrated schematically in figure 6. Fundamental to this type of workstation is the presence of an appropriate conneetion toa host telecommunication facility. This may take a variety of different forms. It might be a simpte modem and 'dial up' conneetion (made through a telephone network) to some other compatible modem attached to a remote host computer. Altematively, the workstation may be connected directly (via a suitable network card) toa local areanetwork and then, through a series ofwide area networks tosome remote site that might be located almost anywhere in the world. Obviously, as wellas supporting distant person-to-person communication such workstations are also able to support access toa wide range of other remote resources such as electrooie libraTies and either down-loadable or shared cyberspaces. As we have suggested earlier, electrooie books that are designed in such a way that they are able to take advantage of a telecommunications infra-structure similarto that described above are referred toas te Iemedia books. We envisage a varietyof different uses for such books within the context of distributed distance learning both within academie and non-academie organisations. Two examples of systems that we are currently developing will be briefly described in order to illustrate how such books may be used. The simplest type of te Iemedia book for use in a delivery platform similar to that shown in figure 6 is one which uses CD-ROM for the bulk pubHeation of large 17 18 Telemedia Books, Cyberspaces and Electron ie Libraries Floppy Disc Ubrary Optical Disc Ubrary USER Conventional Books Fig. 6. Polyfunctional workstation for electronic hook delivery amounts ofinteractive course material for use by students. However, also embedded in the electronic book (in the form of a reader service) is an electronic mail facility that is referenced through an icon. This facility can be used in a variety ofways to support communicative exchanges between students and for the transfer andlor sharing of materials. The work that we are currently undertaking in our 'interactive language leaming project' illustrates this approach totheuse ofteiemedia books for leaming French [24]. The essential course material is embedded upon a CD-ROM (as an electronic book) that allows individual students to listen to and practice speaking French. They can also test their writing ability by sending electronic mail communications to each other and to fellow students and tutors located both within the UK and in France. The second te Iemedia book project that we are currently working on involves the design of support material in the form of 'interactive manuals' for use within an electronic performance support system (EPSS) that is to be used within the context of office automation [25]. The EPSS system is intended to provide just-in-time (JIT) training at a partienlar point of need. The system that we have been designing and prototyping is intended to offer this type of support within a geographically distributed multi-centre organisation. The interactive manuals that embed the technicaland procedural !nformation necessary for the organisation are publisbed on CD-ROM. The communlcations infra-structure running above this basic publication level is accessed as a standard facility within a telemedia 'company services' handbook that enables employees to gain access toeach other and to sourees of expert helpand advice. This enables employeestoshare skilis and when necessary develop new ones through tele-tutoring techniques and nT methods. 6 Coneinsion Books are an important mechanism for the storage and communication of infonna-tion. Conventional hooks are publisbed on paper as a coneetion ofpages of static infonnation. Therefore, in many ways, the concept of a hook is intimately bound to this medium. This bas both advantages and disadvantages- some ofwhich have been discussed in this paper. Of course, there is no inherent reason why the concept of a hook bas to be media dependent Other media could equally well be used to publish hooks or embed the hook concept. For example, conventional hooks could be 'televised' on television but because ofthe low interactivity ofthis medium users would loosethe abilityto 'turn pages'. However, byusing a more interactive medium (such as a computer) to publish a hook, many ofthe properties of conventional hooks can be emulated. It is therefore important to realise that the properties of a hook will depend very much upon the nature of the medium ( or 'media mix') used for its publication. In this paper we advocate the use of telemedia electrooie hooks as a useful resource for promoting and supporting distance education and cooperative group learning at a distance. Two examples ofthe use of electrooie hooks for di stance learning have been briefly described. Obviously, many more exciting possibilities of this approach yet remain to be explored. 7 References 1. Barker, P.G. and Manji, K.A., New Books for Old, Programmed Learning and Educational Technology, 25(4), 310-313, (1988). 2. Barker, P.G., Interactive Electrooie Books, lnteractive Multimedia, 2(1 ), 11-28, (1991). 3. Paris-Roth, J., Re-writable Optical Storage Technology, Meckler, Westport, CT 06880, USA, 1991. 4. Lambert, S. and Ropiequet, S., CD-ROM: The New Papyrus- The Current and Future State ofthe Art, Microsoft Press, Redmond, WA 98073-9717, USA, 1986. 5. Crichton, M., 'Jurassic Park', The VoyagerCompany, SantaMoniea, CA, USA, 1991. 6. Rockman, S., Sony Data Discman DD-1EX, Personal ComputerWorld, 15(3), 262-266, (1992). 7. Hutchinson's Guide to the World, E1ectronie Book Version 1.0, Helicon Pub-lishing, London, UK, 1992. 8. Bogaerts, W .F. and Agema, K.S., Active Library on Corrosion, Elsevier Science Publishers, Amsterdam, The Netherlands, 1992. 9. Barker, P.G. and Giller, S., Design Guidelines for Electrooie Book Production - Overview Report, lnteractive Systems Research Group, School ofComputing and Mathematics, University ofTeesside, Cleveland, UK, 1992. 19 20 10. Giller, S., Design Guidelines for Electrooie Book Production, MPhil Thesis, University ofTeesside, Cleveland, UK, 1992. 11. Barker, P.G., Electrooie Books, Lesming Resources Joumal, 6(3), 62-68, (1990). 12. Barker, P.G. and Manji, K.A., Designing Electrooie Books, Joumal of Artificial Intelligence in Education, 1 (2), 31-42, (1990). 13. Richards, S., End-User Interfaces to Electrooie Books, Draft PhD Dissertation, Interactive Systems Research Group, School ofComputing and Mathetnatics, University ofTeesside, Cleveland, UK, 1992. 14. Barker, P.G., Electrooie Books, Special Edition ofEducational and Training Technology International, 28(4), 269-368, (1991). 15. Barker, P.G. and Giller, S.A., Design Guidelines for Electrooie Book Production, Final Project Report submitted to the Lesming Technology Unit, Training, Enterprise and Education Directorate, Department ofEmployment, Moorfoot, Sheffield, 1992. 16. Barker,P.G. andGiller, S.,Electronie Books, 179-184inAspectsofEducational Technology, Volume XXV: Developing and Measuring Competence, edited by D. Saunders and P. Race, Kogan Page, London, 1992. 17. Grolier Inc., The New Grolier Encyclopedia- User's Guide, Grolier Electrooie Publishing Inc, Sherman Tumpike, Danbury, CT, USA, 1988. 18. Britannica Software, Compton's Multimedia Encyclopedia, Britannica Soft-ware Inc., San Francisco, California, USA, 1990. 19. Weatherall, D.J., Ledingham, J.G.G. and Warren, D.A., The Oxford Textbook ofMedicine on CompactDisc, Second Edition, Oxford University Press, Oxford, 1989. 20. Barker, P.G., Hypermedia Electrooie Books, in Proceedings of the Seventh Canadian Symposium on Instructional Technology, Montreal, Quebec, 6th-9th May, 1992. 21. Barker, P.G. and Giller, S., Electrooie Books for Distance Lesming, 759-761 in Volume 2 ofthe Proceedings ofthe Ninth International Conference on Technol-ogy and Education, Paris, France, 16-20 March, 1992. 22. Bark er, P. G., Portable Interactive Lesming Environments, paperpresented atthe AETT '92 International Conference, University ofYork, 6th-9th April, 1992. 23. Barker, P.G., Electrooie Books and their Potential for International Distance Learning, Invited paperpresentedat theEast-West Conference on 'Emerging Computer Technologies in Education', Moscow, 6-9 April, 1992. 24. Interactive Language Lesming Project, National Council for Educational Tech-nology, SirWilliam Lyons Road, SciencePark, UniversityofWarwick, Coven-try, CV4 7EZ, 1993. 25. Banelji, A.K., Designing Electrooie Performance Support Systems, Draft PhD Thesis, Interactive Systems Research Group, School of Computing and Math-ematics, University ofTeesside, Cleveland, UK, 1993. Summary NATO Summerschool Robbert-Jan Beun Institute for Perception Research/IPO P.O. Box 513 5600 MB Eindhoven, The Netherlands A framework for speech-act generation in cooperative dialogue A central theme in Artificial Intelligence approaches to speech acts is the modeHing of the cognitive-state changes of the participants in terms of beliefs and intentions (e.g., Perrault 1990). Although this so called context-change approach seems to offer an attractive formal treatment of cognitive-state changes of dialogue participants, the approach lacks the power to model the generation of speech acts in a conversation. Generation of a speech act is of crucial importance to determine, for instance, adequate behaviour of an information exchange system in responsetoa user's primary communicative goals. To determine adequate communicative behaviour of participants (human or machine) in a cooperative dialogue, a theoretica! framewerk will be presented that is based on the concepts of.dialogue game (Carlson 1985), preferenee organization (Levinson 1983) and cognitive state(-change) of the participants. Main goal of the dialogue is to exchange information in a collaborative manner so that, if the information is available, in the final state the participants mutually believe the answer to an initial question. The participants' communicative strategy is determined by the rules of the dialogue game. Three types of rules will be considered: a. preferenee rules that teil us which speech act is preferred in a certain state, b. prohibitory rules that forbid the performance of certain speech acts and c. closing rules that defme the criteria for ending the conversation. The resulting structure of the information exchange can be complex, since the knowledge to find the answer to the initial question may be distributed among the participants and may therefore result in subdialogues and the generation of counter-questions. By means of simple examples, it will be shown that the application of different communicative strategies and different initial belief states give important theoretica! insight in the conversational structure of the dialogue. Moreover, it is claimed that, if the type of dialogue is well-restricted, only three types of belief are significant todetermine adequate behaviour of an information exchange system in a cooperative situation: a. the system's belief about the domain of discourse, b. the system 's belief about the mutual belief of the system and user and c. the system' s belief a bout what the user does not believe. References: Carlson, L. (1985) Dialogue Games. Dordrecht: Reidel Publishing Company. Levinson, S. (1983) Pragmatics. Cambridge: Cambridge University Press. Perrault, C.R. (1990) An application of default logictospeech act theory. In: P.R. Cohen, J. Morgan & M.E. Pollack (Eds.), Intentions in Communication. Cambridge, Mass.: MIT Pre ss. 23 Cooperative Problem Solving as a Basis for Computer Assisted Learning Sviatoslav Brainov lnstitute of Mathematics, Sofia, Buigaria Abstract. In this paper we present a new approach tor computer assisted learning. This approach incorporates features of the intelligent tutoring systems and distributed problem solving. lt has been designed to permit the multiperson - multimachine interaction and to stimulate the social activity of the · students. A framewerk called DEE (Distributed Educational Environment) that supports multiagent learning activity on the basis of cooperative problem solving is described. The DEE is organized as a netwerk of autonorneus problem solvers. Keywords. Computer assisted learning, distributed problem solving, cooperation, task allocation, conflict resolution, blackboard-based systems. 1. Introduetion The proliferation of the computer networks and the recognition that much human activity involves groups of people have provoked the interest in distributed problem solving. One of the main approaches in the distributed problem solving is the person-machine coordination. This approach provides useful means for rnanaging a collection of people and machines werking tagether in coordinated and cooperative way. Within this approach research in distributed problem solving has great impact in the area of computer assisted learning. Currently accepted paradigm in the computer assisted learning of one (student)-to-one(computer) interaction may be improved by absorbing some ideas trom multiagent interaction theory. In this paper we present a framewerk called DEE (Distributed Educational Environment) that supports multiagent learning activity on the basis of cooperative problem solving. The DEE is organized as a netwerk of autonorneus problem solvers. Some of the problem solvers are extended with teaching abilities. Students may participate in the netwerk by one of the existing nodes. The DEE may be considered as a departure trom the conventional one-to-on~ teaching model to one that engages many learners 25 26 and teachers. In our apinion the process of the learning as a coordinated multiagent activity more directly reflects the nature of human learning. The remainder of this paper cammences with the motivations behind the our approach. We then describe the architecture of the DEE and organization of communications. Finally, we summarize our approach and propose some open problems. 2. Motivation One of the major difficulties which arises when using a conventional educational system is connected with resource limitations. lndividual computational agents have bounded rationality, bounded resources for problem solving and bounded bath domain and didactic expertise. The primary contribution of the DEE is that it makes it possible to combine resources of different intelligent nodes. The combined abilities of the agents transeend their individual abilities so that the scope and the quality of the teaching activity they perfarm as a group may increase considerably. Another significant problem that aften arises in computer assisted learning is the incompatibility between the computer knowledge representation and the students' mental models. Discrepancies in conceptual rnadeis aften make the students unsusceptible to following the computer guidance. The DEE permits different knowledge representation schemes and knowledge perspectives. This makes it possible for the system to capture non-trivial salution paths and exhibit more extensive diagnostic policies. The architecture of the DEE fits in the open-systems model [1). Open systems have no fixed boundaries and are easily extendible with new knowied ge bases or intelligent agents. The main intention behind the DEE is to support the social activity in the learning process. Interaction between the students participating in the netwerk may simulate their motivation. Since cooperation is an integral part of social interaction [2], a group of students may achieve more by werking tagether than by werking alone. The grouping of students may appear on the basis of: common interests, common goals, unsolved problems, similar behaviour and so on. One of the major requirements to the educational systems is their adaptation ability. In the computer assisted learning systems the only way to obtain the adaptation abilities is by using and developing the user model. In the DEE, adaptation abilities reside in the multiagent organization which may be changed dynamically in the course of teaching. In this case the user model is built by several agents, playing different roles and functions. Repeatedly changing needs of the students may be met in the DEE by dynamically rearranging the problem solving structure, the structure of roles and priorities. Furthermore, the multiegent structure is tolerant to the failures of single agents and degrades its performance gracefully. 3. Distributecl Educational Environment The DEE is organized as a netwerk of autonorneus intelligent nodes. The distribution can arise because of spatial distance, deductive ditterenee or semantica! distance. Each student may participate in the netwerk by means of particular nodes supplemented with teaching abilities and didactic expertise. These nodes are called 'tutors'. Out of the netwerk context, the tutor node may be considered as a single intelligent tutoring system [3]. Besides the tutor nodes the netwerk may include other types of nodes: problem experts, teaching experts and so on. The interface between the netwerk and the student is realized through the tutor node. Because of bounded resources and bounded rationality this node is not able to deal effectively with all the problems to be solved. lt is more reasanabie to use the knowledge and problem solving skilis of multiple nodes, each of which handles some part of the total problem. In the case of the functional and knowledge specialization among the nodes this approach guarantees for the better use of the netwerk resources. In the DEE the initia! task allocation is done by the tutor node. After that each node manages its activity on its own, allocating subproblems of the original problem to other nodes. Each node in the netwerk has an extended blackboard-based architecture [4,5]. The node bleekboard is intended to contain only dynamically local context of computations. The global context of the current situation is maintained by global bleekboards associated with each student. The student bleekboards play twofold function. On the one hand, they contain all temporary hypotheses, current goals and active data received by different nodes. In this way the multiegent planning and synchronization is supported. On the other hand, student blackboards play the role of the students models and action spaces because of their purpose to contain all the necessary intermation about the students. The student bleekboard is divided into two areas. The first one, called student area, represents student's belief model. Belief changes occur as implications of some student or node action. The secend area, called netwerk area, contains netwerk goals, partial solutions, current hypotheses, plans and so on. All the nodes are able to obtain and modify intermation trom the netwerk area. 27 28 Belief rev1s1on in the student area is managed by the tutor node associated with the student. The student may exchange messages with other students communicating with their tutor nodes. After such communication the tutor nodes are responsible tor the subsequent belief revision in the student areas. Another case of belief revision arises when some actions are. taken by the netwerk. One category of actions may modify problem space, other actions are connected with the student, a third category of actions change netwerk organization. Besides the monitoring the student area the tutor node is also responsible tor the monitoring of the netwerk area in the student blackboard. All the read and write data-access requests trom the other nodes are handled by the tutor node. Since the different nodes are nat guaranteed to modify the blackboard uninterruptedly, the tutor node has to ensure the syntactic and the semantic integrity of the blackboard information. This integrity is achieved by supplementary labels tor each blackboard element denoting the nodes the element is associated with. As an example, the plans in the netwerk area are labelled with the addresses of the nodes executing them, the hypotheses are labelled with the addresses of the nodes proposed to them and so on. lf some modification in the blackboard is made all the nodes it involves are informed by the tutor node. In such way the redundant work is avoided and the node processing power is concentrated in more perspective salution paths. Another advantage of such labelling mechanism is that it provides the netwerk with the global view of the current situation. Given such global view, many contiiets between the nodes may be avoided and better distribution of the processing laad may be achieved. The plan synchronization is carried out by the tutor node. Since the plans of different nodes are connected by temporal and spatial resources and causal relations, interaction in the plans must be controlled. When incompatibility between the plans occurs the tutor node may postpene some plan execution or reallocate the task tor replanning to another node. This is possible because of the tutor node ability to manage the student blackboard content. After the modification of the blackboard plans the nodes executing these plans must raarrange their activity according to the modifications. In such a way the tutor node may nat only keep a watch over the activity in the netwerk but control this activity. The central role the tutor node plays is relative to its student blackboard. They may be other tutor nodes and other students in the netwerk: In this case all the tutor nodes have the same authority and contiiets between them must be resolved by negotiations [6]. 4. Communication protocol and internode interaction A convenient methad for internode communication is contract net protocol [7,8]. lt provides powertul mechanism for task allocation, since it permits a more informed choice from among the alternative nodes to which tasks may be allocated. The key element in the task allocation is the negotiation procedure. The negotiations involve two nodes which are called manager and contractor. The manager generates a task to be done and the contractor has agreed to perfarm the task. The manager announces the existence of the task to all potential contractors. Each contractor evaluates its own level of interest with respect to the type of the task and the avaitable resources. lf the task is found of sufficient interest the contractor informs the manager about its readiness to perfarm the task. The manager rriay choose trom among the several potential contractors. lt selects these contractors which better match the task requirements. A contract is thus an explicit agreement between a node that generates the task and a node that executes the task. The nodes are nat constrained to be only managers or only contractors. lt is possible for a node to be simultaneously bath manager and contractor tor different contracts. As an example, in the process of executing the task the contractor may deliver parts of the task to another contractors. In this case this node is a contractor in respect to the whole task and a manager in respect to the parts of the task. In the DEE the initial tasks are generated by the tutor node. The announcement of these tasks is carried out by placing them into the student blackboard. After negotiations the tasks are allocated to contractors and the contractors' narnes are placed into the blackboard. In this way all the nodes are acquainted with the netwerk activity. Furthermore, by virtue of this publicity, several contractors in a given contract may share partial results and communicate between each ether. In the conventional contract net protocol such communication is not possible because the task allocation information is inaccessible. The mechanism of task allocation by global blackboard helps the work duplications to be avoided. As an example, a node may check the student blackboard whenever it receives a new task. lf such a task is performed by a nother node, the first node may simply copy the results. The global blackboard approach is also well suited for the communication management. In the complex networks with limited communication bandwith, when each node must communicate with many other nodes, the communication policy is of great importance. By examining the student blackboard the nodes may reason about each ethers' present activities. lt allows the nodes to b~tter predict the effects a message will have on the 29 30 ether nodes. Predietiens lead to reducing communications because only necessary data need to be communicated. In fact, the student bleekboard plays the role of the activity map in which all the points of task and result sharing are marked. The nodes use the activities allocation information to recognize and establish communication links, to discover how activities may be reordered to avoid harmful interactions and to promote helptul interactions. More interesting communication problems arise when there are many students and many tutor nodes in the netwerk. While the tutor nodes have higher priority in respect to ether kinds of nodes, the priorities of all tutor nodes are equal. This requires more complex decisions about the interaction between the students and the tutor nodes. The DEE allows the students to communicate between each ether, to form groups of interests, to pursue a goal by common efforts and so on. In such a way different student's viewpoints may be analyzed and a common cognitive space may be constructed. The ability of the DEE to coordinate different users resembles to a great extent the participant systems [9]. In contrast to conventional participant systems, the DEE has its own problem solving abilities and domain expertise. This makes it ditticuit to coordinate the activity of the netwerk and the students groups. In the DEE three heuristics are proposed tor conflict resolution between the netwerk and the students: 1. Group formation. When conflict arises between the students the current individual student goals may be changed to equivalent ones which promotas cooperation. 2. Group destruction. When conflict arises between a student group and the netwerk the students are given competitive goals. 3. Changing the focus of attention. When internal conflict arises between the tutor nodes the students are given ether problem situations. In such a way plans and preferences of the tutor nodes are changed. 5. Conclusions In this paper we have presented a new approach for computer assisted learning. This approach incorporates features of the intelligent tutoring systems and distributed problem solvers. lt has been devised to permit the multiperson - multimachine interaction and to stimulate the social activity of the students. Saveral perspectives for further research remain. A mechanism tor internode cooperation must be developed. On its basis the tutor nodes may control the student groups and influence the behaviour of each student. For these purposes the economie and game-theoretic models may be useful [10,11]. Raferences 1. Hewitt, C.: Offices are Open Systems. In: B. Huberman (ed.) Ecology of Computation, 5-24. Amsterdam: Elsevier Science Publishers, North Holland 1988 2. Axelrod, R.:The Evolution of Cooperation. Basic Books 1984 3. Sleeman, D., Brown, J. S.: Intelligent Tutoring Systems. New Vork: Academie Press 1982 4. Hayes-Roth, B.: A Blackboard Architecture for ControL Artificial lntelligence 26, 251-321 1985 5. Durfee, E. H.: Coordination of Distributed Problem Solvers. Boston: Kluwer Academie Publishers 1988 6. Rosenschein, J. S., Zlotkin, G.: Negotiation and Task Sharing Among Autonorneus Agentsin Cooperative Domains. In Proceedings of the 1989 International Joint Conference on Artificiallntelligence, 912-917. Morgan Kaufmann Publishers 1989 7. Smith, R. G.: The Contract Net Protocol: High- Level Communication and Control in a Distributed Problem Solver. IEEE Transactions on Copmuters C-29, 12, 1104-1113 1980 8. Davis, R., Smith, R. G.: Negotiation as a Metaphor for Distributed Problem Solving. Artificiallntelligence 20 (1), 63-1091983 9. Chang, E.: Participant Systems for Cooperative Work. In: M. Huhns (ed.) Distributed Artificiallntelligence, 311-339. London: Pitman 1987 10. Rosenschein, J. S .• Genesereth, M.R.: Deals Among Rational Agents. In Proceedings of the 1985 International Joint Conference on Artificial I ntelligence, 91-99 1985 • 11. Malone, T. W.: Modeling Coordination in Organizations and Markets, Management Science 33 (10), 1317-1332 1980 31 A change of view The parameter projection problem in threedimensional view setting Eric Brok1 and Peter van Splunder 1 Open university, Faculty of Engineering, Box 2960,6401 DL Heerlen, the Netherlands. 2 PTI Research, Insitute for Applied Social Science Research, Box 421, 2260 AK Leid-schendam, the Netherlands. Abstract. An experimental on screen control panel for 30 view setting is described. The design requires only a regular 20 mouse input device. The user sets the desired viewpoint by direct manipulation of metaphoric icons, which is more facile and natural than by setting several slide control bars. This specific control panel is also adressed as an illustration of a general interface design issue: the internal technica! parameters of the computer program must be projected onto the user interface with great care to avoid confusion in the user. Keywords. User/machine systems, human factors, interface design, computer graphics, 30 graphics, interactive graphics, interaction techniques, input devices, virtual controllers, rotation con trol, view setting, parameter projection problem. 1 Introduetion 1.1 The parameter projection problem Usually he functioning of a computer program is determined by a set of internal parameters. Many of these parameters are to be controlled by the end user. Seen from the view of a computer pro grammer, it would be most obvious to ex pose these parameters directly to the user. But in many cases the technica! parameters by itself are not organized in a way that the user can grasp easily. The user probably relates to the situáiion at hand by very different conceptual entities. In short there is a gap between technically significant parameters and psychologically significant parameters. To facilitate human-machine communication these two sets have to be projected carefully onto each other. We will call this issue the parameter projection problem (PPP). We will discus~ this issue further by an example interface. 1.2 Threedimensional communication In genera!, computer systems cornmunicate information from and to people. This information of course can take many forms, one of which is graphical in three dimensions. It can be graphical by nature (e.g. a landscape) or by modeHing (e.g. the results of a factor analysis). Communication of threedimensional information is important in various fields of interest such as artistic illustration, teehoical design, architecture, city planning, math, chemica! engineering, medical diagnosis and so on. These fields have developed numerous practices of education and training. So it is clear that 30 grapbics are important to educational settings as well. Of course the students and trainees should not be forced to divert their attention from learning the prime subject to figuring out the user interface. Therefore particularly in education the PPP issue deserves explicit attention. Display of 3D information usually involves 30 input from the user. Often the content matter itself is generated by the user such as in drawing and design. But even when the computer system generates the scene, the user might have to communicate 3D information such as the desired viewpoint from which to look at the scene. This paper concerns this type of spatial input which we will call view setting. 1.3 Optical projection issues Before turning to the PPP-issue, we want to make clear that in 30 grapbics there is also an optical projection problem. First threedimensional objects have to be projected onto the flat screen. Mathernaties (stereometry) deals with that quite well. Second and more probiernatie is the twodimensional on-screen manipulation of threedimensional objects. Simply pointing at an object can be already ambiguous (Bier, 1990). It is now widely acknowledged that creating and manipulating 30 objectsis a complicated task (e.g. Veniola, 1993). In recent years, as 3D graphical systems became increasingly available and cognitive ergonornies gained attention, researchers turned to the issue. 1.4 Related studies One way to deal with this issue is to develope actvaneed mechanica! interaction devices. Datagloves, 30 glasses, 30 mouse devices (e.g. Venolia, 1993) and the like seem tobring truly threedimensional interaction literally within reach. Existing CAD/CAM applications often use hardware dials to rotate threedimensional objects. This is another way of diverting interaction away from the flat screen. We acknowledge those approaches, but this paper will discuss an interface that only requires on-screen interaction with 20 mouse. (The design of this interface however could easily be adapted to true 3D input.) Chen, Mountford and Chellen (1988) describe several virtual controls for rotating 3D objects. Most promising seems what they call the 'virtual sphere' (which they also compare with a resembling control originally described by Evans, Tanner & Wein (1987)). This control is a virtual track ball that can be manipulated by mouse cursor. Rolling it up or down makes the selected object pitch (see also table 1). Dragging sideways makes the object yaw. And (unlike physical trackballs) dragging along the circumference makes the object roll. However promising, this control concerns only rotation of a single object. Object selection and translation throughout the scene might call for another approach. The current paper therefore explicitely focusses on the combined task of translation and rotation. 2 The example program 2.1 Turning tables in PLEXI The control panels we will discuss are part of a prototype application program. We will first take a quick glance of this program called PLEXP itself. PLEXI enables the user to furnish a threedimensional room (see figure 1). Because arranging furniture mainly concerns movements along the floor, this interaction takes place in a 20 ground plan window. To add furniture to the room, the user can piek pieces of furniture from a pallet (the 'storage room') by mouse and drop them somewhere on the ground Fig. 1. A room with a view. plan. Once present in the room, the pieces can be moved and turned around by dragging. They also can be deleted, copied and assigned color. The program is unlike a mere drawing program because it checks all user rearrangements for violations of certain syntactical constraints. For example achair can be moved under a table, ho wever excluding its back and not at a corner of the table. 1 If you wonder why the program is called PLEXI this might interest you. In a realistic 30 . picture, objects obscure each other. Simuiaring this effect in computer grapbics requires considerable computing effort, in particular when objectscan obscure their parts mutually. We did notwant to focus on the mathematica! intricacies of this issue which is called hidden surfacé removal. Instead we simply designed the program to picture wireframe furniture only. And in a corny mood we decidedly claimed that the program was still perfectly realistic; it just is custom made for those fancy interior decoraters who furnish entire rooms by plexiglass fumiture only. The room is also continually depicted in a seperate '3D window'. (Of course the window itself is 2D, but a 3D structure is projected onto it.) So through this window the user can view the room from various angles and distances. To obtain a certain view on the room, the user must communicate the desired viewpoint to the computer. Por this, PLEXI supplies two very different control panels, on which this paper will focus. 2.2 The parameters To appreciate the PPP-issue in this example some technica! details should be noted. The program is written in Pascal. The programming environment came with several special purpose library modules. One of those modules calculates stereo-metrical projectionsof 3D wire y Yaw frame structures onto the screen. It forms an opaque module that is not accessible by the prograrnmer. In order to generate a picture the module neects two kinds of input information: J--X J-eP,oh - A complete description of the scene by coordinates in a virtual3D space; - Several parameter values to determine the view on the scene. Z (out of page) Rol! Fig. 2. The axes and rotations as defined by the library module. These parameters are of key importance to the current discussion (see table 1 and figure 2). Table 1. The parameters that determine a view. Parameter Effect Reference Yaw Rotate camera around ... y axis Pitch Rotate camera around ... x axis Roll Rotate camera around ... z axis Sc ale Closeness of camera to ... center of space Move X Displace rotation center along ... x axis MoveY Displace rotation center along ... y axis MoveZ Displace rotation center along ... z axis 3 1\vo different views 3.1 The slide bars control panel Seen from the viewpoint of a programmerit would be most obvious to ex pose these teehoical parameters directly to the end user. This approach results in the control panel in tigure 3. There is a seperate slider for each parameter. The yaw, pitch and roll vary from -180 to +180 degrees. The move parameters vary from-100 length units to + 100 length units. Using this control panel we experienced some problems. Pirst of all. the narnes of the parameters poorly explain their effects, ~SET UIEUJ ~ especially for non English native speakers. And any ( Switch panel ) ( Bock to defatJ Yaw Roll scale l§Jim,lmmm:m~ better narnes or translations are hard to imagine. But most confusing of all, the different parameters interact mutually. E.g. the effect of rotation around y axis depends on the displacement of the center of the space. This is of course stereometrically coherent but can be psychologically hard to interprete and predict. The control panel does not support a consistent mental model about the relative effects of each of the parame-ters when combined. Related studies too suggest that users often find cummulative rotation around all three axes hard to monitor and control (Chen, Mountford and Sellen, 1988). And problems become only worse when the room content is so complex that each redrawing takes considerable time (e.g. two seconds). Users wil! resort to larger parameter changes at a time. As a result the rotations and displacements in the picture will be too jumpy to communicate the effect of the changes directly. Moreover the user has to Fig. 3. The slide bars panel. campose a eertaio viewpoint by setting the seperate parameters one at a time. Intermediate pictures are often confusing. In short the user gets easily lost in trial and error behavior without proper feedback. 3.2 The icon based control panel We decided to change our view from the programroers' to the users'. Por the moment we forgot all we knew about the 3D Pascal module. Another control panel was designed from scratch that would make more sense to the user. The new desig~ wasbasedon two general interface design concepts: methapars and direct manipulation. Metaphoric interfaces explicitly use everyday objects and events from outside the computer realm to support computer based tasks. A classic Viewfi~ Fig. 4. Throwing a look. Fig. 5. The first draft. example is the desktop metaphor of the Xerox Star system (Smith, Irby, KimbaH and Harslem, 1982) and the Apple Macintosh. Metaphor guides the construction of a mental model about the functioning of the computer program. Metaphor is a powerlul resource for design because it exploits prior knowledge; by analogical reasoning even novice users will know what to expect from the system and how to respond to specific situations (Gardiner and Christie, 1987). A second and strongly related design concept is direct manipulation. It requires visual representation of the domain of discourse (e.g. by metaphoric icons). Then the user can perfarm operations by pointing, dragging and editing objects. The benefits of direct manipulation as opposed to cammand lines or menus, include rapid leaming, less errors, high user satisfaction and encouragement of explorative user behavior (Schneidermann, 1991). The new PLEXI control panel involves direct manipulation of methaporic icons. It introduces a third-party perspective in which the user sees herself, watching a part of the scene (see figure 4). The user canthen simply displace the icon representing herself to the desired viewpoint and direct her look towards a eertaio spot. SET U lEW ( Switch panel ) [ Baok to defieJ l J Pitch [~FJ!ilmll!!ili!ll@ Rou l~lmmlnnl!mm:l~ Scale l~Hfiil1ml!!i!Wä<>J MoveX [~[!!!lil 1m l!i!iii!i[~ MoveV l~liii!iil nnliii!ii!l<>) Movez l~Fii![nn]m!i!iilcg Vaw [~l!i!i![tm l!iiiiiii@ Pitch I ~llnu mmmmm@ Roll l] Move V (~Jliiii! u!i)i!i!ii!i@ MoveZ l~lii!ii!ltm li!Wi!i(i2 Vaw l~[iiii!!W!îilnalit<>J Pitch @l[nn li!!i!!!i!il!!i&f Rou ~Fi!ilnnliii!iiii~] Scale @l!i!Hinn[!!!!!!!i@ Movex @I!W!il,lllliiiiiiiil~ Move V @liiiiil nn[!i!!l!!!@ MoveZ I~FH tlllb!lii!@ vaw l~lmllmnmmmii<>J Pitch (~IJ nnf!il!!i!ii!iii<>J Roll @l:m!ilmûi!i!WH<>] Scale J~lnnJHiii!ii!l!i!@ Movex @l:m:mmn 1111Ji!&l Movev l~lmm:mm mll!i!<>l MoveZ L4ilmm:mm 1111 m<>J to its shadow determines the y-displacement. Thus Viewfinder represents the center of space. The direction in degrees from Eyecon to Viewfinder along the floor determines the amount of yaw. The difference in heigth (icon-to-shadow distance) between Eyecon and Viewfinder determines the pitch. The roll parameter is not implemented but could be assigned to local rotation of Eyecon (by dragging at one of its sharp ends, see figure 8). Scale was calculated as the 3D-Euclidian distance between Eyecon and Viewfmder, multiplied by a beuristic factor. Of course this parameter projection is both Fig. 8. Possible mouse input ofroll parameter. psychologically and mathematically rather unsophisticated, but merely serves as an illustration of the general problem of parameter projection ('3P in 3D'). Table 2 shows in a glance how the visual appearance of the two control panels relate to the changing 3D view. The first three views show merely how seperate parameters effect the view. View no. 4 however serves to demonstrate the clear advantage of the eyecon panel in combined translation and rotation. Would you be able to guess the slide bar settings right to get this peep through the window? 4 Experiment In order to leam more about the usability and leamability of the eyecon panel, a small scale experiment was carried out. This experiment was designed to compare the eyecon panel directly to the slide bars panel regarding leamability by novices. 4.1 Metbod Subjects. Five male and five female right handed adult subjects were tested. Five of the subjects had an academie degree, three of which in mathematics. All subjects were familiar with using a mouse device. They had no prior experience with PLEXI or any other 3D grapbics systems. Apparatus. The experiment was run on an Apple Macintosh LCII computer with color display and a one button mouse. The software (PLEXI itself and the experiment control) was written in Pascal. Accuracy was recorded on line. Task completion time was recorded by a hand held stopwatch. The roll parameter was bidden in the slide bar panel because it was oot implemented in the eyecon panel either. Tasks. Subjects were asked to perfarm a series of matching tasks. On the screen they were presented a plain room seen from a certain position. Then they were shown a picture on paper of a certain desired view and they were asked to match the screen picture in angle and size to the paper copy. Both target and screen picture were wireframe rendered. Subjects decided themselves when the match was sufficient. Each series consisted of four matching tasks: task 1 from view 1 to view 2 (see table 2) task 2 from view 1 to view 3 task 3 from view 1 to view 4 task 4 from view 4 to view 1 Both task completion time and accuracy of match were measured. Accuracy was calculated as the sum of the squared differences between the two views on the parameters yaw, pitch, scale, moveX, moveY and moveZ. Subjects were not allowed to switch to default view settings during the tasks. Design. According to a within subject design, each subject performed the same task series with both control panels. Order of control panel was counterbalanced: half of the subjects used the icon based panel first, the others used the slide bars panel first. This condition was equally divided over sex because gender might relate to ease or style of spatial reasoning. Verifying such effectshowever would require a much broader study. Procedure. All instructions were provided by the test leader. In actdition to a general introduetion to the procedure, screen layout and upcoming tasks, each session consisted of twice the following sections, once for each control panel: - Introduetion to the control panel - Self-discovery with this panel for five minutes - Supplementary explanation by ex perimenter if neccessary - Task 1 to 4 using this panel To make sure that subjects really tried to understand each control panel during the discovery period, they were asked to explain verbally how each icon or slide bar related to the 3D picture. If this revealed serious a priori misconceptions (which was rare) some supplementary explanation was provided by the test leader. Each session took about halfan hourand tasks were hardly repetitive so there was no need for a break. At the end of the session the subject was asked a 1 to 10 rating for the usability of each control panel. 4.2 Quantitative results Performance in style and completion time differed even more between subjects than we already expected. Figure 9 to 11 show the quantitative results. Figure 9 shows that averaged over subjects all tasks were completed fastest with the eyecon panel. The difference is particwar large regarding task 3. Note the relatively large standard deviation. Figure 10 shows accuracy of task result. For the simpler tasks ( 1 and 2) the slide bars gave least error. For the more complex tasks the eyecon panel gave least error. Figure 11 shows that the average rating was higher for the eyecon panel than for the slide bars panel. 1/) i ::::::: ~------~ 0:03:00 c 'E o:o2:oo 5 J: 0:01:00 0:00:00 Task 1 Task 2 Task 3 Task 4 Overall Standard mean deviation Fig. 9. Task completion time averaged over subjects. 4.3 Discussion Different strategies. Completion time very much dependedon choice of strategy. The 3D window always depiets the center of space (which is center of focus and rotation as well). But because this center point is projected onto the 2D screen, it does not reveal its actual 3D position compared to the room. Often the center point only appears to coincide with the target part of the room, and drifts away when rotating the room around. Therefore, using the slide bars panel, a strict strategy should be used to first set the center point in all three dimensions which requires watching the room from at least two orthogonal views. And not until then the room should be rotated to achieve the desired angle of view. Only one of the subjects developed this strategy. When using the eyecon panel there is noneed for such a cumhersome strategy because the 3D position of the center point is explicitely represented by the Viewfinder. Also the two control panels proved to call for very different strategies according to the use of feedback inforrnation. Using the slide bars panel it proved very hard to predict the effect of each slide shift; the subjects continually watched the 3D window to recieve indispensable feedback. But using the eyecon panel users should focus entirely on the control panel itself. While moving the icons around it proved unneccesary to interprete the interrnediate results. It can be even confusing; when no part of the room is between Eyecon and Viewfinder the 3D window is (of course) left almost blank. Thus the control panel itself explains the situation better than the 3D output window. From this observations we can conclude that the eyecon panel has a clear advantage to the slide bars panel when the room content is 5000r---------------------------------4500 +---------------l • Slide bars panell-----4000 ~------.... ; 3500 +--------g ~ 3000 i i 2500 ~ .i!:S 2000 i~ 1500 -g ~ 1000 ~ 500 ~ 0 Task 1 Task 2 Task 3 Task 4 Overall Standard mean deviation Fig. 10. Error in result view averagedover subjects. so complex that each drawing takes considerable time. Task difficulty and completion time interaction. We expected an interaction effect between control panel type and task difficulty. Tasks 1 and 2 did not require moving the focus spot at all. So in the eyecon panel the Viewfinder should not be displaced and in the slide bars panel the moveX, Y and Z slides should not be scrolled. Therefore both panels were expected to be quick and easy for these tasks. The advantage of the eyecon panel was expected to emerge only in tasks 3 and 4. Results for accuracy (figure 10) actually revealed such interaction. The results for task completion time did not, because task 1 and 2 did not turn out to be as simple as was expected, which depended strongly on first action choice. Although tasks 1 and 2 did not require changing the focus spot, subjects still often did. Subjects that scrolled the move slide bars at the beginning of task 1 or 2, were immediately lost in space. Likewise though much less severe, subjects diverted from the right track when they first started moving the Viewfinder first instead of the Eyecon. So a rather accidental first choice resulted often in a large task completion time. As an afterthought we regret that the 'back,to defaults' button was disabled during the experi-ment. 10,--------------------------------9t--------------------------------1 0 Mean rating Standard deviation Fig. 11. Subjective rating of ease of use averaged over subjects. We are primarily interested in the usability of the eyecon panel, so user difficulties with the bars panel will not be discussed in detail any further. The major drawbacks for the eyeconic panel proved to relate to three seperate issues which we will now adress briefly. Dragging technique. In the experiment version of the eyecon panel, an icon can only be dragged along the floor by dragging its shadow. The unnaturalness of this technique proved persistent; some subjects repeatedly tried to drag an icon itself horizontally even when they already rationally knew they should drag its shadow. Moreover, if anicon lies exactly on the floor, it obscuresits shadow. To move the icon along the floor the icon should be lifted first to reveal its shadow. Subjects found this very annoying. Fortunately these techniques are not intrinsic to the eyecon panel. In response to these observations an alternative version of the panel was implemented. In this version a modifier key is added to the scheme. The icon can be dragged along the floor immediately. It doesnotmatter whether it is dragged itself or by its shadow. To move the icon further or closer above its shadow the modifier key must be depressed during the dragging. This scheme, which is only a first and yet to be tested altemative, brings along extra movements (it involves a second hand) but probably will prove more natura!. Relating different views. The second issue is inherent to the used roetaphor of a person walking around an object (as opposed to e.g. turning an object around in your hands). Remind that the control panel shows a static third party perspective in which both viewer and object are represented. So if the room is watched from behind,left in the control panel becomes right in the 3D window. (This also happens when using the slide bars panel.) This extreme example demonstrates that the user must be able to relate the directions in the panel to the directions in the 3D window. Fortunately most subjects succeeded in this quite well. Cluttering windows. PLEXI supplies a 20 ground plan window and a 30 view window. The eyecon control panel inroduces yet another view-like window. Too much windows can be cumbersome. Therefore we think it would be wise to integrate some of these windows. The control panel intrinsically can not coincide with the 3D view because a viewer can not see herself standing in her own view on the scene. But the control panel could coincide with the ground plan, simply by allowing the user to create and rearrange the contours of furniture directly on the floor in the control panel. 5 Conclusions Regarding to a small scale test and opposed to the slide bars panel, the eyeconic control panel clearly shows considerable impravement in leamability and ease of use withoutlossin accuracy. Subjects generally rated the eyecon panel higher in ease of use. Observations gave rise to several ideas for further impravement of the eyecon panel such as change of dragging technique and integration with the ground plan window. Whenever one writes a computer program based on existing library modules it can be very tempring to duplicate the internal organization of parameters directly to the user interface. In general and certainly in our example this can lead to communicational problems. Technica! parameters can differ from psychological entities and they can interact in difficult to predict ways. Then a deliberate change of view can be necessary to design a more user oriented organization of parameters. References Bier, E. (1990) Snap-ctragging in three dimensions. In: Proceedings Workshop on lnteractive 3D Graphics. ACM/SIGGRAPH, 183-196. Chen, M., Mountford, S. J., & Sellen, A. (1988). A study in interactive 3D rotation using 2D control devices. Computer graphics 22, 4, 121-129. Evans, K. B., Tanner, P.P. & Wein, M. (1981). Tablet based valuators that provide one, two or three degrees of freedom. Computer Graphics, 15, 3, 91-97. Gardiner, M.M. & Christie, B. (eds.). (1987). Applying cognitive psychology to user-interface design. Chisester: Wiley. Schneidermann, B. (1991). A taxonomy and rule base for the selection of interaction styles. In: B. Schakel, S. Richardson (eds.) Human factors for informaties Usability, 325-342. Cambridge: University Press. Smith, D.C., Irby, C., Kimball, R. & Harslem, E. (1982). The Star user interface: an overview. Submitted to the AFIPS 1982 National Computer Conference. Veniola, D. (1993). Facile 3D direct manipulation. In: Proceedings Interchi '93, 31- 36. 0 V Partners Dialogue as a Metbod for Non-Pormal Problem Solving lgorChmyr Instltute of Low Temperature Engineering and Energetics 1/3 Petra Velikogo st., Odessa 270100, Ukraine Abstract. Man-machine dialogue can be used not only for interface organization but also as a "solver" for those problems that consist of a set of beuristics united by "expert logic". Using a special kind of relationship "stimulus-reaction" we can build a scenario that imitates the expert work. The dialogue problem solver contains a scenario that is an access metbod to the set of stimuli as described. 1 Dialogue imitation of expert work and "dialogue solver'' organization Practically all the methods used in knowledge delivery or educational systems are non-formal, being invented and realized by experts. We can imitate the expert work by computer if we are able to present it in a dialogue form which will satisfy the following requirements: - the computer remains an active partner during all the dialogue process and at every step it shows the passive partner a stimulus according to the scenario; - every stimulus contains a direct or indirect form reaction that is expected; - for every applied problem in scenario building there are a finite number of stimuli and reactions. Therefore, during the dialogue imitation of the expert work, the computer had to engender an aim-directed sequence of stimuli in accordance the scenario. We will understand by scenario the metbod that allowed us to generate the ordinary stimulus after the current reaction was recognized. One of the key ideas is, that a scenario does not "calculate" the ordinary stimulus, but finds the stimulus in the certain direct access memory -StimulStore. The scenario has to support two functions: - to store an "expert logic" of problem solving; - togenera te a current stimulus index for StimulStore. 43 44 And in the previous passage the scenario is a eertaio access method, which will be marked as DiAM. A various number of applied scenario analyses allows us to make a condusion that in general it must have a netwerk organization. Every node of this netwerk is corresponding to the dialogue process step, and every step- to the elementary dialogue contact act: - generation of the stimulus and transmittion to the passive partner; - reaction perception; - reaction recognition and current stimulus index definition. DiAM netwerk may be described by various mathematica! methods (Petri nets, for example), but it is more constructive to describe DiAM as an abstract relational database. This description bas two important advantages: - it does not prevent a complicated node elements classification to engage; - it converts the DiAM into data which is simpte to edit. The dialogue problem solver is biult on DiAM base and contains the following components: - stimuli permanent memory (a direct access memory that stores an indexed set of stimuli descriptions); - dialogue process temporary memory (this memory stores a sequence of passive partner reactions, which is used by eertaio type of DiAM nodes); - dialogue access method (database that stores a problem solving method. For every input reaction DiAM provides an access to a single indexed element of StimulStore); - dialogue processor (a resident process that provides step by step scenario interpretation); - demons-procedure library (set of external procedures that realizes a function caooot be covered by DiAM) 2 Stimulus structure The active partner generated stimulus and the answering reaction are in close relationship. Every stimulus predetermines several reactions that take place in that stimulus. This assumption is valid with the question-answering logic results. In accordance to this logic, every question predetermines the answer. The stimulus is interpreted via two channels: the video-channel and the audio-channel. I t exists as a chain of video and audio-objects. It is natural to assume that one of the video-objects can figure as a reaction. In this meaning, all video-objects divide into three classes: - an object that is not a reaction. This object is characterized by it's place in the chain only; - an object that is a direct reaction. This object cao be chosen as a reaction. It is characterized by its place in the chain and individual identificator; - an object that is a mediate reaction. After being chosen this object must be added by data. It is characterized by it's place in the chain, individual identificator and meaning. The video-object visual expression is given by two grope of descriptions: - attributive descriptions; - non-attributive descriptions. Non-attributive descriptions are a binary file which is associated with the video-object. Attributive descriptions give the mode of this binary file interpretation (video-object location on the monitor, dynamic effects, animation and so on). 3 Conclusions A complex approach to the building of a system that is oriented to non-formal problem solving by means of the expert dialogue work imitation. The probable application sphere of this approach-expert work imitation is: knowledge-transmission, education and design. A system which is described on the level where technica) implementation of its elements is inessential, but at the same time this description reflects a modern computer archi tecture. 45 The Design of Interacting Agents for U se in Interfaces David Connah Institute for Perception Research, P.O. Box 513, 5600MB Eindhoven, The Netherlands l.O. Introduetion The central topic for this summer school is man-machine communication in educational systems. My lecture will not refer very much either to communication or to educational systems but I think I can justify its subject matter because of the ioclusion in the title of one other smal! word. The full title of the summer school is 'The Basics of Man-Machine Communication for the Design of Education Systems' (my emphasis) and it is my intention to describe a kind of agent which I think has broad potential in man-machine communication generally and in educational systems in particular. I will not attempt a rigorous definition of what I mean by an agent but I will rely rather on the common-sense, everyday meaning that people give to the word. Broadly speaking agents 'do' things; they are (apparently) capable of acting autonomously. I would like however to repeat the distinction made by Laurel (Laurel 1991) between two varieties of agent. Agents can be seen as acting on behalf of someone else; for example, an estate agent who sells your house for you or an insurance agent. They can also be seen simply as acting in their own interests and most 'biologica!' agents (people, animals) are in fact acting in this way most of the time. Much of the literature has dealt with the first kind of agent - Apple Guides come to mind immediately - but the second kind is in fact more general and subsumes the first kind. In what follows I am thinking of this more general kind of agent; The use of the word agent is not new in Artificial Intelligence (AI). I believe that it goes back to the very beginnings of the discipline. What is more recent is the notion of situated agents. If there is a date at which people in AI began talkingabout situated agents that date is probably about 1984. Many of the ideas which contributed to this idea are however much older than that and have often been drawn from completely different disciplines such as sociology, anthropology and philosophy. 1984 (if that is the right date) was when people in Al began to realise that some of the problems that were causing such trouble in AI might be eased (or even go away completely) if a different approach was tried. Let me start by talking about models of action. 47 48 2.0. Models of Action Until a few years ago, it was almost universally the case that the actions of agents were described and implemented in terms of a planning model. Roughly speak:ing this means that agents were thought to have goals. In order to achieve these goals they constructed intemal models of (part of) the world and then, by reasoning about these models, they constructed a plan of action. In its simplest form such a plan is a sequential list of actions to be taken by the agent. Once the plan had been constructed it would then be executed by the agent usually resulting in some extemally observable behaviour or actions. In this model goals, plans, intemal representations of the world and reasoning go hand-in-hand to cause the behaviour of the agent. This is perhaps not the whole story; the agent, on this theory, has in some sense to intend to act before action becomes possible. This gives rise to a belief-desire psychological model to explain intention. All of these things toa large extent hang or fall together. Nowadays, however, this is not the only model. There is another theory called situated action theory (Suchman 1987). In this model an agent acts in a way which is 'appropriate' to its situation. Here 'appropriate' means in a way which evolution or leaming has engendered in the agent to ensure its survival and the satisfaction of its interests. The 'situation' is a potentially complex combination of extemal and intemal events and states. In this model there is no suggestion of planning, desires, beliefs etc. but there are other ideas that are implied by the model. The most important of these is that agents have to be closely-coupled to their environment. This is so because it is only through their direct perception of the world that they can be aware of their situation and only by this awareness that they can act appropriately. Thus perception is essential for this model but the model also has implications for such things as representation, emergent behaviour, social behaviour and skill. To some extent the planning model and the situated action model are extremes appropriate to different circumstances. The planning model was developed when AI was seen to be mainly about formal problem solving (chess playing, theorem proving). Situated action is a more recent theory which is principally concemed withall those aspectsof an agent's behaviour which relate to the mundane business of surviving, moving about, performing fairly routine tasks and generally 'living a life' as Chapman puts it (Chapman 1991). lt is unlikely that the simple planning model described here is in fact of practical use in any real situation except, perhaps, such relatively marginal activities as playing chess; the world in general changes too fast and too unpredictably to allow plans of this sart to work. People do, of course, plan but their planning activity is much looser than AI planning and only works at a reasonably large granularity. For example one might plan to go to the airport by car but one cannot plan each move of the steering wheel or each gear change since these are dependent on the as yet unknown behaviour of other road users and road conditions. This example hints at the way in which a loose kind of planning model and a situated action model are complementary to each other. Many of the things that we want to do with agents can be done by skilied (as opposed to 'intelligent') agents and situated action theory is very well suited to the description and implementation of such agents. The rest of the lecture is about situated agent theory and about some of the implications that the theory has for the design of situated agents. One thing that may have already become apparent is that we shall he talking a bout behaviour rather than knowledge (as is more usual in Al). There is not time to discuss this here but some indication of the reasons behind it can he found in (Wavish and Connah 1990, Maes 1993a). I will start to describe some of the implications which the theory has for the design of agents by discussing, in the next section, the particular kind of representation that is a common feature of situated agents. 3.0. Representation Representation has always been a central issue in Al. lf you see intelligence as being a bout problem solving and if your strategy for obtaining a solution is the use of formal methods operating on an abstraction from the problem in question then much hangs on choosing the right representation. An interesting early example of the importance attached to representation in this context can be seen in (Amarel 1968). There are problems with creating such an intemal representation. First there are the overheads associated with the construction of the model. Then there is the difficulty of ensuring that the model is always up-to-date. This is particularly difficult if there are other agents in the world causing it to change in unpredictable ways. (lt is a remarkable fact that for many years AI was concemed only with situations where there was only one agent in the world). Finally, given that the model is properly maintained, there is the computer power necessary to cope with searching, inferencing and planning and this bring us up against not only the practical problems of resource limitations but also the more serious problems of tractability in principle (Chapman 1987). Is there an alternative kind of representation that doesn 't come up against these problems? The answer provided in the situated agent paradigm is to say that the world is the best representation of itself (Brooks 1991). In this case there are no overheads for model construction and maintenance, and the model is always accurate as well as being as rich as only the real world can he. But there is more to it than this. When an intemal model has to he constructed there is no way in which the agent can know at the time of construction which parts of the model are going to he of importance and which not. Indeed as this is likely to change with time, any information that could conceivably be required must be embodied in the model. The model is very 'flat'; nopart is intrinsically more important than any other part. lf, 49 50 for example, the agent is in a lecture theatre containing many chairs, each chair will have to be individually represented at all times in as much detail as could be required at any time. In the 'no-internal-representation' case the agent need only, at any given moment, perceive that part of the world that is currently of interest. In other words there is a focus of auention. Let me try to clarify some of these ideas with the help of an illustration1• The task at hand is a task in the blocks world. The agent is presented with a table containing pil es ('columns') of blocks. Each block has a letter on its front face. The task of the agent is to use the blocks on the table to create a copy of the rightmost column on the table. The world in which the agent has to perform its task is shown in Figure 1. Figure 1. Faced with such a task an AI programroer would typically try to deal with it in terms of planning (Rich 1983, p.259, Nilsson 1980, p.275). This would involve creating a representation of the world in the form of symbolic descriptions and a set of operators to act on those symbolic descriptions. There would also be an explicit symbolic description of the goal of the program or agent and some reasoning process by which the agent would discover how to change the current state of the world into the goal state. The planning paradigm also requires some metbod of detecting when a solution has been found and, in some cases, a method of recognising when a solution is nearly correct (Rich 1983). A partial description of the above world in the style of a planning program might look something like: ON(c,a) CLEAR(t) 1. Chapman in (Chapman 1991) said "I believe that implementations in Al mostoften serve as illustrations of approaches" and "Approaches are leamed in part from the practice of reimplementing illustrative programs". The work described hereis taken from my re-implementation and extension of Chapman 's Blockhead program (Chapman 1989). ONTABLE(s) There would also be a robot arm which could be used to piek blocks up, putthem down, and stack or unstack them. The arm (hand) can be empty so we can have states and operators such as: ARMEMPTY PICKUP(k) STACK(v,h) and so on. The goal state might be described as: CLEAR(f) 1\ ON(f,r) 1\ ON(r,u) /\ •.• I\ ON(k,e) 1\ ONTABLE(e) (All of this ignores that, as here, there might be more than one block marked with a particulat letter). A simple plan is then a list of operators taking the original state into the goal state step by step. The situated action approach to the same task differs in every respect from that of planning. There is no symbolic description of the world; there is only the world itself. There is no reasoning process2 but only actions which are appropriate to the situation of the agent. In the current implementation at least, there is no explicit goal; the designer has an explicit goal but the goal-like behaviour of the agent is emergent from its atomie situated actions. (In the situated action paradigm, theré is an important distinction to be made between the point of view of the designer or observer and that of the agent. This distinction is aften blurred or non-existent in planning agents). Situated action rules may appear superficially to be similar to operators with pre-conditions and post-conditions but whereas the latter workon the intemal representation, the situated actions of the agent form part of a loop linking the agent directly into the rest of the world via perception and action Perception in this example is simulated by intermediale level vision. This simulation is accomplished by means of markers (Chapman 1989, Chapman 1991, Uilman 1984). A marker is a device within the visual system of the agent which can be used to (detect and) mark specific features in its visual field. For example the bottorn or the top of a column of blocks. It can also be used to track some ongoing activity such as marking the place where the next block is to be placed. This kind of marker merely marks a position in the visual field of the agent. It is the 2. In a sensethereis a reasoning process but it is 'bUilt-in' to the actions which are determined by the situation. This kind of implicit reasoning is a comrnon feature of skilled activity. 51 52 responsibility of the agent to get the marker to the right location in the first place. Certain markers can also return directly the identity of a block (i.e. the letter on its front face) as was mentioned above. Markers are the only visual interaction that the agent has with the world. In other words if there is no marker at any given time on a particular spot the agent cao have no knowledge (at that time) of what is there or even of the existence of the location3. If sarnething happens in a location where there is no marker the agent will be totally unaware of it. This, in effect, constitutes a focus of attention for the agent and once again it is the responsibility of the agent to eosure that its markers are handled in such a way that it is always attending to those things that require its attention if it is to perform its task satisfactorily. The manipulation of markers is accomplished by the situated actions of the agent. By manipulating a small set of markers it is possible to find blocks, decide which ones should be moved and subsequently keep track of them in such a way that a copy of the right hand column of bricks cao be made. It is oot a difficult matter to extend the program so that if the construction of the copy is interrupted either maliciously or by some 'natural' event like the pile collapsing, the agent can still reeover from the situation using exactly the kinds of actions it used in constructing the pile in the first place. This kind of recovery is difficult or impossible when using an intemal representation but then it is never seen as necessary since nothing cao interfere in the only world it knows: the intemal representation itself. This is why the blocks world, as usually described, is so unrealistic. 3.1. Summary Here is a brief summary of the advantage of dispensing with intemal representations. • the representation does oot need to be updated. The world is never out of date. • no representation of the world cao be as rich as the world itself. The accessibility of this rich resource is limited only by the perceptual abilities of the agent. In particular temporal and spatial relationships are automatically maintained in the world. • the representation is automatically shared between agents and cao form the foundation of communications between them. This is by no means the full story on representations; in particular, I am oot suggesting that agents never use internal representations. I am simply saying that for many of their activities intemal representations are oot only oot necessary but may be counter-productive. I shall have a final comment to make about representations in the sec ti on on the future of situated agents. 3. No te that markerscan also be used tomark 'virtual' objects such as the gap between two objects or the space into which sarnething has to be placed. 4.0. The Social Nature of Agents It was remarked earlier that much of AI has been concemed with the operation of a single agent in a passive world. The apparent assumption has been that once one agent had been successfully designedit would simply be a question of actding other rather similar ones. Although one can think of applications where an agent acts in isolation, the majority of cases, and particularly those falling under the rubric of this Summer School, involve cooperation between agents. Even in cases of apparent isolation (e.g. a Mars rover) the behaviour of the agent will have been learned or given to it, and its concepts will have been acquired, in a multi-agent society. Without that society its actions would have no purpose and it would not have formulated the concepts which appear to drive it. In other words, the science of autonomous agents is, in an important sense, a social science. This idea of a society of agents is an extension of the idea of situated agents where account is taken of the fact that the agent is not merely situated in the physical world but specifically in a world where there are other agents with which it has to interact The behaviour of real people in a society consists for the most part of complementary activities: questions require answers, one kind of gesture elicits another, handing over an object triggers the taking of the object and so on. Such actions only have any significanee in a social context. When we are designing agents which are going to interact with one another we have to bear this in mind. We have to design the actions of the agents so that they can engage in this kind of mutually supportive activity. An example of a specifically social activity is provided by cooperation. Here there is an implicit communication between agents which is effected by the actions of the agents in the world. Of course in one sense all communication teavels via the world (sound, light etc.) but I am referring to sarnething different in this case. Suppose two bricklayers are building a wall. As one of them lays a brick in place the shape of the wall is changed and the other bricklayer is guided in his placement of the brick by the new shape of the wall. This is a very simple example but other examples can be quoted. For example the cooperation between two agents assembling sarnething can be achieved in a similar way (Hickman and Shiels 1990). All that is required is the ability of the agents to perceive the current state of the wall or the assembly and to have actions which are appropriate to that state. The significanee of each action is dependent on the state of the wall or assembly. I will illustrate this process with examples suggested by a demonsteation program (Wavish 1991). This program is in the nature of a game or entertainmentbasedon the idea of a sheep dog rounding up sheep and driving them into a pen. The demonsteation is a simulation of a system containing seven agents: a sheep dog, five sheep and a shepherd. The idea is that the dog should round up the sheep and drive them towards the shepherd. For this to happen both the sheep and the dog must, in some simple way, be able to perceive each other. As far as the dog is concemed, its 53 54 surroundings are notionally divided into eight sectors centred on the dog itself. It has a rudimentary 'vision' system which informs it of which octants cantaio sheep. (This is all; it doesn 't know how many sheep or how far away they are). The sheep on the other hand are 'aware' of the presence of the dog when it gets too close for comfort. Given this rather basic perceptual ability, what behaviour should we write down that will cause the dog to round up the sheep and herd them towards the pen? The following four behaviours are sufficient: • if the dog is running along with the sheep on its right hand side and there are no sheep in the dog's front-right octant, it should turn right until sheep appear in that octant (similarly if it is running with the sheep on its left). • if, at any time, the shepherd appears in one of the dog's three front octants, it must reverse its direction of traveL • if the dog approaches a sheep too closely the sheep must move (run) away from the dog. • the sheep must avoid colliding with each other. The result of these four behaviours is that the sheep are compressed into a flock and that the dog weaves to-and-fro behind the flock, driving it towards the shepherd. This behaviour cao be seen in Figure 2. 0 ..... . ha : ..... ==············ .•. u """ y ·· .... ti _.; , .. /'l• ··,. ··, ·······;· 9····~.:~:-.:,"~.rs.,/' ·~~~. !·/ /// .... ·, . .. ··· .. .. · .......................• ~· ·· . .......................................... Figure 2. 0 0 0 0 0 0 0 0 0 0 lgnoring the details of the diagram, one cao clearly see the tracks of the sheep and of the dog as it weaves to-and-fro. What points are illustrated by this fragment of the demonstration? 1) The behaviour of the agents is situated. In this simple example it is easy to write down, for example, what conditions must hold forthesheep to run away from the dog. 2) The four behaviours described are (indeed have to be) present concurrently. It makes no sense tothink in termsof priorities or sequences forthese behaviours; any or all of them must be triggered in the appropriate situation. 3) The flocking and herding of the sheep is not achieved solely by the dog's behaviour as one might assume but requires the complementary behaviour of the sheep. It is, in this sense, a kind of social behaviour in which all the agents (dog and five sheep) trigger behaviour in each other Ieading to stabie behaviour of the social group as a whole. The last point really contains the essence of the complementary nature of social behaviour and soit is worth emphasising. An analysis of what is happening when a sheepdog is used to round up sheep might suggest that the dog is forming the sheep into a small group (a flock) and then driving the flock towards the shepherd. However, this analytica! statement is couched in terms which are appropriate to the observer of the scene; they are, as Maturana would say (Maturana and Varela 1970), descriptive terms in the observer's domain. In this instance, because we know exactly how the program has been written, we can say categorically that this description is misleading. There is no sense in which the dog is actively forming a flock and the sheep are passively allowing themselves to be herded in this way or that the dog is driving the sheep towards the shepherd. It appears this way because it is the purpose of the designer that this should happen and thus it is described by the observer in these terms. The situation is much more symmetrical between the dog and the sheep. All that one can say objectively is that the dog has one set of behaviours and the sheep another and that the result of the interaction of these behaviours in the system as a whole is that dog and sheep move in a way which satisfies the designer's purpose. Notice that no mention has been made of the goals of the dog or the sheep or of their beliefs or desires or, indeed, of their plans. These concepts are unnecessary for the implementation of the various agents in the scenario. A belief, in this context, is not an explicit sentence or state in an agent but a concept used by an observer to make sense of the behaviour of the agent from the observer's point of view. Of course it is possible to verbalise plans, goals and beliefs but this is simply the agent observing its own behaviour and even then they may not fulfil the role assigned to them in most AI literature. Plans, for example, as (Suchman,1987) puts it, are just one resource for action amongst many. Except in the simplest cases they can not be a list of actions to be undertaken sequentially in order to achieve a pre-determined goal. 5.0. Emergent Behaviour It has been observed by a number of people (Wittgenstein 1953, Dreyfus 1992) that 55 56 complex human behaviour cannot be explained by supposing that the person is (even unconsciously) obeying a fixed set of rules. There are difficulties, for example, in the handling of exceptions or in the ftexible interpretation of the rules. The latter problem can lead to an infinite regress because of the need to have rules for the interpretation which are themselves subject to interpretation. I do not know if there is a 'solution' to this problem, but there is an approach which I think is worth pursuing. When watching the demonstration of the sheep and sheepdog game, observers find it is natural to describe it in terms of words or phrases such as 'ftock', 'rounding up', 'herding the ftock' and so on. None of these concepts is explicitly represented in the program but they are ascribed by the observer to the behaviours of agents in the system. Furthermore the apparent behaviour that they are descrihing is, as was pointed out earlier, behaviour which is emergent from the interaction of other behaviours such as running towards the sheep or running away from the dog. I do not know whether this kind of behaviour is what others mean by emergent behaviour - there seems to be a considerable degree of confusion as to what it is or, indeed, as to whether it exists at all - but whatever it is called it is of considerable interest for two reasons. In the first place it seems to be a way of achieving a desired behaviour e.g. rounding up sheep without explicitly programming it and in the second place such behaviours (the 'ftock') can be very stable. Going back for a moment to the non-explicit way in which this is achieved, I think that trying to get this kind of stabie behaviour by explicit means would have been very difficult. One would have needed some kind of definition of ftock which would have come up against precisely the kind of objections made by th9se who object to rule-following: if its size were slightly larger than that proposed in the definition it would technically not be a ftock any more or, if the shape were not sufficiently circular (say), it could no longer be called a ftock within the program and so on. When we ascribe behaviour these problems don 't exist. Of course we are still using rules but these rules are, as it were, at a different and sim pier level; they are rules about the motion and the perceptual acts of the dog and sheep and not about such abstract things as rounding up. The rules at this level can be quite rigid without theemergent behaviour to which they give rise appearing to be rigid also4. The stability of the emergent behaviour in this case is important The overall behaviour of the group of agents is not predictabie in detail {although it is deterministic) and different starting positions of the agents lead to sequences which, in detajl, are completely different. However the behaviour at the level of the ftock (and this indeed is what leadsus to ascribe the term 'ftock') is extremely stabie and robust; it can be deliberately disrupted (e.g. the player can piek up and move the dog or any or all of the sheep) and will fairly rapidly return to the stabie ftocking and driving behaviour. This combination of stability with lack of any detailed 4. One can imagine some of these lower level behaviours, intheir turn, being emergent in the above sense. I am ignoring that here for the sake of simplicity. predictability is characteristic of a chaotic system and we might speculate that 'ftock' is the name we give to an attractor in such a system. We have not as yet pursued this idea any further. The applicability of chaos theory to agents has been discussed by Kissin (Kiss 1991). 6.0. Skill The idea of skilied agents has already figured in this lecture. In fact one of the changes in viewpoint that I would like to argue for is that agents should primarily beseen as skilied rather than intelligent. There are at least three reasans for this. The first is that skill is in many ways a more straightforward concept than intelligence. There have been endless debates about what constitutes intelligence and how it should be measured. Skili on the other hand is not so controversial, at least at the level of simple physical skills. We all recognise it and it is perfectly possible to acquire skilis by well-defined methods of training. Measurement of this type of skill in particular instances is also relatively straighûorward. A second reason for prefeering the idea of skilied rather than intelligent agents is that for many, perhaps most, of the things we want agents to do in real applications, skill is what is required. This fact has been obscured by the tendency in the past for AI to concentrate its attention on those highly intellectual pursuits that farm only a tiny proportion ofhuman activity. Al has always had difficulty with common sense, for example, because it has tried the approach of trying to formalise common sense knowledge and then treating it in a rational way. In fact common sense is what we leam from our embodied existence in the world and in this it has much more in common with skill than with intelligence. Agents in the interface may not require great reasoning powers but rather skill at the job and some smal! degree of common sense to avoid gross and, to people, unintelligible mistakes. The final reason for concentrating on skill is that one can speculate that reasoning is based on skilied activities in the real world and is not sarnething that is pursued by people in a completely abstract way (Chapman and Agre 1986). Let me give just two examples to support this last speculation. Solving probieros and puzzles of various kinds has always formed an important part of the subject matter of mainstream AI; missionaries and cannibals, monkeys and bananas not to mention the prisoner and his perennial dilemma all figure prominently in the literature. But what about sarnething like a jigsaw puzzle? It seems to me fairly clear that an approach to doing a jigsaw puzzle which relied on reasoning with a forma! representation of the shape of each piece tagether with a forma! description of the fragment of the picture appearing on the piece would be a very difficult way of trying to solve the puzzle. This is clearly not what people do. As we discussed at some length in the section on representation, in this case the puzzle exists in the world and does not require a complete intemal representation on the part of the agent. The point I am trying to make here however is that whatever reasoning takes place in solving the puzzle also requires the agent to be able to see the pieces, to handle them, and to move them, aften actually trying them out befare deciding 57 58 whether they fit or not. It is the strong physical element in the reasoning which I want to emphasise. The other example I wish to give concerns the solution of mathematica! problems. Here, at first sight, we have the prototypical intellectual activity. Surely abstraction and formality are the very essence of mathematics. And yet when one is actually doing rnathematics there is a strong element of manipulation; terms are grouped together, moved from one side of an equation to the other or rearranged in other ways which are as much a product of skill and experience as of abstract reason. We talk about seeing a solution and a computer program which has no notion of seeing, touching, or moving will, I submit, be considerably handicapped when trying to tackle this kind of problem. Wavish (Wavish 1993) has written a program which demonstrates precisely these kinds of skilis in an agent. His agent plays the game of Tetris. In this game (if you are not familiar wjth it) blocks of various shapes appear to fall under gravity and land either on the 'ground' or on top of blocks which have previously fallen down. The aim of the player is to manipulate these blocks as they are falling either by rotating them or by moving them sideways ( translating them) in such a way as to fit them into holes in the already existing heap of blocks. If the player does this successfully (by which is meant by completely filling rows ofblocks) then he or she scores points. Additionally, when a complete row is achieved, the row is deleted and the general level of the heap goes down. It is an important part of the strategy to keep the height of the heap down since otherwise it reaches a level at which the game is terminated. The game is shown in progress in Figure 3. Figure 3. (The other icons visible in the diagram represent the various markers that the agent uses in playing the game. See the section on representation). The agent plays the game in a situated manner. It scans the heap (hence the 'eye' icon) looking for certain pattems in its top surface and it then applies rotations and translations to the block depending on which block is falling and which pattems it finds. It is important to realise that within the agent there is no representation of the heap of blocks; the agent plays rather as one might do the jigsaw puzzle described above. In fact the agent does not scan the falling block also but this would be an obvious extension of its behaviour. The game is actually rather more complex than I suggested above since one gets more points if two, three or four rows are completed simultaneously. Thus it sametimes pays to defer an obvious move in the hope that a later block will complete several rows. The impression that one gets on watching the agent play is of some (fairly smal!) degree of intelligence but in fact nothing that we would normally think of as intelligence is involved. Most of the time, although the agent is by no means optimised, it seems to choose a reasanabie Iocation and orientation for the block. At all events the scores it achieves are roughly speaking an order of magnitude better than my best score. I would not like to think that this was a true reftection of our relative intelligences although I can bear to think that it is more skilful than I am. 7 .0. The Future of Situated agents Research on situated agents has not been going on for very long. The theory is still patchy and there are few commercial applications extant. Nevertheless, it is an area which is developing fast and which seems to have great potential. An attempt has been made in this lecture to introduce the topic by discussing some of the important issues but how will the theory de vel op and what kinds of application can we expect in the future? Probably the hottest topic at present is that of learning in situated agents. The Tetris agent described in the previous section was hand-crafted; the various combinations of block shape and heap pattems were chosen and refined by the programmer. Whilst this is feasible for smal I programs it would constitute a bottle-neck in the writing of larger ones. For this reason it has been suggested by several people that it will be necessary for agents of this kind to be learning agents. Work in this direction has been done by Maes (Maes 1991, Maes 1993b) and Chapman (Chapman 1991). This workis still in its early stages and Chapman, in panicular, has reported that his attempts to harness backpropagation methods has been disappointing. He concluded that in future it would be necessary (because of the situatedness of the agents) to look at learning methods which are much more directly linked to the ways in which the perceptions and actions of the agent are effected. As well as being practically motivated there is a strong theoretica! interest in situated agents which can learn. It is typical of the situated approach that it engenders Iw listic solutions; it seems very likely that a proper treatment of learning will be impossible without simultaneously developing theories on perception and memory (Chapman 1991, Edelman 1987). This is in clear contrast to the mainstream AI approach which would (indeed, has) separated out these functions and treated them independently. This is a difficult task. However not only does the situated approach require such holism, it also makes it possible by virtue of the way in which agents are embedded in their environments. I expect this area to be one of the most exciting areasof research in the next few years. The discussion of representation in the lecture was confined to those situations in 59 60 which the world of the agent could be used as the only representation required by the agent. This was an important insight made in some of the early workon situated agents (Brooks 1986, Agre & Chapman 1987). However it is unlikely to be the whole story; the issue of representation goes beyond this. There is, for example, the particularly interesting situation in which dynamic external representations (e.g. speech, drawings, writing) are used by an agent as part of its current situation. As Clancey says (Clancey 1991): "We don't know what we want to say until we say it." There is an enormous amount of work to be doneon this question of representation; the most that can be said of work to date is that it has challenged the accepted view about internal representations and opened up the whole field to a radical re-appraisal. There are still, as it were, two schools of thought in AI. Mainstream workers are still interested in the problem solving aspectsof the subject whilst the situated agent people see things much more in terms of skilied behaviour. There have been no successful attempts to bridge the gap. In fact, the gap is rather large and therefore daunting but sooner or later someone is going to have to try to cross it. The topic of naturallanguage has been conspicuous by its absence from the body of the lecture. The principal reason is that whatever work has been proceeding on situated language, very little of it has filtered through to the situated agent community and yetI believe that ultimately this must happen for two reasons: first is the intrinsic importance of naturallanguage to the cognition of agents and second is the revitalising thrust which I believe a holistic approach would give to the field. The only work along these lines of which I arn aware is described by Ward (Ward 1992). In the section on the social behaviour of agents I emphasised the complementary nature of the behaviour of agents. This is really just a hint of a much more radical position which I think will eventually have to be embraced in this paradigm although it is not clear at present how this can be done. I arn referring to the idea that cognition and intelligence, if you wil!, are not things that are contained within the head of the agent but are products of the social interaction of agents (Coulter 1979, Costal i 1991 ). If this line of argument holcts up it is fruitless to continue to try to make agents more intelligent simply by putting more and more 'clever' things into them. They wil! only behave in ways which we would remotely cal! intelligent if they have a rich interaction with each other and with the world. It is because of this requirement for a rich and close interaction that I fee! that situated agents are perhaps our best bet at the moment. Finally applications. I am nota ware of any genuinely commercial applications of situated agents that have been made so far. There have, on the other hand, been quite a lot of projects which have demonstraled their potential. It seems likely that the man-machine interface wil! be one area where applications will appear before long. In the slightly Jonger term multi-media, virtual reality and interactive drama (Bates 1990) are all areas where skilied agents could be expected to play important roles. Agents are being introduced into educational systems: work is proceeding in this area at IPO (van Hoe and Masthoff 1993). They can also be expected to appear in training scenarios and command and control training exercises. In short there is no lack of potential applications and the technology is maturing rapidly. We can expect the results of the fusion of the two in the fairly near future. References Agre, P.E. (1995) "Routines", MIT Al Memo 828. Amarel, Saul (1968) "On Representations of Problems of Reasoning about Actions", Machine Intelligence 3 (ed. Michie), Edinburgh UP, (1968) 131-171. Bates, Joseph (1990) "Computational Drama in Oz", Working notes of the AAAI Workshop on Interactive Fiction and Synthetic Realities, Boston, July 1990. Agre, P.E and Chapman, D. (1987) "Pengi: An Implementation of a Theory of Activity", Proceedings of the AAAI Conference, Seattle, Washington. Brooks, R.A. (1986) "A Robust Layered Control System Fora mobile Robot", IEEE Joumal ofRobotics and Automation, vol. RA-2, No. 1, March 1986, 14-23. Brooks, R.A. (1991) "Intelligence without representation", Artificial Intelligence 47 (1991) 139-159 Chapman, D. (1987) "Planning for Conjunctive Goals", Artificial Intelligence 32, (1987), 333-377. Chapman, D. (1989) "Penguins Can Make Cake", Al Magazine, Wmter 1989, pp.45-50. Chapman, David (1991) "Vision, Instruction, and Action", MIT Press, Cambridge MA, London, England Chapman, D and Agre, P.E. (1986) "Abstract Reasoning asEmergent from Concrete Activity" in "Workshop on Reasoning about Actions and Plans", ed. Georgeff and Lansky, Timberline, Oregon, publ. Morgan Kaufmann. Clancey, W.J. (1991) "Israel Rosenfield, The Invention of Memory: A New View ofthe Brain", Artificiallntelligence, 50 (1991) 241~284. Costall, A. (1991) "Graceful Degradation", in "Against Cognitivism: Alternative Foundations for Cognitive Psychology", Arthur Still and Alan CostaU (Eds., London, Harvester Wheatsheaf (1991).) Coulter, J. (1979) "The Social Construction of Mind: Studies in Ethnomethodology & Linguistic Philosophy", publ. Macmillan Press. Dreyfus; H.L. (1992) "What Computers Still Can't Do: a Critique of Artificial reason", Cambridge, MIT Press 1992. Edelman, O.M. (1987) "Neural Darwinism: The Theory ofNeuronal group Selection", New York, Basic Books, 1987. Hickman, S.J. and Shiels, M.A. (1990) "Situated Action as a Basis for Cooperation", in Proceedings of the 2nd. European Workshop on ModeHing an Autonomous Agent in a Multi-Agent World, Paris, 1990, publ. Elsevier-North Holland. 61 62 van Hoe, R.R.G and Masthoff, J.F.M. (1993) "A multi-agent approach to interactive leaming environments" in 'Collaborative problem solving: theoretica! frameworles and innovative systems' Al-ED 93 Workshop, Edinburgh, Scotland. Kiss, G. (1991) "Autonorneus Agents, Al and Chaos Theory", in "From Animals to Anirnats", eds. Meyer and Wilson, MIT Press. Laurel, B. (1991) "Computers as Theatre", Addison-Wesley, 1991. Maes, P. (1991) "Leaming Behaviour networles from Experience", Proceedings of the First European Conference on Artificial Life, MIT Press, 1991. Maes, P. (1993a) "Behavior-Based Artificial Intelligence", Proceedings of the 2nd. Conference on Adaptive behavior, MIT Press, 1993. Maes, P. (1993b) "A Leaming Interface Agent for Scheduling Meetings", Proceedings ACM - SIGCHI, Florida, 1993. Maturana, H.R. and Varela, F.J. (1970) "Autopoiesis and Cognition: The Realization of the Living", D. Reidel Pub!. Co. Dordrecht, Holland and Londen England, 1970. Nilsson, N.J. (1980) "Principles of Artificial Intelligence", Tioga Publishing Co. Rich, Elaine (1983), "Artificial Intelligence", McGraw Hili. Suchman, L. (1987) "Plans and Situated Actions: the problem of human-machine communication", Cambridge University Press. Ullman, S. (1984) "Visual Routines", Cognition, 18, pp. 97-159. Ward, N. (1992) "A Parallel Approach to Syntax for Generation", Artificial Intelligence 57 (1992) 183-225. Wavish, P.R. ( 1991) "Exploiting Emergent Behaviour in Multi-Agent Systems", Proceedings of the 3rd. European Workshop on Modelling an Autonorneus Agent in a Multi-Agent World", Kaiserslautem, 5-7 August 1991, eds. Y. Demazeau and E. Wemer, Elsevier Science Publishers B.V. (North Holland), 1991. Wavish, P.R. and Connah, D.M. (1990) "Representing Multi-Agent Worlds in ABLE", Philips Research Laboratories, Redhill. England, Technica! Note Number 2964. Wavish, P.R. (1993) Private Communication. Wittgenstein, L. (1953) "Philosophical Investigations" Blackwell. Referring in a Shared Workspace Anita H.M. Cremers Institute for Perception Research, P.O. Box 513,5600 MB Eindhoven, The Netherlands Abstract. In this paper referring acts to objects (expressions as well as gestures) that were used in dialogues during a block building experiment in a shared work-space are described. A classification of in formation types that were used in the refer-ring expressions is given, which consists of four main categories, i.e. reference to physical features of the object, reference to the location of the object in the domain, reference to the orientation of the object in the domain, and reference to the so-called history of the object. Also the influence of mutual knowledge of the partici-pants (either knowledge about the dialogue or about the domain) on the used refer-ring acts is described. The focus of attention tums out to play an important role in the use of references. It is argued that, beside the effect of dialogue focus on refer-ring acts, especially the effect of the domain focus needs to be investigated further. Keywords. Referring expressions, referring gestures, dialogues, mutual knowl-edge, focus of attention 1 Introduetion When two people are talking about a task they should perfarm together, they will have to indicate to cachother which of the available matcrials or tools they are going to use. So, in this type of situation they will use a lot of referring acts, i.e. verbaland non-verbal means to single out an object in space, so that the partner will be able to locate it. Research on referring behaviour to objects or places in the extra-linguistic con-text has mainly focuscd on the linguistic part of the referential act. Spatial concep-tion can be encoded in linguistic structures in many ways, for instance, by means of deixis, relative locations or motions, absolute systems and place narnes or descrip-tions (Levinson 1992). Thc decision of a speaker to utter a particular expression to refer to an object largely depends on the type of coordinate system that is used, the place of the origin of this system and, possibly, the chosen relatum, i.e. the object or person that is chosen as a reference point with respect to which this object is located (Levelt 1989). However, in face-to-face conversations, in many cases these utter-ances are accompanied by non-verbal communicative acts, such as gestures, facial 65 66 expressions and/or bodily oricntations. Especially (poiming) gestures are of vital importance for the hearer to be able to identify the right object or place, since they can provide a very precise indication of their location. Although much is already known about linguistic means of reference, it has still not been investigated enough how people refer, verbaBy as well as by means of gestures, to objects or places in an actual dialogue situation. The goal of the present research is to investigate how the speaker's assumption of the hearer's knowledge influences the information that is included in the referen-tial acts. This research is based on the hypothesis that form ('what information is included in the expression?') and content ('which object does the expression refer to?') of referential acts strongly depend on the mutual knowledge of speaker and hearer (the common ground, as Clark and Wilkes-Gibbs call it (Clark and Wilkes-Gibbs 1986)). This can either begeneral knowledge about the domain of discourse (e.g. properties of objectsin the domain), knowledge about the actual state and his-tory of the domain of discourse (e.g. the past or the present location of a certain object) or knowledge about the preceding utterances in the dialogue (e.g. a name that has been given toa certain object). In this paper, first the methodology of a dialogue experiment, that was carried out to study referring expressions to objects in a relatively restricted domain, will be described. Then, an overview will be given of the types and amounts of referring expressions that were used in the dialogues. Finally, the influence of the speaker's and the hearer's knowledge on the type of referential act used will be outlined. 2 Methodology In order to test the hypothesis that was mentioned in the introduetion a dialogue experiment was carried out in which 10 pairs of Dutch subjects participated. The experimental set-up and task were designed as to evoke as many varled referential acts as possible. One of the dialogue parUiers (A) was told to instruct the other part-ner (B) in detail to first build, and second, rebuild a block-building on a toy founda-tion plate on the basis of a provided example. The block-building consisted of blocks in four different colours, three sizes and four shapes.The parUiers were seated side by side at a table, but were separated by a screen. Only their hands were visible to each other, provided that these were placed on top of the table in the neighbour-hood of the foundation plate. All subjects were allowed to observe the building domain, totalk about it, and to gesticulate in it, but only the respective builders were allwed to manipulate blocks. The set-up of the experiment is depicted in tigure 1. The resulting 20 building sessions, of which the dialogues were much like Grosz's task dialogues (Grosz 1977), were recorded on video-tape. tab ie foundation plate blocks ® 0 example building Fig. 1. Experimental set-up (top view) 3 Referring expressions in the dialogues In the dialogues four main categories of referential means could be distinguished, namely reference to physical features of the object, reference to the location of the object in the domain, reference to the orientation of the object in the domain and, finally, reference to the so-called history of the object which was developed in the course of actions that were carried out in the domain and the topics that were dis-cussed in the dialogue. These four categones will be discussed in the subsections 3.1 through 3.4 below. In this paper only the results of the re-building sessions are included for analysis, since the referring expressions in these 10 dialogues showed more variation, since the participants had to carry out more varied actions. In the selected dialogues a total amount of 665 referring expressions occurred, of which 59.5% (396) actually contained information out of one or more of the four catego-Ties mentioned above. These expressions either acted as direct references to objects in the domain (62.4% of them, which means an amount of 247) or as anaphora (37.6%, 149). The remaining 40.5% (269) only consistedof pronomina or demon-stratives.The four categoriesof information-containing references will be discussed below. 3.1 Reference to Physical Features of an Object By far the majority of the referring expressions that were used included information about physical features of the object that had to be identified. In fact 92.2% (365) of the referring expressions included this type of information, and 77.5% (307) of the expressions consisted of this information type only. The physical properties that were used in the dialogues were colour (e.g. red1), size (e.g. large) and shape (e.g. square) of the blocks, which actually were the three distinguishing features that 1. Since the examples of referential expressions provided in this paper are merely abstractions of the Dutch expressions that were used in the dialogues, they are only given in English. 67 68 were present in the set of blocks available. There was a feature preferenee for col-our, since this was used in 97.3% (355) of the references that contained information about physical features, whereas size and shape were mentioned in, respectively, only 25.2% (92) and 17.8% (65) of the cases. 3.2 Reference to the Location of an Object Reference to the location of an object was mainly used by dialogue participants who did not use pointing gestures at all. This definitely makes sense, because both refer-ential means seem to accomplish the same effect, one by means of language, the other through gesturing. 1. Location in general Dialogue participants somelimes used general referring expressions to objects in the domain, e.g. by means of here and there. In 3.8% (15) of the total amount of infor-mation-containing referring expressions used this was done, and in 1.8% (7) of the cases this reference type was used in isolation. When these references were used to refer directly (non-anaphorically) to objects in the domaio pointing gestures were always added. Most of the occurrences of this type of reference occurred at points where the speaker shifted his or her attention to another region of the domain, for instanee to give building instructions about a new part of the building. We shall cal! this kind of shift a transition of the focus of auention. 2. Location with respect to the participants The location of an object was sometimes indicated by staling its relative position with respect to both participants, e.g. in the blue one on the right. This information type was used in 6.1% of the total amount of references and in 1.5% of the cases it was the only type. Since the participants were seated side by side, their perspectives towards the domaio were almost the same, and no distinction between hearer- and speaker-based reference had to be drawn. These expressions were hardly ever accompanied by pointing gestures, probably because they already contained enough information to identify the intended object. Reference by means of indicating the location of an object with respect to the participants was again mainly used to insta!! a new focus of attention in the domain. 3. Location with respect to (an)other object(s) The location of an object was also indicated by means of staling its position with respect to another object or other objects in the domain, e.g. in the blue block behind the yellow one. Of course, in these expressions the position of the participants should also betaken into account. These expressions were used in 3.5% (14) of the cases, and in 0.8% (3) of the cases this was the only information type supplied. Ref-erence to a location of an object with respect to (an)other object(s) is mainly used when the current block is located in the neighbourhood of the block referred to pre-viously. In those cases the previous block is used as the relatum of the current one. Since in these cases the current object is located close to the previous one, no focus transition occurs, it is rather maimained. That could also be reason why no pointing gestures were being used to accomapny this type of expression, since the location was already more or less clear. 4. Location with respect to the hand of a participant within the domain The occurrence of referring expressions using the hand of a participant as a relatum is probably a consequence of the specific set-up of the task. In this set-up the partic-ipants share the same perspective, and the only difference between each other's bod-ily positions they can actually observe is the position of their hands, which may change constantly. lnstances of this type of referring expressions occurred in 1.3% (5) of the cases, and in 0.8% (3) of the cases it was the only disambiguating informa-tion offercd. No gesturcs accompanicd these expressions. The position of the partncr's hand was used in two ways. First, the speaker can inform the partner on where a block is located with respect to the location of his hand at the moment of the utterance, e.g. the blue block lO your right. Second, the speaker can teil the partner in what direction his hand should move in order to reach the intended object. So, in this case, the location of the hand is presupposed, and instead of informing the partner about the location of the object, the action he or she has to carry out is supplied. This action can either be a default-action or an explicitly indicated one. In the case of a default-action the partner is already moving in the right direction and should only be encouraged to proceed doing so, e.g. in a bilfur-ther. In the latter case the explicit direction should be provided, like in go lO the left. 3.3 Reference to the Orientation of an Object The different ways in which an object can be positioned in the domain all result in different object orientations. Speakerscan make use of an object's orientation to dis-tinguish it from other (identical) objects, e.g. in the horizontal yellow block. In 1.0% (4) of the total amount of referring expressions participants made use of this disam-biguating device, but the information was always accompanied by other types of information. No accompanying gestures were used. 3.4 Reference to the History of an Object in the Domain or in the Dialogue Finally, an object can be disambiguated by means of reference to earlier events in which the object was involved, e.g. in the red one you have just put down. Also, when one of the participants has already talked about the object earlier in the dia-logue this can be used for disambiguation, e.g. in the red one youjust mentioned. In 7.1% (28) of the total amount of referring expressions participants made use of historica) aspects, in 2.8% (11) of the cases this was the only information they provided. References to the domain history and the dialogue history occurred equally often. Hardly any accompanying gestures were used. The only ones that were used referred to objects that had been talked about before, and were still 69 70 located in the domain. 4 lnfluence of Mutual Knowledge on Referring Expres-sions During the course of a dialogue and of events in the domain of conversation, knowl-edge is built up about these issues. Later in the dialogue and in the building process, partners may make use of the knowledge they assume the other has available. In fact, the assumption of the presence of this knowledge may to some extent deler-mine the type of information that is being used in the current referring expression, although it may be fully disambiguating in its own right. This will be discussed fur-ther in the sections 4.1 and 4.2. Beside fully disambiguating referential expressions, dialogue participants also use reduced expressions, i.e. expressions that would not be fully disambiguating when used in isolation. A reduced expression may be used when the speaker implic-itly assumes that the partner already knows or can easily get to know the missing part of the fully disambiguating information he should actually have provided. In the sections 4.1.1 and 4.2.1 occurrences of reduced in formation will be discussed. 4.1 Inttuenee of the Dialogue Context on the Type of lnformation In the dialogues it could be demonstraled that participants made use of the contents of the preceding dialogue when uttering the current referring expression. In the first place, when panicipants refer to the history of the dialogue (see sec-tion 3.4), they assume that there is mutual knowledge about the fact that this partic-ular topic has been talked about previously in the dialogue. Second, participants may, either explicitly or impicitly, agree on a name for a partic u lar object, or a particular type of object. For example, in the dialogues a block with a deviant shape was agreed upon to be called de glijbaan ('the slide') by a cou-ple of participants. Another couple agreed on calling the smallest block type de kleine ('the smal/ one') in contrasttoa larger type, which was called de grote ('the large one' ), because there had been some misunderstanding earlier on how to tell the two types apart. Later in the dialogue the respective participants were able to use these names, when appropriate, without any problem. Third, when one participant is busy trying to make clear which object he or she is referring to, but the other is nat able to find out which object is hinted at by the description (e.g. the large blue block), the speaker will have to add some more infor-mation to help the hearer identifying it.2 In the second 'turn' the speaker will usually use other types of information than the information types provided previously (e.g. 2. Note that this does nol necessarily have to mean that the referential expression was ambiguous in itself, since thc reason for the misunderstanding could also have been an inability on the part of the hcarcr. the one on the right). So, in this case the speaker assumes that the hearer still knows the previous information, and provides only information that should be added to that knowledge. This means that, in genera!, given information does not have to be uttered for the second time, unless the hearer indicates that he or she has not under-stood what had been said. Finally, if an object has been talked about recently, and the current block is located close to that block, then the previous block can be used as a relatum for descrihing the current block. In those cases a reference to the location of an object with respect toanother object (see section 3.2) is aften used. In fact, in 71.4% (10) of these expressions the relatum was the last object that had been mentioned before. For example in place a large yellow block, place a smalt blue block to the right of the yellow one, the large yellow blockis used as a relatum for indicating the place of the small blue block. 4.1.1 Reduced Information as a Result of Dialogue Knowied ge Assumed knowied ge about the dialogue on the part of the partner is reflected in the use of pronomina! anaphora and ellipsis in the referring expressions. Anaphora can be used when the speaker assumes that the object referred to is in the focus of atten-tion of both participants (Grosz 1977), in which case a pronomina! reference suffices for making clear which object is meant (e.g. take a smalt red block, put it on top of the large green one). Ellipsis occurs when the referring expression in an utterance is omitted altogether, or when parts of the utterance are omitted. Total omissions may take place when both participants have already agreed upon which object is being referred to, and only some predication about the object is left to express, for exam-ple the destination of a particular blockafter having carried out the action on it (e.g. take a smal/ red block, put it on top of the large green one, ( ... ) the green one on the right). Partial omissions may occur when information in the current expression par-tially coincides with information in the preceding expression, e.g. in place a small yellow block, put a blue one on top of it. In this example the 'blue one' is taken to be a small block as well, although the explicit information is omitted. 4.2 Influence of the Domain Context on the Type of Information In the dialogues, participants also made use of the knowledge about the history and the state of the domain of conversation they assumed their partner possessed. As was already discussed in section 3.4, participants can refer to events that have hap-pened in the domain, like in the block that just feil off. Note that domain events can be referred to even when they have not been discussed earlier in the dialogue. Partic-ipants make use of the state of the domain by making sure that their referring act is as much disambiguating as possible within the domain. The speaker can do this by camparing the physical features of the block and the orientation of the block within the domain with features and orientations of the other blocks that are present. Also the position of the block with respect to the participants, with respecttoother blocks 71 72 and with respect to the hand of a partner cao be used to effectuate the disambigua-tion. 4.2.1 Reduced Information as a Result of Domain Knowledge: Focus of Attention Knowledge about the domain of discourse and the manipulations that are or have been carried out in it mayalso give rise totheuse of reduced expressions. Triggers for reduced reference within the domain that were observed in the experiment are the existence of either a spatial or a functional focus of attention. 1. Spatial focus The spatial focus in the domain is the part of the domaio that is being attended to more strongly than the rest of it. For objects that are located in a highly attended area the speaker does not have to provide fully disambiguating information in the used referring expressions. In these cases, the disambiguation should only concern the part of the domain that is attended to. The spatial focus can be created either explic-itly verbally or implicitly or by means of (pointing) gestures. • Explicit The speaker may announcc explicitly that objects that are located in a particular sub-domaio should be attended to more strongly, e.g. in let's move to the right upper part. In this way the speaker makes sure that the hearer focuses his attention on the indicated subdomain, so that he could use less information in expressions referring to objects that are located there. In the dialogues 6.1% of the total amount of refer-ring expressions contained this type of inforrnation, and 1.5% of the same amount consisted exclusively of this inforrnation type. • Implicit The speaker may implicitly make use of the assumed focus of attention of the part-ner. The speaker can have well-founded reasoos for believing that the partoer's focus is actually directed to this particular sub-domain, for example when he has just referred to an object that is located there. In this case, the speaker might argue that the partner is inclined to consider the next reduced expression as a reference to an object in the neighbourhood of the one just mentioned. For example in please remave these blocks (points at a green, a yellow and a blue block), the red one can remain seated there, the speaker refers to a red block without fully disambiguating it. However, it is clear for the listener that the speaker refers to the red block in the neighbourhood of the three blocks that were pointed at earlier. • Gestures The partners may use their hands to point at objects, to indicate their orientation in the domain, to touch them, to piek them up or to hold them. Both the instructor and the builder are allowed to point at blocks and to touch them, but the latter cao in addition piek up the blocks and hold them in his or her hand. Both partners can also make use of the fact that the other is (incidentally) pointingator touchinga block. In those cases the speaker can refer orally to that particular block by means of a mini-mally informative expression, like that one. The instructor can also use this mecha-nism when the builder is picking up a block or holding it. In all of these cases the speaker can use less information than he would have needed if there had not been some involvement of a hand. In the dialogues participants used some kind of accom-panying gesture in 16.8% of the total amount of referring expressions. However, three out of ten instructors did not use any gestures at all during the dialogues. Since the participants had been instructed to act as spontaneously as possible, this can just be considered a matter of preference. Th is Jack of gesturing did not have a notabie effect on the percentage mentioned above, because in these dialogues the builders had to use more gestures in order to verify whether they had correctly identified a particular block. This was less necessary for builders that had a pointing instructor as a partner. 2. Functional focus Next to the assumed knowledge about spa ti al focus the speaker may make use of the partoer's assumed knowledge about the current functional focus. Functional focus is related to the actions that have to be carried out in the domain. The concept of func-tional focus applies when the action that should be carried out more or less restricts the amount of blocks that can reasonably be involved in the action. In that case reduced information is possible, based on the assumed acquaintance of the partner with the preconditions- or post-conditions of the action. • Preconditions A partner can make use of the preconditions of an action when an object has to be removed. In that case, the object referred to is most likely the one that is located at a position that is easily reachable, e.g. in remove the little yellow one, which is taken to be the small yellow block on top of the building, although there are many other small yellow blocks available at positions that are not so easy to reach. • Post-conditions Post-conditions of an action can be used when an object is to be (re-)placed. Then the object referred to is most likely the one that fits best at the indicated location. For example, when there is an opening between two yellow blocks that can only contain a smal! block, the 'green block' in the utterance p/ace a green b/ock between the ye/-low ones is probably taken to be a smal! one. In total the dialogues contain about 30 (12.1% of the total amount of information-containing direct references) cases of the implicit sparial focusing and the functional focus mechanism together. In these cases, it could be demonstraled that the partner was actually making use of one or bothof these mechanisms, because the provided information was not fully distinguishing the referent object from the surrounding ones, and no pointing gestures were used. Actually, it was not always possible to distinguish between the use of implicit focusing or functional focusing, because these mechanisms coincided many times. For example, when a speaker has just been talkingabout a yellow block, and subsequently says that a large red block should be placed on top of a small green one, he probably means the green block that is placed in the neighbourhood of the yellow block, and is suitable for placing a large block on top of it. 73 74 5 Discussion and Conclusions In this paper an analysis was given of referring acts ( verbal as wellas gestural) that occurred in 10 Dutch so-called task dialogues, and of the inftuence of mutual knowledge on the usage of these acts. In the dialogues four main categones of referring expressions to objects occurred, namely reference to physical features, to the orientation or the location of objects, and to the history of an object in the domain or in the dialogue. Research on the naming of objects (Herrmann & Deutsch 1976, Herrmann 1983) bas resulted in the finding that expressions used to single out an object in space depend on the objects surrounding the referent object. Speakers normally describe an object in such a way that it can be distinguished from other objects. In fact, these object naming are informative as well as non-redundant. Also, speakers tend to use the distinguishing feature that expresses the most pro-nounced difference between the referent object and the surrounding ones. In the dialogues the participants showed a strong preferenee for using physical fea-tures, especially colour. However, this may be due to the type of objects that were present in the domain and the task that had to be carried out. It could be demonstraled that the participants made use of mutual knowl-edge about the contents of the preceding dialogue as well as the state and history of the domain of conversatien when uttering the current referring expression. This was particularly obvious in cases where participants used reduced expres-sions, i.e. expressions that would not have been disambiguating if they had appeared in isolation. Reductions in information due to dialogue knowledge occurred in the form of pronominal anaphora and ellipsis. Reductions as a result of domain knowledge occurred especially when objects were positioned in the focus of attention of the participants, either as a result of their location and/or function or by means of hand gestures. Knowledge about the saliency of a particular object in the domain of con-versatien may lead to the use of a reduced reference as wel!. Clark et al. (Clark, Schreuder and Buttrick 1983) showed that listeners select referents based on estimates of their mutual beliefs about perceptual salience, the speaker's goals, and the speaker's presuppositions and assertions. In our building domain sali-ency may relate to the physical features of an object, to the location of an object, to its orientation, or to its relative static or dynamic state. However, no clear examples of saliency effects could be found in the dialogues, probably because there were no objects with a clear inherent saliency present, and the task did not allow for using other saliency triggers. The focus of attention appeared to be a central notion in the use and inter-pretalion of referring expressions. At focus transitions participants tended to use more referring expressions to the location of an object in generaland to the loca-tion with respect to the participants. Reference to the location of an object with respect to other objects was rather used when the focus did notshift (see section 3.2). Moreover, the focus of attention played an important role in the production and comprehension of reduced reference to objects. The effect of dialogue (or discourse) focus on the possible use of pronouns and ellipsis is well-known (Grosz 1977, Grosz 1981). However, the effects of domain focus, sparial as well as functional and gesture-driven, deserve further research. References Clark, H.H., Schreuder, R., Buttrick, S. (1983) Common ground and Lhe understanding of demonstralive reference. J. of Verbal Learning and Verbal Behavier 22, 245-258 Clark, H.H., Wilkes-Gibbs, 0. (1986) Referring as a collaborative process. Cognition 22, 1-39 Grosz, B.J. ( 1977) The representation and use of focus in dialogue understanding. Techni-ca! Note 151. Menlo Park: SRI International Grosz, B.J. (1981) Focusing and description in naturallanguage dialogues. In: A. Joshi, B. Webber, I. Sag (eds.) Elements of discourse understanding. New York: Cam-bridge University Press Herrmann, Th. (1983) Speech and situation: a psychological conception of situated speaking. Berlin: Springer Herrmann, Th., Deutsch, W. (1976) Psychologie der Objektbenennung. Bern: Huber Levelt, W.J.M. (1989) Speaking: from intention to articulation. Cambridge: MIT Press Levinson, S.C. (1992) Primer for the field investigation of spatial description and concep-tion. Pragmatics 2(1 ), 5-47 75 User Responses to Constraint Management in Computer-Mediated Design Activities: A Conceptual Framework Donald L. Day1 1 School of Infonnation Studies, Syracuse University, Syracuse, NY 13244-4100, USA DOlDA YXX@SUVM.SYR.EDU (Internet) Abstract. This paper discusses an ongoing study of user responses to constraints in toois used to design and generate computer software. The study also addresses the communication of design process ruies and mental modeis from tooi deveiopers to tooi users.2 Keywords. Constraint management, decision support, automated design, cognitive fit, computer-aided software engineering, process engineering. 1 Introduetion 1.1 Relationsbip to Institute Tbemes 1.1.1 Human Learning, Perception and Cognition This work proposes that spreading activation and semantic networks be used to characterize the decision-making process, and investigates user evaluation of relationships among constraints and design task options. It also examines user onderstanding of process modeling, and user perceptions of self within the work context. 1.1. 2 Learning As An End-User Goal Design toois examined in this study teach users appropriate procedures. Some tutor them about the software deveiopment principles underlying constraint implementation. Others inform individuals of the enterprise model followed by their organizations and alert them to quality assurance concerns. 1.1. 3 Intelligent Support and Assistance To Users Findings of studies such as this may be helpful in specifying constraint management systems that operate as knowledge bases. In such systems, 2 Portions of this paper appeared originally as "Precis of Behaviaral and Perceptual Responses to Constraint Management in Computer-Mediated Design Activities" in the Electronic Journal ofCommunication!La Revue Electronique de Communication ,Vol. 3, No. 2, April 1993. 77 automatically recorded transaction data descrihing constraint negotiation and user execution of design tasks may be used to create customized user-process profiles. Sufficient onderstanding of the interplay between constraint implementation and user behavior also may help to develop intelligent assistants which can calibrate constraint flexibility dynamically, within task context, and project the impact of current decisions upon future choices and end-product characteristics. The result would be design tools that are as flexible as they can be without degrading product quality unacceptably. 1.1.4 Technologies, Tools and Techniques For Human-Machine Communication The tools applied by users included in this study incorporate advanced knowledge representation, decision process and user interface design features which may affect responses to constraint management techniques. For example, graphical user interface tools and user interface management systems provide a wide range of visual design and process structuring capabilities which allow users to communicate with the machine on a conceptual level, rather than being forced to address details ofunderlying implementation technology. 1.1.5 Knowledge Transfer Systems Decision support in the context of this study includes the transfer of design and process rules from developers to users, via computer-mediated tools. The tools in effect sponsor developers' process roodels as appropriate and efficient means to develop high quality software. In this sense, leaming which takes place during tooi use constitutes knowledge transfer, not merely process assistance. 1. 2 Other Work In This Series Related work by this author includes a doctoral dissertation proposal [1], a paper in the Electronic Journal of Communication [2], a presentation at the Symposium on Human Factors in Infonnation Systems [3], and a special issue of the Electronic Journal on Virtual Culture [4]. 2 Domaio 2 .1 Domaio Generalization Although the domaio featured in this study is software engineering, the conceptual framework, analytic approach, and onderstanding of user responses developed bere may facilitate attempts to refine computer-mediated design tools used in other domains (e.g., pubHeation design, mechanical engineering, architecture, industrial chemistry and bioengineering). 2. 2 Domaio Selection and Characteristics Software engineering is an appropriate first domaio for the study of computer-mediated design tools. Computer-aided software engineering (CASE) tools are used to create software packages, including other tools. The creation of tools for other 78 domains allows CASE tools to influence design activities in a number of professions and industries. The software engineering domaio is populated by a wide variety of creative, perceptive, process-oriented and logica! individuals who are especially capable critics of the tools which they use. Since they themselves are software developers, users in this domaio have the experience and knowledge to evaluate CASE tools in detail. In significant ways, design tooi users in other domains are somewhat like typical motorists, who may know how their cars should perform, but would oot be capable of criticizing internal engineering features of automobile engines. Design activities in the study domaio take place amidst a host of somelimes conflicting priori ties, sponsored by a number of major actors. Staleebolders include costomers for software created with CASE tools. tooi developers, tooi users, user management. and developer management. This study focuses on tooi developers and tooi users. Developers' perceptions are influenced by concerns about risk assessment, resource management, process control, quality assurance, style preferences, and tooi user satisfaction. Users' perceptions are affected by project requirements, training and design experience, the work environment, style preferences, costand time-on-task (schedule), and satisfaction with the tools. 2. 3 Computer-Aided Software Engineering Tools CASE tools facilitate the creation and implementation of design elements and the generation of executable software. They do so in part by imptementing constraints made necessary by a variety of economie, quality, schedule and maintainability concerns. The tools make it possible for system analysts, software engineers and programroers to access a common, current, and comprehensive repository of design and implementation information. thereby facilitating cooperation essential to the creation of quality software on large. complex projects. CASE tools range from single-purpose "programmer's workbench" modules to intertwined hierarchical or object-oriented tooisets which facilitate process and quality control tbrooghout much of the software development life cycle. Included are tools which support planning, analysis and design; user interface development; code generation; reverse engineering; maintenance; test and evaluation; configuration management, and documentation. 3 Conceptual Framework 3. 1 Mental Mode Is, Creativity and Standards Computerized design tools provide a meeting ground between users' and developers' mental models of desirabie processes, functions, tasks and constraints. It is a premise of this sbldy that user responses to design tools may be affected substantially by the tools' constraint management systems and by the degree of match (cognitive fit) between the mental modelsof tooi developers and tooi users. Conflicts sametimes occur between users' sense of design creativity and the need to proteet product and process standards via constraints. These conflicts are due in part to a mismatch between values and procedures preferred by tooi developers versus tooi users. This study proposes that this mismatch is in fact misonderstanding caused by incomplete cognitive fit between the mental models of developers and tooi users~ It seeks to improve oor onderstanding of design decision 79 80 support, the communication of mental models, and user behavior, so as to facilitate design creativity while ensuring reasonable adherence to product and process standards. 3 • 2 Tbe Transfer of Knowied ge The transfer of knowledge during CASE tooi use (Section 1.1.5) is hampered not only by creativity and constraint conflicts (Section 3.1), but also by inherent limitations of cognitive transfer processes. Por example, researchers [5] have found that onder eertaio circumstances some subjects have difficulty generalizing leaming. The unique context of many design decisions (i.e., situational specificity) frustrates tooi users' attempts to generalize process roodels from prior experience. Realizing this, some tooi developers may favor inflexible constraints because they feel it is better to limit user creativity than to risk degradation of end-product quality because of inappropriate user generalization. 3. 3 Constraint Issues and Typology The constraint management system (CMS) within a CASE tooi represents the weighted process and product quality requirements associated with each stage in the software development life cycle. A properly implemented CMS can be a valuable decision support feature, guiding users in the selection of design options which are consistent with requirements. In many instances, a user may be involved in only a few stages of product development. Since the CMS represents requirements of all stages, it can help imprave product quality and reduce error by evaluating user actions in the context of the project as a whole, compensating for the inherently narrow view of a user involved in only a few stages. From the user viewpoint, however, constraints can be frustrating precisely because they imptement concerns from outside of the immediate task environment In order to study constraint management systematically and rigorously, a typology of constraints is necessary. A carefully constructed typology reduces ambiguity, facilitates creation of appropriate measurement instruments, and helps to validate the analytic framework. The typology devised for this study consists of four discontinuons axes. It seeks to characterize the key attributes of any constraint implemented by a developer in a tooi. The frrst axis is a human-computer control scale (Table 3.3); the second consistsof implementation styles; the third includes the sourees of authority for requirements which trigger constraints; the fourth is the relative importance (weight) of the constraint (i.e., the severity of impact if the constraint is violated). Constraint satisfaction does not necessarily require the application of inflexible rules; it only mandates that a series of decisions taken as a whole roeets necessary and sufficient standards. It is end-product characteristics, not the constraints intended to eosure those characteristics, that are important. Therefore, constraint flexibility does not necessarily result in product degradation. Without an appreciation of this aspect of process control, constraints implemented by developers in design tools can be unnecessarily restrictive. An analogy from elsewhere within software engineering might be that of critical path metbod (CPM) scheduling. Under CPM, project milestones are plotted sequentially in a chronological graph. Some of the activities represented are Table 3.3. A Human-Computer Control Scale 1 . Human considers alternatives, makes and irnplements decision. 2. Computer offers a set of alternatives which human may ignore in making decision. 3. Computer offers a restricted set of altematives, and human decides which to implement. 4. Computer offers a resuicted set of alternatives and suggests one, but human still makes and implements fmal decision. 5 . Computer offers a restricted set of alternatives and suggests one which it will imptement if human approves. 6. Computer makes decision but gives human option to veto before irnplementation. 7 . Computer makes and irnplements decision, but must inform human after the fact. 8. C()mputer makes and implements decision, and informs human only if asked to. 9 . Computer makes and irnplements decision, and informs human only if it feels this is warranted. 10. Computer makes and irnplements decision if it feels it should, and informs human only if it feels this is warranted. a Adapted from [ 6]. performed concurrently by several individuals on the project. Some tasles take langer than others; eenaio (antecedent) tasles must be completed befare (dependent) others cao be started. If it were to take longer than scheduled to complete an antecedent task, the delay would ripple through following dependent tasles and place final delivery off schedule. Antecedent-dependent tasles are on the "critical path". Other tasks oot on the critica! path can take longer than anticipated without serious repercussion, justas constraints which are nöt intrinsically vitalto product quality, cost or schedule can be relaxed. 3. 4 The Problem Space The problem space within which this study is being conducted may be envisioned as two interrelated conceptual graphs characterizing several key aspects of computer-mediated design activities. The frrst graph, "Process Engineering", describes design functions and tasles on one axis and constraints on the other. Design activities are described as functions comprised of component tasks. Constraints are characterized by a four-axis typology (Section 3.3), and are arrayed against the constraints to which they might apply (Figure 3.4(a)). The second graph, "Human Factors", is a graph which describes groups of independent variables which might impact user behavior on one axis, and user behaviaral and perceptual responses (dependent variables) on the other (Figure 3.4(b)). 4 Relevant Literature This study lies at the intersection of several disciplines in the humanities and applied sciences. Included are information science, psychology, computer science, engineering and management science. Wormation science addresses man-machine 81 82 Task_A + + + Function_l Task_B + + + Task_C + + + Task_A + + + Function_2 Task_B + + + Task_C + + + I Constraint_l Constraint_2 Constraint_3 - control - control - control - style - style - style - authority - authority - authority - weight - weight - weight Fig. 3.4(a). The Process Engineering Problem Space Independent V ariables Constraint Types + + + + Individual Differences + + + + Environmental Factors + + + + Tooi Characteristics + + + + Cognitive Factors + + + + Potential Confounds + + + + Assistance Factors + + + + I I I I Constraint Job Of Of Negotiation Performance Self Constraints BEHAVIORS PERCEPTIONS Dependent V ariables Fig. 3.4(b). The Human Factors Problem Space systems (e.g., human-computer interaction); psychology contributes cognition (e.g., mental modeling); computer science offers software engineering; management science provides decision support, and engineering adds process design and control (e.g., eenstraint management). 4 .1 Sourees and Authors Conference proceedings and joumals contain the majority of material relevant to this study. Especially relevant are the annual proceedings of the ACM SIGCHI conferences, the triennial INTERACT meetings (Europe), and the biennial symposia on Human Factors in Information Systems (U.S.). Key joumals include Behaviour & lnfonnation Technology, thelnternational Journalof Man-Machine Studies, Software Engineering Notes, the Software Engineering Journal, ACM Computing Surveys, IEEE Software, Management Science, Decision Sciences, and Communications of the ACM. The key publisher was Elsevier/North-Holland. Of the scores of authors whose work has contributed thus far to this study, a few whose research seemed especially pertinent are listed in Table 4.1, "Key Authors". Table 4.1. Key Authors Author Ackermann, D. Bullinger., H.-J. Carroll, I. Curtis, B. Dechter, R. Diaper, D. Fishbein, M. Galetta, D. Martin, I. Norman, D. Rasmussen, J. Rouse, W. Salvendy, G. Silver, M. W asserman, A. Woods, D. Area of Contribution mental models interface techniques intelligent advising process rnadeling eensiraint networles task analysis user satisfaction cognitive fit information engineering user error behavier human information processing adaptive advising performance rnadeling user decision behavier automated tools decision aids 4.2 Representative Works The literature relevant to this study has been organized into two units dealing with (a) the design activity context and (b) user responses to constraint management (Figure 4.2, "Organization of the Literature"). Space limitations prevent detailed discussion of the literature. However, representative works are cited in "References" (Process Modeling and Design [7-17], Domaio [18-38], Technologies [39-95], Cognitive Factors [96-120], Potential Confounds [121-135], Behaviors [136-145], and Perceptions [146-154]). 5 Research Questions and Methodology S. 1 Research Questions 1. How are characteristics of constraint management within CASE tools (e.g., flexibility) associated with user behavior and perceptions in response to such tools? 2. To what extent are factors such as cognitive fit, trust, mental workload, complexity and users' and developers' mental roodels associated with user behavior and perceptions in response to CASE tools? 83 3. How might potential workplace confounds contaminate apparent relationships among constraint management, cognitive factors, and user behavior and perceptions? 4. To what degree does interaction with a design decision support system educate users regarding principles of software development, and equip them to negotiate constraints successfully? Activity Context process modeling and design I I I I I ",NCi"ta ~ tiltlort 10ftware irttegnJtiOtl stnlkgy do ma in technologies ~ utrip -llodologia I I bowlalge rqwamtati0t1 deelsimt ~ dnelopment etii>ÎrOtlmerots - 1- llfl't1TJOrl wwliatimt cow"""""'t mgineer/CSCW h(llci) CASE tooi$ artificial irttelligence constrairtt assistmu:e cognitive factors potential confounds ~ mental molhls ~ COfU:III7UI copitl."e fit orgrtl rerp to tedt co"."ÜDri'!/ ~- ~----t 1'S!fdu>Social..."..cts deelsion procasa sitrliltional factors _".tal 'IDOtklotul Responses behaviors perceptions constmirtt negotlatiOtl job~ of self ofconstmirtts ~~ ~ prodlldiTri'!l ~-- r~· co"."liance efficimcy self-ateerr< .."",..,~-· "."oid4nce prodw:t lf'U'li'!l frrlstmtiOtl tliffrrentlatiort subrJersi0t1 ",."ficimcy creAtlmty flcibility negatlon Fig. 4.2. Organization of the ·Literature 84 5. 2 Methodology 5.2.1 Experimental Design The frrst of three stages planned for the study is currently underway: Interviews of key staleebolders in the computer-medialed design process. In Stage 2, tooi users and non-tooi software developers (a control group) will be asked to complete a survey questionnaire about their behavior and perceptions; in Stage 3, several tooi users will be observed as they execute a standard script of design tasks. 5.2.2 Subjects This study examines computer-mediated cooperation among skilled users who create program modules as end-products, software engineers and system analysts who define intermodule relationships, set schedules and monitor quality assurance, and developers who create the CASE tools which the first two groups use. Users apply CASE tools in the design, fabrication, testing, integration, maintenance and en bancement of computer programs. Eacb user bas bis or her own professional perspeelive regarding appropriate roodels and procedures for software development. Some feel that their trade is an art more than a science, and resist the application of constraints wbicb restriet their options. Tooi developers generally are higbly skilled software engineers and system analysts employed by firms whose marketabie products are CASE tools. They often are not the ultimate authority for requirements which they imptement as constraints. However, tbe details of constraint implementation are their responsibility. Developers define the processes, functions, tasks and constraints incorporated in a CASE toolset; users select and manipulate functions and tasks in a process sequence to create a product (executable software). 5.2.3 Data Coneetion A structured interview guideline of 12 questions was drafted, based upon a conceptual analysis of the problem area and upon a fully defmed and operationalized list of variables. Data are being colleeled in private interviews with user managers, team leaders and analysts at two Portune 500 firms, and tooi developers at the software engineering unit of a national information systems company. Interviews are being audio recorded and transcribed, for verification and analysis. 5.2.4 Analysis Transcripts are being separated into sections corresponding to the interview outline structure. Next, responses to similar questions will be compared and aggregated. Common and contrastlog themes will be identified and matebed to the research questions. (Preliminary findings related to the third research question will be reported in (3].) Where possible, preliminary conclusions will be drawn. Refmements needed in the instruments will be noted, and themes for future research recorded. 85 86 6 Condusion Automated design bas become common in several professions. This study of constraints, user behavior and the transfer of mental models incorporates many concerns characteristic of any process control or decision support situation. Improved onderstanding of constraint management problems, user behavior and perceptual responses, and the coupling of requirements to constraint implementation can be of considerable value to any enterprise. References3 1. Day, D.: Behaviaral and Perceptual Responses to Constraint Management in Computer-Mediated Design Activities: A Proposal For Research. Unpublished dissertation proposal, School of Information Studies, Syracuse University 1993 2. Day, D.: Precis of behaviaral and perceptual responses to eenstraint management in computer-mediated design activities. Electronic Journal of Communication 3:2 (1993) 3. Day, D.: Contextual aspectsof CASE tooi use in information system design and development. (fo be presented at the Fifth Symposium on Human Factors in Information Systems, Cleveland (October 1993)) 4. Day, D. (ed.): Computerized Tools As Intermediades in the Communication of Mental Maps. Special issue of the Electronic Journal on Virtual Culture (in preparation) 5. Singley, M., Anderson, J.: The Transfer of Cognitive Skill. Cambridge, Mass.: Harvard University Press 1989 6. Sheridan, T.: Computer control and human alienation. Technology Review 83:1 (October), 65-67, 71-73 (1980) Process Modellog and Design 7. Bras, B., Mistree, F.: Designing design processes in decision-based concurrent engineering. In: Aerospace Product/Process Design Interface, 15-36. Warrendale, Pa.: Society of Automotive Engineers 1991 8. Curtis, B., Kellner, M., Over, J.: Process modeling. CACM 35, 75-90 (1992) 9. Gibson, D., Salvendy, G.: Knowledge representation in human problem solving. BIT 9, 191-200 (1990) 10. Newell, A., Simon, H.: Human Problem Solving. Engleweed Cliffs, N.J.: Prentice-Hall 1972 11. Norman, D.: Design rules based on analyses of human error. CACM 26, 254-258 (1983) 12. Peckham, J., Maryanski, F.: Semantic data models. ACS 20, 153-189 (1988) 13. Sharp, H.: The role of domain knowledge in software design. BIT 10, 383-401 (1991) 14. Smith, F.: Towards a beuristic theory of problem structuring. Management Science 34, 1489-1506 (1988) 3 Abbreviations are as fellows. ACS: ACM Computing Surveys; ACM: Association for Computing Machinery; ASME: American Society of Mechanica} Engineers; BIT: Behaviour & Information Technology; CACM: Communications of the ACM; CUP: Cambridge University Press; DSI: Decision Sciences Institute; ERIC: Educational Resources Information Center; DSS: Decision Support Systems; IEEE: Institute of Electrical and Electtonics Engineers; IJMMS: International Journal of Man-Machine Studies; MIT: Massachusetts Institute of Technology; SEN: Software Engineering Notes; SEJ: Software Engineering Journal 15. Sonnenwald, D.: Developing a theory to guide the process of designing. Proceedings, 15th International SIGIR'92, 310-317, Copenhagen. New York: ACM 1992 16. Todd, P.: Process tracing methods in DSS research. MIS Quarterly 11, 493-512 (1987) 17. Webster, D.: Mapping the design infonnation representation terrain. IEEE Computer 21, 8-23 (1988) Domaln 18. Baroudi, J., Ginzberg, M.: Impact of the technological environment on programroer/analyst job outcomes. CACM 29, 546-555 (1986) 19. Baxter, 1.: Design maintenance systems. CACM 35, 73-89 (1992) 20. Davies, S.: Characterizing the program design activity. Brr 10, 173-190 (1991) 21. Deepak, J., Maher, M.: Combining expert systems and CAD techniques. In: Artificial Intelligence Developments and Applications, 65-81. Amsterdam: North-Holland 1988 22. Fisher, A.: CASE: Using Software Development Tools (2d. ed.). New York: Wiley 1991 23. Forte, G.: Tools fair: Out of the lab, onto the shelf. IEEE Software 9, 70-79 (1992) 24. Houghton, R.: Annotated bibliography on software engineering environments. SEN 10, 62-76 (1985) 25. Humphrey, W.: Software and the factory paradigm. SEJ 6, 370-376 (1991) 26. Jones, M., Arnett, K.: CASE use is growing, but in surprising ways. Datamation 38:10, 108-109 (1992) 27. Kemerer, C.: How the learning curve affects CASE tooi adoption. IEEE Software 9, 23-28 (1992) 28. Klaczko-Ryndziun, S., Goller, M.: Autornatic reasoning for creativeness decision support. In.: Human Aspects in Computing, 894-898. Amsterdam: Elsevier 1991 29. Liker, J., Fleisher, M., et al: Designers and their machines: CAD use and support in the U.S. CACM 35, 77-95 (1992) 30. MacDonald, 1.: Autornaring Information Engineering. In: D. Benyon, S. Skidmore (eds.) Automating Systems Development. New York: Plenum Press 1988 31. Majchrzak, A., Collins, P., Mandeville, D.: A quantitative assessment of changes in work activities resulting from computer-aided design. Brr 5, 259-271 (1986) 32. Martin, J.: Computer-Aided Software Engineering. Marblehead, Mass.: James Martin Report 1987 33. Minsky, M.: A framework for representing knowledge. In: Mind Design: Philosophy, Psychology and Artificial lntelligence. Montgomery, Ala.: Bradford Books 1980 34. Perlman, G.: Software tools for user interface developmenL In.: M. Helander (ed.) The Handhook of Human Computer lnteraction. Amsterdam: North-Holland 1988 35. Seppanen, V., Heikkinen, M., Lintulampi, R.: SPADE- Towards CASE tools that can guide design. In: R. Andersen, J. Bubenko, A. Solvberg, Advanced lnformation Systems, 222-239. Berlin: Springer-Verlag 1991 36. Stefik, M.: Planning with constraints. Artificial lntelligence 16, 111-140 (1981) 37. The, L.: Bridging the CASElOOP gap. Datamation 38:5, 63-64 (1992) 38. Wassennan, A., Pircher, P., et al: Developing interactive systems with User Software Engineering methodology. IEEE Transactions on Software Engineering 12, 326-345 (1986) 87 88 Technologles 39. Ambardar, A.: Human-computer interaction and individual differences. In: G. Salvendy (ed.) Human-Computer lnteraction, 207-211. Amsterdam: Elsevier 1984 40. Baeker, R., Buxton. W. (eds.): Readings in Human-Computer lnteraction. Los Altos, Calif.: Morgan Kaufman 1987 41. Billman, B., Courtney, J.: Automated discovery in managerial problem solving: Formation of causal hypotheses for cognitive mapping. Decision Sciences 24, 23-41 (1993) 42. Boettcher, K., Tenney, R.: Distributed decisionmaking with constrained decisionmakers. IEEE Transactions on Systems, Man &: Cybernetics 16, 813-823 (1986) 43. Boming, A., Duisberg, R., et al: Constraint hierarchies. SJGPLAN Notices 22, 48-60 (1987) 44. Bowne, J., O'Grady, P.: A technology for building life-cycle design advisers. In: G. Kinzel, S. Rohde (eds.) Computers in Engineering 1990, 1-7. New York: ASME 1990 45. Browne, D., Norman, M.: Adaptive User Interfaces. London: Academie Press 1992 46. Carroll, I., Aaronson, A.: Learning by doing with simulated intelligent help. CACM 31, 1064-1079 (1988) 4 7. Carroll, J ., McKendree, I.: Interface design issues for advice-giving expert systems. CACM 38, 14-31 (1987) 48. Cesta, A., Romano, G.: Explanations in an intelligent help system. In: H.-I. Bullinger (ed.) Human Aspectsin Computing. Amsterdam: Elsevier 1991 49. Curtis, B.: Human Factors in Software Development. Washington, D.C.: IEEE Computer Society 1985 50. Darses, F.: The constraint satisfaction approach to design: A psychological investigation. Acta Psychologica 78, 307-325 (1991) 51. Dechter, R. (ed.): Working Notes of the 1991 Spring Symposium on Constraint-Based Reasoning. Tech. Rpt. 91-65, University. of Califomia. at lrvine. American Association for Artificial lntelligence 1991 52. Duncan. N., Paradice, D.: ldentifying creativity enhancing features for expert systems. Proceedings, Decision Sciences lnstitute, 591-593 Atlanta: DSI 1992 53. Elam, I.: Can software influence creativity. lnformation Systems Research 1, 1-22 (1990) 54. Ettema, J.: Studies in creativity and constraint ERIC ED 259 367 (1985) 55. Fischer, G., Mastaglio, T.: A conceptual framework for knowiedge-based critic systems. DSS 7, 355-378 (1991) 56. Freedy, A., Madni, A., Samet, M.: Adaptive user models: Methodology and applications for man-machine systems. In: Rouse, W. (ed.) Advances in Man-Machine Systems Research 2, 249-293. Greenwich, Conn.: IAI Press 1985 57. Gould, J., Lewis, C.: Designing for usability: Key principles and what designers think. CACM 28, 300-311 (1985) 58. Guindon, R., Curtis, B.: Control of cognitive processes during software design: What tools are needed? In: Human Factors in Computing Systems CH/'88, 263-268. Proceedings 1988. New York: ACM 59. Hancock, P., Chignell, M.: Intelligent Interfaces. Amsterdam: North-Holland 1989 60. Harrison, M., Thimbleby, H.: Formal Methods in Human-Computer lnteraction. Cambridge, UK: CUP 1990 61. Hartson, H.: Human-computer interface development: Concepts and systems. ACS 21, 5-92 (1989) 62. Holtzman, S.: Intelligent Decision Systems. Reading, Mass.: Addison-Wesley 1989 63. Jacob, V., Pakath, R.: The roles of computerized support systems: A decision subprocess-based analysis. BIT 10, 231-252 (1991) 64. Kaiser, G., Feiler, P., Popovich, S.: Intelligent assistance for software development and maintenance. IEEE Software 5, 40-49 (1988) 65. Kieras, D., Polson, P.: An approach to the formal analysis of user complexity. IJMMS 22, 365-394 (1985) 66. Kling, R., lacono, S.: The control of information systems developments after implementation. CACM 27, 1218-1226 (1984) 67. Laurel, B.: Constraints. In: B. Laurel, Computers As Theatre, 99-112. Reading, Mass.: Addison-Wesley 1991 68. Liang, T.-P., Jones, C.: Design of a self-evolving decision support system. JourNJI of Management lnformation Systems 4, 59-82 (1987) 69. Mackay, J., Barr, S., Kletke, M.: An empirica! investigation of the effects of decision aids on problem-solving processes. Decision Sciences 23, 648-672 (1992) 70. Mackworth. A.: Constraint satisfaction. In: S. Shapiro (ed.) The Encyclopedia of Artificial Intelligence, 205-211. New York: Wiley 1987 71. Marxhall, R., Bregant, G.: The Overseer: An approach to design management. In: F. Rammig (ed.) Tooi Integration and Design Environments, 289-309. Amsterdam: Elsevier 1988 72. McKendree, J., Zaback, J.: Planning for advising. In: E. Soloway, D. Frye, S. Sheppard (eds.) Human Factors in Computing Systems CH/'88, 179-183. New York: ACM 1988 73. Moffitt, K., Hershauer, J., Reneau, H.: Knowledge transfer from interacting with expert system explanation facilities. In: J. Carey (ed.) Human Factors in lnformation Systems: An OrganizatioNJI Perspective, 253-269. Norwood. N.J.: Ablex 1991 74. Nielsen, J.: Usability Engineering. London: Academie Press 1992 75. Papstein, P., Frese, M.: Transferring skills from training to the actual work situation: The role of task application knowledge, action styles and job decision lattitude. In: Soloway, E., Frye,. D. and Sheppard, S. (eds.) Human Factors in Computing Systems CH/'88, 55-60. Proceedings, Washington, O.C. 1988. New York:ACM 76. Potosnak, K.: Mental models: Helping users understand software. IEEE Software 6, 85-88 (1989) 77. Preece, J .• Keiler, L.: Human-Computer lnteraction, Selected Readings. Englewood Clüfs, NJ.: Prentice-Hall 1989 78. Rasmussen, J.: lnformation Processing and Human-Machine lnteraction. New York: North-Holland 1986 79. Rouse, W.: Adaptive aiding for humanicomputer controL Human Factors 30, 431-443 (1988) 80. Sata, T., Warman, E. (ed.): Man-Machine Communication in CAD/CAM. Proceedings, IFIP WG 5.2-53, Tokyo 1980. Amsterdam: North-Holland 1981 81. Savolainen, T.: Expanding human-computer interaction by computer-aided creativity. lnteracting with Computers 2, 161-174 (1990) 82. Schauer, H.: What kind of involvement do we envisage for designers and users. In: Briefs, U., Tagg, E. (eds.) Education for System DesignerlOser Cooperation, 13-17. Amsterdam: Elsevier 1985 83. Serrano, 0.: Managing constraints in concurrent design: First steps. Computers in Engineering 1990, 159-164. New York: ASME 1990 84. Shackel, B. (ed.): Human-Computer lnteraction: Interact '84. Amsterdam: North-Holland 1985 85. Shneiderman, B.: Designing the User Interface. Reading, Mass.: Addison-Wesley 1987 86. Silver, M.: Systems That Support Decision Makers. Chichester, UK: Wiley 1991 89 87. Silvennan, B.: Survey of expert critiquing systems. CACM 35, 106-127 (1992) 88. Stohr, E., Konsynski, B.: lnformation Systems and Decision Processes. New York: IEEE 89. Tan, B., Lo, T.: The impact of interface customization on the effect of cognitive style on information system success. BIT 10, 297-310 (1991) 90. Taylor, R.: Perception of problem constraints. Management Science 22, 22-29 (1975) 91. Todd, P., Benbasat, 1.: An experimental investigation of the impact of computer based decision aids on decision making strategies. lnformation Systems Research 2, 87-115 (1991) 92. Weber, 0.: Information management: C:ombining computer-controlled with human-eontrolled functions in CIM. In: H.-J. Bullinger (ed.) Hurnan Aspects in Computing, 1026-1027. Amsterdam: Elsevier 1991 93. Wetzenstein-Ollenschlager, E., Wandke, H.: Toward adaptive human-computer interfaces. In: D. Ackermann, M. Tauher (eds.) Mental Models and Human-Computer Interaction 1, 231-252. Amsterdam: Elsevier 1990 94. Williges, R., Elkerton, J., et al: User assistance in human-computer interfaces. In: R. Ehrich, R. Williges (eds.) Human-Computer Dialogue Design, 291-317. Amsterdam: Elsevier 1986 95. Zmud, R.: Individual differences and MIS success: A review of empirica! literature. Management Science 25, 966-979 (1979) Cognitive Factors 96. Anon.: Mental workload: Research on CAD work and on the implementation of office automation. Espoo, Finland: Helsinki University of Technology. NTIS PB92127372XSP, May 24, 1991 97. Ackennann, D., Tauber, M.: Mental Models and Human-Computer lnteraction. Amsterdam: North-Holland 1990 98. Barki, H., Huff, S.: Change, attitude to change, and decision support system success. lnformation and Management 9, 261-268 (1985) 99. Bloomfield, B.: Understanding the social practices of systems developers. Journal of lnformalion Systems 2, 189-206 (1992) 100. Brookes, R.: Towards a theory of the cognitive processes in computer programming. /JMMS 9, 737-751 (1977) 101. Carey, J.: The issue of cognitive style in MIS !OSS Research. In: J. Carey (ed.) Human Factors in lnformation Systems: An Organizational Perspective, 337-348. Norwood, N.J.: Ablex 1991 102. Carroll, J.: lnterfacing Thought. Cambridge, Mass.: MIT Press 1987 103. Dagwell, R., Weber, R.: Systems designer's user models: A comparative study and methodology. CACM 26, 987-997 (1983) 104. Darses, F.: Constraints in design: Towards a metbod of psychological analysis. In: Diaper, D., Gilrnore, D., et al Human-Computer lnteraction., 135-139. Proceedings, INTERAIT90. Amsterdam: North-Holland 1990 105. Gardiner, M., Christie, B.: Applying Cognitive Psychology to User-Interface Design. Chichester, UK: Wiley 1987 106. Hirschheirn, R.: The effect of a priori views on the social implications of computing: The case of office automation. ACS 18, 166-195 (1986) 107. Hockey, R.: Styles, skilis and strategies: Cognitive variability and its implications for the role of mental models in HCI. In: D. Ackennann, M. Tauher (eds.) Mental Models and Human-Computer Interaction 1, 113-129. Amsterdam: Elsevier 1990 108. Kydd, C.: Cognitive biases in the use of computer-based OSS. Omega 17, 335-344 (1989) 109. Lindsay, P., Norman, D.: Hurnan lnformation Processing. New York: Academie Press 1977 90 110. Muir, B.: Trust between humans and machines, and the design of decision aids. IJMMS 27, 527-539 (1987) 111. Nonnan, D. Some observations on mental models. In: A. Stevens, D. Gentner (eds.) Mental Models, 7-14. Hillsdale, N.J.: Lawrence Erlbaum 1983 112. Orlikowski, W.: Social implications of CASE tools for systems developers. Proceedings, Boston 1989, 199-210. 10th International Conference on Infonnation Systems 113. Rasmussen, J.: A cognitive engineering approach to the modeling of decision making. In: Rouse, W. (ed.) Advances in Man-Machine Systems Research 4, 165-243. Greenwich. Conn.: JAI Press 1988 114. Rossen, M., Maass, S.: The designer as user: Building requirements for design tools from design practice. CACM 31, 1288-1298 (1988) 115. Suchman, L.: Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge, UK: CUP 1987 116. Vessey, 1., Galletta, D.: Cognitive fit: An empirica! study of information acquisition. lnformation Systems Research 2, 63-84 (1991) 117. Waern, Y.: Cognitive Aspects of Computer Supported Tas/cs. Chichester, UK: Wiley 1989 118. Winograd, T., Flores, F.: Onderstanding Computers and Cognition. Wokingham, UK: Addison-Wesley 1986 119. Woods, D.: Cognitive engineering: Human problem solving with tools. H uman Factors 30, 415-430 (1988) 120. Woods, D.: Coping with complexity. In: Goodstein, L., Andersen, H., and Olsen, S. (eds.) Mental Models, Tasles and Errors. London: Taylor & Francis 1988 Potenttal Confounds 121. Day, D.: Human factors issues in the electtonic newsroom. In: M. Nunninen, G. Weir (eds.) Human Jobs and Computer Interfaces, 81-100. Proceedings, Tampere, Finland 1991. IFIP WG 9.1 Werking Conference. Amsterdam: North-Holland 1991 122. Springer, J., Langner, T., et al: Experimental comparison of CAD systems by stressor variables. International Journal of Human-Computer Interaction 3, 375-405 (1991) 123. Taylor, R.: Infonnation use environments. Progressin Communication Science 10, 217-255 (1991) 124. Cohen, B.: Psychosocial environments created by computer use formanagers and systems analysts. Human-Computer lnteraction, 379-384. Amsterdam: Elsevier 1984 125. Dawson, P., Mclaughlin, 1.: Computer technology and the redefmition of supervision. Journalof Management Studies 23, 116-132 (1986) 126. Karake, Z.: lnformation Technology and Management Control: An Agency Theory Perspective. New York: Praeger 1992 127. Kling, R.:, Scacchi, W. The web of computing: Computer technology as social organization. Advances in Computers 21, 1-90 (1982) 128. Leifer, R., McDonough, E.: Computerization as a predominant technology affecting work group structure. Proceedings, International Conference on Information Systems 1980, 238-248 129. Markus, L.: Power, politics and MIS implementation. CACM 26, 430-444 (1983) 130. Srinivasan, A., Kaiser, K.: Relationships between selected organizational factors and systems development. CACM 30, 556-562 (1987) 131. Turner, J.: Computer mediated work: The interplay between technology and structured jobs. CACM 27, 1210-12 (1984) 132. Weber, R.: Computer technology and jobs: An impact assessment model. CACM 31, 68-77 (1988) 91 92 133. Wisner, A., Daniellou, F., et al: Place of work analysis in software design. In: G. Salvendy (ed.) Human-Computer lnteraction, 147-156. Amsterdam: Elsevier 1984 134. Yaverbaurn, G.: Critical factors in the user environment: An experimental study of users, organizations, and task. MIS Quarterly 12, 75-88 (1988) 135. Zuboff, S.: In the Age of the Smart Machine: The Future of Work. New York: Basic Books 1988 Behaviors 136. Bailey, R.: Human Error in Computer Systems. Englewood Cliffs, NJ.: Prentice-Hall 1983 137. Banker, R., Datar, S., Kemerer, C.: A model to evaluate variables impacting productivity on software maintenance. Management Science 37, 1-18 (1991) 138. Ciampi, A., Silberfield, M.: Measurement of individual preferences: The importance of situation-specific variables. Medica/ Decision-Making 2, 483-495 (1982) 13 9. Czaja, S., Hammond, K., et al.: Aged related differences in learning to use a text-editing system. BIT 8, 309-319 (1989) 140. Egan, D.: lndividual differences in human computer interaction. In: M. Helander (ed.) Handbook of Human Computer lnteraction, 543-568. Amsterdam: E1sevier 1988 141. Einhorn, H., Hogarth, R.: Behaviaral decision theory: Process of judgment and choice. Annual Review of Psychology 32, 52-88 (1981) 142. Payne, I.: Contingent decision behavior. Psychological Bulletin 92, 382-402 (1982) 143. Rasch, R., Tosi, H.: Factors affecting software developers' performance. MIS Quarterly 16, 395-409 (1992) 144. Wickens, C.: Engineering Psychology and Human Performance. Columbus, Ohio: Merrill 1984 145. Zavala, A.: Stress and factors of productivity among software deve1opment workers. In: G. Salvendy (ed.) Human-Computer lnteraction, 365-370. Amsterdam: Elsevier 1984 Percep ti ons 146. Bittner, U., Schnath, I., Hesse, W.: Werkzeugeinsatz bei der entwicklung von software-systemen. Software Technik Trends 13, 14-23 (1993) 147. Fishbein, M.: Attitude and the predierion of behavior. In: Fishbein, M. (ed.) Readings in Attitude Theory and Measurement. New York: Wiley 1967 148. Hanson, S., Rosinski, R.: Pregrammer perceptions of productivity and programming tools. CACM 28, 180-189 (1985) 149. Huuhtanen, P., Seitsamo, I.: Changes in mastery of computer applications and endusers' evaluations of the implementation process and psychological well-being. In: H.I. Bullinger (ed.) Human Aspects in Computing: Design and Use of lnteractive Systems and lnformation Management, 1075-1079. Amsterdam: Elsevier 1991 150. Knauth, P., Ioseph, J., Gemunden, H.: User acceptance of computer aided design (CAD). In: H.-I. Bullinger (ed.) Human Aspectsin Computing: Design and Use of Jnteractive Systems and lnformation Management, 1070-1074. Amsterdam: Elsevier 1991 151. Melone, N.: A theoretica! assessment of the user-satisfaction construct in information systems research. Management Science 36, 76-91 (1990) 152. Norman, R., Nunamaker, J.: CASE productivity perceptions of software engineering professionals. CACM 32, 1102-1108 (1989) 153. Rushinek, A., Rushinek, S.: What makes users happy? CACM 29, 594-598 (1986) 154. Salancik, G., Pfeffer, J.: An examinatien of need-satisfaction models of job attitudes. Administrative Science Quarterly 22, 427-456 (1977) A design for accessing multimedia information using cohesion Roger Espinosa and Patricia Baggett School of Education, University of Michigan, Ann Arbor, MI 48109-1259, USA Keywords. Accessing information, hypermedia, multimedia, cohesion Abstract. This paper presents a design for multimedia information access basedon cohesion and reportsonsome implementations giving it a limited test. Elementsin a database are cohesive if they refer to the same concept. The designer selects cohesive elements and sequentially links those referring to the same concept, forming a "subway line" for each concept. Cohesive elements can be in different modalities, e.g., graphics, text, photos. Users are given access via the "subway lines." At a given "stop" the user can either fellow the subway line he or she is currently on (by default), or change to another subway line which crosses that stop but which links elements referring toa different concept. At each stop, a list of concepts, and thus subway lines crossing at the stop, can be made available for the user, acting as an index of currently active concepts. This methad of accessing data was designed as an alternative to menus and keyword approaches. We have implemented the methad in several domains, including a repair task, literature analysis, a history of mathematics, and elementary rnathematics lessen plans. 1. Overview 1.1. The problem Often when one is presented with a large amount of computerized information, one is faced with developing a way for it to be browsed. This paper describes a design for accessing computerized information. The information to be browsed can be lengthy and can consist of material from different modalities, such as text and pictures. A unique feature of the approach given hereis that after the material is divided into units by the designer, accesslinks between units arebasedon cohesive elementsin the materiaL Two elements are cohesive if they refer to the same concept, even if they occur in different modalities. Users can fellow "subway lines" through the material; each line sequentially links elements referring to the same concept. At a given "stop" the user can either follow the subway line he or she is currently on (by default), or change toanother subway line which crosses that stop but which links elements referring toa different concept. 1.2. Why this approach? 1.2.1. It is different from a hierarchical or menu structure and is advantageous for loosely connected heterogeneaus materiaL It is also convenient for hierarchical material in which the hierarchy is not known to the designer. 95 96 1.2.2. The schema is motivated by a theoretica! framework in cognition (expanded on below). 1.2.3. At a given "stop" on a subway line, the user is not forced to make a choice about where to go next; there is always a "default," namely, continue on the current subway line to the next stop. Only if the user wishes to follow a concept other than the one he or she is currentl y following must a decision be made. 2. Description of the basic design 2.1. The data structure and visual cohesion Suppose that there is a large amount of material that is to be computerized and made accessible for browsing. The material is first divided into units (sometimes called frames or pages or screens), and each unit can be viewed as a node in a graph. The nodes are connected together by links, and a person using the system can traverse the material by traveling along the links. Soa link represents the possibility of going from one node to another. A determination of which nodes are linked together is made as follows. In each node, there is a set of designated elements. A link 1 between node i and node j is arelation between a unique element x in node i and a unique elementyin node j. The link means that x and y are cohesive elements. For example, node i might contain a picture of a motor, and node j the word "motor." The picture and the matching verballabel refer to the same concept, and so can be designated as cohesive elements. To give a brief background, the concepts of text cohesion and coherence have been used in similar ways by many authors (Halliday, 1985; Halliday & Hasan, 1976; Grimes, 1975; Harris, 1952; Kintsch & vanDijk, 1978). Cohesion is asemantic relation between two items in a text. For Halliday & Hasan (1976) cohesion occurs through word repetition, a noun and its pronoun referent, use of synonyms, etc. According to these authors, the semantic continuity provided by cohesion is a primary factor in a text's intelligibility. Baggett & Ehrenfeucht (1982) extended text cohesion notions to visual, and to text and visual, or between-media cohesion. They described a "cohesion graph" for 23 frames (still photos) taken from an animated movie of James Thurber's The Unicorn in the Garden. Ten of the photos (which were drawn for us by Pam Hoge), and the cohesion graph created from all23 arranged in the order in which they occurred in the movie, are shown in Figures 1 and 2. The cohesion graph was formed as follows. Nineteen concepts whose referents occurred in the photos, such as characters (e.g., husband, unicorn, wife) and specific locations (garden, bedroom, etc.), were selected. If a referent to any of the 19 concepts (i.e., a cohesive element) occurred in a photo, a mark was made in the cohesion graph, as shown in Figure 2. For example, it was determined that a picture of the husband occurred in 14 of the 23 photos. In the cohesion graph, the number of times cohesive elements occurred in two adjacent photos (when the photos were put in the order in which they occurred in the movie) was indicated. For the picture of the husband, this number was 11. The sum of these adjacencies was defined as ;, ·- ....... , 'f.ll, riT~ ; ~~··~ Figure 1. Ten of the 23 photos used in the Unicorn intheGarden ordering task. Nurnbers in upper left corners indicate the position in the actual story. • Coheelve El emer'lt huebend ~ unlcorn ~ wlfe ~ peychletrlel 0 pollee • gerden ~ bedroom ;; hall ! llvln'foom L huebend ë ~ unlc:orn J ~wil• ~! mythlc•l ,.. ~ beaat 'i :t•ll i- ... heno booby >o ~ horn l ltlyiJMt•l• Photo tlumber • • • • 5 6 1 1 • 10 n n 13 14 • • • • 11 • • I I • • • • I I • • I • • • • • • • • • • • I 15 16 17 u •• 20 21 ttU '"' " I • I I • I s • • • I • • 1 • • • I • • 2 1 0 • • D • • a • I 10 1 ' - 0 0 0 0 0 J 0 0 0 46 Figure 2. Cohesion graph of movie photos in correct order, as they occur in the actual story. Markers indicate that a particwar cohesive element occurs in a given photo. The nurnber of times a given element occurs in two adjacent photos is recorded in the far right column under "Number of Adjacencies." 97 98 the cohesion value for the photos. · The cohesion value for the 23 photos in their correct order was 46. One experimental result from the study was that people whohad not seen the movie, when asked to put the 23 frames in an order so that they "make a good story," gave orders which on average had a cohesion value of 55.1, or nearly 20% greater than in the actual story. One interpretation for this result was that associations in memory may be made by an awareness of the existence of similar elements (e.g., as postulated by the 1978 theory of Kintsch et al.). Baggett, Ehrenfeucht, & Guzdial (1989) developed ·a prototype interactive graphics-based instructional system for assembly and repair, in which access was via visually (or graphically) cohesive elements. The graphics implementation, tagether with prototypes in literature analysis, a history of mathematics, and elementary rnathematics lesson plans, will be briefly described in section 3 below. 2.2. An example of a graph and its traversal Figure 3 shows a hypothetical small example of a graph created from eight units of information. (In actuality, of course, graphs are considerably larger.) There are 3 types of cohesive elementsin the figure, T, 0, and •!•. Note that the path linking tagether cohesive elementsof a given type does not branch, but farms a single line. In a given frame, the user can select, for a specific cohesive element, the "next" or the "previous" frame that contains that element, or the "home" (or first) frame that contains the element. For example, in frame 3, where there is a second occurrence of 0. selecting "next" for 0 takes one to frame 6; selecting "previous" takes one to frame 1, and selecting "home" takes one to frame 1. Similarly, in frame 3, selecting "next" for •!• gives frame 5; "previous" gives frame 2, and "home" gives frame 8. Although it was not done in Figure 3, "frame" could also be designated as a type of cohesive element. When this happens, "frame" acts as a page turner. By selecting "next" for frame, the user would go from frame 1 to frame 2, etc. 1 2 3 4 5 6 7 The physical crder is indicated by 11 21 31 ••• There are three different cohesive elementsl 0 I<- 1 and T. Figure 3. A graphof eight units of information and three types of cohesive elements. 8 •!-2.3. Options available to the user The following options can be made available totheuser in every frame: For any cohesive element in the frame, the user may be able toaskabout (1) the name of the cohesive element; (2) the number of occurrences of the element in the entire material; and (3) the number of the occurrence in the frame the user is currently visiting. So for the cohesive element "page number," the user could find out how many pages (or units) of information there are in the whole presentation, and which one is currently bf7ing visited. In Figure 3 the user might be in frame 6 and ask about cohesive element 0. Suppose the presentation is about an object to be repaired. The user might be informed that the name of the cohesive element is motor. He or she can also find out that there are four occurrences of motor as a cohesive element, and that the current occurrence is number three. The actual instances of "motor" might be moving video showing what happens when the motor is turned on, a cut-away diagram shpwing the interior of the motor, a text telling how to troubleshoot the object when the motor is dead, a list of partsin the motor, together with their photographs, etc. The point is that all the instances refer to the same concept. A user following a cohesive element will receive information on the same concept, but possibly in different modalities. As noted above, a path linking together cohesive elementsof a given type does not branch. The order of accessis determined by the designer. Also, not every reference to the concept (e.g., motor) has to be designated a cohesive element. For example, if the reference is of minor importance in a frame, the designer may decide to omit it as a cohesive element in the frame. To help the user keep from getting lost, a stack facility can be provided, so he or she can always go back to previously visited materiaL Other options available to users can be quite elaborate. For example, a user may be able to request, "Take me to the ninth stop on this line," or "Take me to the stop on this line that intersects with such-and-such other line." 2.4. Responsibilities of the designer. To design a presentation, a person must do five things: 1. Select a set of materials to be browsed. 2. Divide the material into units, which will be nodes in the graph. 3. Designate one or morepartsof each unit a cohesive elementsof a given type. 4. Give each cohesive element type (each subway line) a unique short name, which will be available to a user. 5. Specify, for each element in each unit, what its predecessor and its successor will be, in terms of access. 2.5. Graph theoretica! results that indicate this should be a good approach It is desirabie that any two nodes are connected by a short path. (It may be that the us~r needs ~o change subway lines in order to find the short path.) 99 100 There is a theorem (Bollobas, 1985) which states that if a graph is created randomly, then the expected distance between two nodes (i.e., the lengthof a shortest path between them) is almost as smallas is theoretically possible. {The theoretica! minimum depends mainly on the valency of the graph, namely, the number of nodes that can be linked directly to one node). This result indicates that the designer doesnothave to worry about short connections between nodes. This property is expected to occur by itself. 3. Descriptions of implementations In this section we briefly describe some implementations of the design. All were done on the Macintosh and are mouse-driven. 3.1. Graphics-based procedural instructions for repair of a 'string crawler' Figures 4, 5, 6, and 7 show the first four frames from an interactive graphics-based instructional system for assembly and repair, in which access was via visually (or graphically) cohesive elements (Baggett et al., 1989). The object in question was a "string crawler," a battery-powered 'vehicle' which travelled along a string when its switch was turned on. A user of the system could follow any of 21 different concepts (e.g., motor, batteries, wire, switch box) through the 41-frame presentation when given a task (to repair a broken string crawler). Cohesive elements were indicated by stars (see Figures 4 through 7); clicking on a star or its related object moved the user to the 'next' frame in the presentation that contained the object. There were 98 clickable stars in the presentation and thus an average of about 2.4 per frame. 3.2. Emily Dickinson poems Two frames from another implementation of the design, by the first author, are shown in Figures 8 and 9. They involve 183 Emily Dickenson poems and critiques of them offered by four authors. The cohesive elements in this case are 16 abstract themes, together with the poems themselves. In Figures 8 and 9, a poem subway line, [P 303] The Soul selects her own Society, is shown. (The subway line is indicated below the banded line on the lower left-hand si de of the screen.) The user is on stop 2 of 5 for this poem. Stop 1 was the poem itself, while stop 2 is the poem together with a critique of it by Charles R. Anderson. The sentence beginning "However fleeting love's glory ... " is highlighted because it is here in Anderson's critique that hls discussion of The Soul selects her own Society begins. There are three other cohesive elements (subway lines) active in this frame; two are thematic: Love Imagery and · Sexuality Imagery. And the third is the poem [P 249] Wild Nights - Wild Nights (Anderson is also discussingit in this frame). The user can get the menu of concepts, with those that are active in this frame blackened, by clicking on the compass icon. By clicking on any of the three active elements in the menu, the user can switch subway lines. While the Emily Dickenson implementation is not multimedia, it is basedon more than text matching. lts cohesive elements for the most part refer to abstract themes. · Strin Crewier Thls presentetien shows how the string erewier is constructed. At eech display, you houe seueral options. ~ Cllclclng on the torword orrow will moue you torword one frome. Cllclclng on the bock orrow will moue you boclc one frome. Cllclcing on o stor or its releled object will moue you to o frame wlth more lnformation about that object. Cllcking on the preulous boH will return you to the lost frame you ulslted. Cllclcing on buttons such as "Demo• ond "Assemble" will show you how an action Is performed. When you ore finishèd with the presentation, click on the eHit boH. Cllclc on the begin button to start. ( Begin ) Figure 4 Strin erawier === body string erawier ( Demo ) <::J Preuious 10 r:) ~ Figure 5 ----- Strin erawier ( Assembie ) connector motor ball with one grey end Figure 6 Preuious I 0 Pre~ious I 0 Figures 4, 5, 6, and 7. The first four frames from an interactive graphics-based instructional system. Cohesive elements are indicated by stars. Figure 7 ~ ~ ~ ~ ~ m . . " ~ 1 vold1 the eh lef pltft.ll oflhe love lyrk, the tondenc:y to exploit emotle>n for 111 own uko.lnrteod oho ~nero ter out of the connlctlng osp•cls of love, I Ir ecstuy and lts that conlalnr thr The Soul oelect1 her own Soclety-Then- rhutr I he Door-On herdtvlne Mejorlty Ootrode no mono-Unmoved- 1ho note1 I he Chuiots-At her low Gate-Unmoved-an E;.,perorbe kneellng On her Jl.ush met-[Oior lm~grr !I f.PIJOI IIIIH!fl'f!j Nn hun hnnq<~l !..1 ROlJOl lmol)l'f!j ncwlh htlll~l''"l Orstrnlr lmnqrr !I loue lmngr.ry P..imollt!f n- rllllh llllfof" notltlOM, W)Ck.Jd~tM esot,b.ar ,Mld Shtll svmbols. Mathematica! Concepts (1 of 6) The Zero One of the earliest and most Important lndlan contributions to mathernaties is the concept of~: Some scholars claim. however. that this concept developed in ancielf-l=hina or Babylon. The Mayan civilization developed the concept separately. The zero is important because lts use shcrtened the paperwerk involved in solving even sirnple equations. and some types of mathematica! operations were simplyimpossible to carryout without the concept of zero. In addition. cernputers could nothave been invented without the concept of zero. The first reierenee to the concept of zero is fcund in the 7th century AD. in the works of the mathematidan Brahmagupta. although the concept had been in use long befor.e that. I dîëk on any af thëSë irons to lëäïlï more about "zero". Fic:rnr"' 1n 3.3~ Where in history is mathematics? This implementation, done by studentsin two graduate software design classes taught by the second author, involves the six differen~ civilizatio~ which according to current knowledge developed rnathemahes at least m part independently: Africa, Sumer-Babylon, India, China, Maya, and Inca. Concepts whose referents make up the cohes~ve .el.ement lis~ indude zero, counting schemes, tools, games, and famous md1v1duals. F1gures 10 and 11 show two frames, from the Mayan and Indian dvilizations, linked by references to the concept zero. The implementation was originally laid out as a two-dimensional array. Narnes of the six civilizations labeled the columns, and each row was labeled by a concept. In theory, if all civilizations had references toeach concept, the user could travel through the matrix horizontally or vertically. In reality, however, it was not the case that a reference to each concept was included for each civilization. For example, a reference to the concept of zero was included only in the civilizations of Maya, India, and China. The implementation has been updated and extended over two semesters, and it is still in progress. The most recent addition is the concept of mathematica! games insome of the civilizations. 3.4. Elementary rnathematics lesson plans Figures 12, 13, and 14 show frames from a prototype implementation invalving a database of 101 lesson units from a new elementary rnathematics curriculum. The implementation was done by Barry Webster under the supervision of the second author, using materials written by Baggett & Ehrenfeucht (1993). The materials are meant for teachers. Figure 12 shows the 16 concepts which can be foliowed through the materialsin the prototype, and Figures 13 and 14 show two frames linked by the concept "area." The plan is for this prototype to be extendedtoa larger database (currently about 250 lessen units have been written); concepts yielding subway lines will be chosen from typical "scope and sequence" .charts found in elementary textbooks. Teachers will have the materials available electronically, and if they want to browse all units which contain a certain concept (e.g., place value), they can do so. Many lesson units contain illustrations, so that this implementation willas it expands have a multimedia database. 4. Advantages of this approach 4.1. Our work (Baggett & Ehrenfeucht, 1988; Baggett, Ehrenfeucht, & Guzdial, 1989) has shown that when material is presented passively toa learner (e.g., as in a videotape), the hierarchical structure of the material plays an important role in learning. But when the information is presented interactively, so that a user can traverse the material freely, cohesive elements are more important than hierarchical structure. This design gives principled access to information basedon one's being able to fellow a line of interest. 103 104 • Area Average Converston Dl viston Error Formule Fracttons Group Manlpulollve Mensurement Money Multtpltcotlon Negnllve Roundlng Set Volume f-1 Cohestye Elements: Select o cohesive element from the list ot left, then click 1t to see o lesson contolning the element. Click rtght ond left double arrows to go forward ond bock through lessons In lhe group. Keep going In the some directton to evenluolly gel here lo Coheslve Elemcnts. ~ C11ck thls messega to htde tt. 1-'t ~~ of a trianqle The area of a trianqle een be computed trom the formule area • base • heiqht. But how een we compute the area vhen the lenqths of the three sides a. b. end c. of the trianqle are qiven? The tollovinq formule was discovered by Heren of Alexandria (lst century A.D.): area • sqrt(s*(s-a)*(s-b)*(s-c)). vhere s • (a+b+c)/2. and sqrt is the square root. This toraula is net convenient tor hand celculations but een be computed with a calculator. The computation consists ot tvo perts: (1) computinq s. s-a. s-b. end s-e (2) computinq the square root of their product Part 1. Write a lst 2nd 3rd 4th list of tour lines: s-S-8• s-b• s-e• ,.,...,~,,.~;t.;.,..,...,,",..:.,,.uuuuo: j!illAl of a polnon Prerequisite: Computinq the area of a trianqle by Heron's formule. Props end tools: Polyqons cut trom stift paper or cardboard. or even drawn on paper. Ruler vith cm end mm. (Picture 1) Step 1. Divide the polyqon into triengles by dravinq straight lines. nark the edqes end lines by letters a.b •.... (Picture 2) Notlee that a polyqon een be divided into triengles in meny different vays. This does not in!luence the tinal result. The ditterences amonq results vill be because of unevoideble errors in meesurements end roundinq errors durinq calcule ti ons. Step 2. Caretully meesure ell sides of the trienqles. vith en accuracy ot one millimeter. end vrite down_the results. The results tor a tigure may look es tollovs: a • 8.3 cm b • 6. 2 cm 0 Figure 12 Figure 13 Figure 14 Figures 12, 13, and 14. Frames from a prototype implementation invalving lessen plans from an elementary rnathematics curriculum. 4.2. Since individuals are allowed to follow their own "tours" through lengthy material, rather than having to take "guided tours," they may take more active roles in exploring the material and get directly to the information that they want, quickly, without confusion, and without wading through irrelevant and distracting materiaL 4.3. On every frame the information that is available totheuser about cohesive elements can be implemented to act as a constant index. This on-line index helps him or her follow a line of inquiry, quickly spot old and new important topics in a unit, and be kept aware of where in the information he or she currently is. This type of information is not available in a book. It should give users a good "feel" for the material, and help them locate what they want efficiently. 4.4. The system allows the designer to bypass having to find the hierarchical structure of material to be presented. lt is basedon retrieval of information by concepts designated as important by the designer, and independent of the medium in which they are presented. Users explore and learn by browsing and following their own choices of concepts through the data. There have been attempts at developing on-line browsing systems in recent years, and electronic access to large boclies of material is the wave of the future. The novelty of our approach lies in arranging access so the users/learners can follow single cohesive lines through the materiaL The ideas are simple, and theoretica! results in random graph theory indicate that the method can result in high efficiency for users in exploring. locating, and studying information. 105 106 References Baggett, P. & Ehrenfeucht, A. (1982). Wormation in content equivalent movie and text stories. Discourse Processes, vol. 5, 73-99. Baggett, P. & Ehrenfeucht, A. (1988). Conceptualizing in assembly tasks. Huroan Factors, 30 (3), 269-284. Baggett, P. & Ehrenfeucht, A. (1993). A new elementary rnathematics curriculum that incorporates calculators. Unpublished materials. Baggett, P., Ehrenfeucht, A., & Guzdial, M. (1989). Sequencing and accessin interactive graphics-based procedural instructions. University of Michigan School of Education Technica! Report for Office of Naval Research, vol. 2 (1), August. Bollobas, B. (1985 ) . Random Graphs. New York: Academie Press. Grimes, J. (1975). The Thread of Discourse. Mouton Press, The Hague, 1075, Halliday, M. (1985). An Introduetion to Functional Grammar. Baltimore, MD: Edward Arnold. Halliday, M. & Hasan, R (1976). Cohesion in English. London: Longman Press. Harris, Z. (1950). Discourse Analysis. Language. 28, 1-30. Kintsch, W. & vanDijk, T. (1978). Toward a model of text comprehension and production. Psychological Review 85,363-94. "Class, you're oot making enough noise!" The Case for Sound-Effects in Educational Software BillGaver Rank Xerox Cambridge EuroP ARC 61 Regent Street, Cambridge CB2 lAB, UK 1 Introduetion In this paper, I argue that educatlonal software could benefit in a number of ways by the incorporatlon of sound-effects much like those used in movies, radio dramas, and video games. Classes should be tilled with the sounds of roaring machines, running water, even breaking glass - not made by the real thing, of course, but by educatlonal software. lf this seems crazy, consider KidPix™, an award-winning drawing program for children produced by Bröderbund Software (see Figure 1). KidPix is fun: It incor-porates a large number of innovatlve graphical tools with which kids can explore and play. This figure shows what a few of them do. Piek one paint brush and drops of ink follow the cursor. Piek another and trees grow wherever you press the mouse button. Yet another tums smooth mouse motloos into rough scribbles. Use one of the "rubber stamps" to put little drawings wherever you click. And if you don't like the result, you can erase it in a number of highly satlsfying ways -my favourite is the bomb. KidPix is an interestlog program in part because it uses the power of the com- · Fig. 1. KidPix™ uses a variety of sound-effects to accompany drawing. 109 110 poter to go beyond the usualliteral metaphors used in painting programs. But it is the sound that makes it more than just a novel drawing program: all the tools are accompanied by clever, useful, and appropriate sound-effects. Use the dripping paintbrush, for instance, and you hear the drops ofpaint seep down the screen. Use the scribbling paintbrush, and you hear a pencil scribbling on paper. Stamp an owl on the tree branch, and you hear the stamp slap against the page. And when you blow up the screen, you hear the explosion - the effect wouldn't be nearly so pow-erful otherwise. Are these sounds just gimmicks? Perhaps. But I argue that the sounds used in KidPix add significantly to its success in a number of ways, and that these actvan-tages are applicable to many other programs. 1.1 Wbat Sounds Add to Interaction First, sounds increase the tangibility of the interface, and help to explain what is happening. Remember that the users of KidPix are unlikely to trouble with manu-als or help systems - if they can read at all. The sounds help to make the program self-explanatory. Consider how confusing the scribbling paintbrush would beifit wasn't accompanied by the appropriate noise. Or how prosaic the bomb-eraser would beifit "blew up" your screen withoutmakinga sound. Sounds like these reinforce the visual expressionsof a metaphor. By adding redundant information about what is happening, they make the tools seem more real (and thus the inter-face more transparent). Because they also add new information- for instance, about the putative tooi and surface, in the case of the scribbling paintbrush - they can convey new information about what is going on in the interface. Second, sounds can help coordination with others, by allowing other kids (and teachers) to hear what they can't see. You don't need to be looking at the screen to know what tools a kid is using - you can hear the scribbling, dripping, and explo-sions. This may not be so important for programs like KidPix, but it is poten-tially quite useful for educational software, particularly collaborative systems in which several people may be working in the same program at the same time. Finally, sounds promote guided exploration, by selectively indicating attributes of interest. Real sounds inevitably accompany actions, determined by the laws of physics. Sound-effects have to be created; programs do notnatorally make sounds. This means that the designers of sound-effects have the opportunity to design sounds that highlight some aspects of an event and not others (in fact, they can't avoid it). So the scribbling paintbrush stresses the gestures of pencil on paper, while the dripping brush leaves out the surface, and emphasises the viscosity of the "ink." Again, this may not be so important for entertainment software like KidPix, but it can be valuable for educational software. 2 Auditory Icons The sounds used in KidPixTM were designed to be fun, not useful. It is doubtful that the designers sat down and asked themselves, "how can the sounds we use increase tangibility, promote coordination, and help with guided exploration?" They were, after all, primarily concerned with making their program entertaining and useful enough to sell. Thus while most of the sounds in this program are remarkably apt, few are actually that useful. Over the last several years, on the other band, I have been concerned with just these issues. The result is a strategy for creating auditory icons, everyday sounds designed to convey information about computer events by analogy with everyday events. The sounds I use are similar to those used in KidPix. But because the pur-pose of auditory icons is to provide information, ratber tban entertainment, a richer, more principled set of design principles have evolved. The basic idea bebind auditory icons is to map computer events to analogous sound-producing events (see Figure 2). For instance, wben a user selects a file, tbe act of clicking tbe mouse wbile tbe cursor is positioned over tbe icon suggests that a tapping noise migbt be an appropriate mapping. The result is a simple sound-effect, mucb like tbose used in KidPix. But auditory icons go beyond tbe sounds used in KidPix because tbey are parame-terized; tbat is, meaningful parameters of tbe computer and sound-producing events are mapped to one anotber. For instance, the size of tbe file might be indicated by tbe size of tbe virtual object you hear tapped. Tbe type of file can be conveyed by the material of tbe tapped object. And tbe overall disk space migbt be mapped to the overall reverberation applied to tbe sound, so tbat an empty disk sounds like a large, echoing hall, and a full disk sounds liked a small, cramped room. Parametenzing auditory icons bas two effects: First, it allows families of auditory icons to be created, wbicb convey ricb information in lawful ways. Second, it increases the variability of tbe sounds heard from tbe interface, and tbus helps pre-vent any given sound from becoming annoying tbrougb repetition. Auditory icons rely on a new approach to tbe psycbology of sound and hearing that stresses our tendency to bear sounds in terms of tbe events tbat cause tbem (see Gaver, 1993a, 1993b). Tbat is, if you are walking downastreet and bear a sound, you are less likely toponder its pitch, loudness, and timbre (all attributes of sound studied by traditional psycbophysics) and more likely to bear it as a large auto-mobile beading your way- and jump! Tbis experience of everyday listening implies tbat we can start to describe sounds and tbeir perceptual correlates in terros of events. Instead of descrihing a sound in terms of its pitch and tbe acoustic cocre-late of frequency, for example, we might describe it in terms of tbe size of its souree and the associated acoustic correlates. Tbis perspeelive leads to a new liJ ë Q) > w Computer World Everyday World event mapping » Fig. 2. Auditory icons are created by mapping events in the computer world to sound-producing events in the everyday world. They may be parameterized by mapping attributes of the computer and sound-producing events to one another. 111 112 approach to onderstanding the psychology of sound and hearing, as I point out above. More important, for the purpose of this paper, it leads to a new set of con-ceptual tools for building auditory interfaces: We can now think mapping events in the computer directly to sound-producing events. This is the strategy bebind audi-tory icons. In the rest of this paper, I describe auditory icons and their potentlal uses in edu-cational software. My discussion centers around the three functions of increasing tangibility, supporting coordination, and guiding exploration. Each of these func-tions is illustrated by a system that uses auditory icons. The first is the SonicFinder, an auditory extension to the desktop metaphor which makes the world of the computer more tangible and more self-explanatory. The second is SoundShark, and a related application ARKola, which adds sound to a prototype system of collaborative software meant to support distance education. The last is the Environmental Audio Reminder (EAR) system, which uses sound to convey selected information about events going on in our office environment. 3 Increasing Tangibility: The SonicFinder Educational software must achieve a number of goals. The primary one, of course, is to teach students about the subject matter at hand. But to do this, the students must leam to use the software itself. Thus a second goal is that the software should be readily learned and ideally self-explanatory. In addition, software should motivate students both to explore the particular contents it offers and the relevant domain more generally. This is a third challenge facing designers of educational software. Auditory icons may help in creating interfaces that are easy to learn and which motivate users by increasing the tangibility of the interface. A good example of this is the SonicFinder (Gaver, 1989). This is an extension to the Finder, the application used to organise, manipulate, create and delete files on the Macintosh. Creating the SonicFinder required ex tending the Finder code at appropriate points to play sampled sounds modified according to attributes of the relevant events. Thus a variety of actions make sound in the SonicFinder: selecting, dragging, and copying files; opening and closing folders; selecting, scrolling, and resizing windows; and dropping files into and emptying the wastebasket Most of these sounds are parameterized, although the ability to modify sounds was limited by the sampling software used. So, for instance, sounds which involve objects such as files or folders not only indicate basic events such as selection or copying, but also the object's types and sizes via the material and size of the virtual sound-producing objects. In addition, the SonicFinder incorporates an early example of an auditory process monitor in the form of a pouring sound that accompanied copying and that indicates, via changes of pitch, the percentage of copying that had been completed (c.f. Cohen, 1993). Figure 3 shows an example of an interaction with the SonicFinder. When a user selects a file icon (Figure 3A), a tapping sound is played which provides feedback about the event and also provides information about the file size and type. As the user drags the icon towards the wastebasket, a seraping sound is played (3b), which not only indicates that the object is being moved, but also reflects the size of the file and what it is being dragged over (i.e., its home window, another window, or the desktop). This sound stops either when the user releases the object, or when it Fig. 3: In the SonicFinder, many user-initiated events are accompanied by meaningful auditory icons, increasing tangibility and ease of learning. hits a possible container such as a folder. Finally, when the user throws the icon into the wastebasket (3C), marking it for deletion, the sound of crashing glass is heard. This provides useful feedback about the nature of the interaction, and also reflects the number of other objects in the wastebasket Sounds like these serve to provide redundant infonnation about events in the sys-tem, and thus increase its tangibility. Once users become accustomed to the SonicFinder, they feel somehow removed from the interface if the sound is tumed off. The sounds seemed to make interactions more immediate to users. Sounds also convey infonnation that is not conveyed graphically. For instance, the size and type of files is not indicated in most modes of the graphical display, yet can be readily judged (at least qualitatively) from the selection sounds. Sametimes sounds seem to convey infonnation more effectively than grapbic feed-back. For example, hearing when a dragged object is over a container seems more effective than highlighting for helping people avoid playing chase-the-wastebasket, and in rnaving files to new windows without losing them in unnoticed folders. Finally, there is some anecdotal evidence that the SonicFinder helped some new users understand the underlying me taphor of the Finder; by emphasising the tangi-bility of icons, they support the interpretation of simple line-drawings as "objects" that can be moved, opened, etc. In sum, the SonicFinder demonstrates the potential for auditory icons to provide rich information in intuitive ways. This can be helpful in making systems obvi-ous, and thus in allowing users - whether students or not - to focus on the content affered by the software rather than the task of figuring out how to use it. In addi-tion, auditory icons can increase the feeling of direct engagement (Hutchins, Hollan & Norman, 1986) with interfaces. They reinforce the feeling that the model world presented by graphical interfaces is a real one that can be interacted with in mean-ingful ways. Finally, interfaces that use auditory interfaces are more fun: Justas erasing the screen in KidPix would be boring if the bomb were silent, so selecting a file in the Finder is somehow less satisfying when it doesn't make a noise. 4 Supporting Coordination: SoundShark and ARKola One of the potential problems with using educational software is that it may sepa-rate students from each other and from their teachers, tuming classrooms into collections of individual users, each staring intently into bis or her own screen. Thus another challenge for the designers of educational software is to develop prod-113 114 ucts that support communication and coordination amongst students and their teachers. Auditory icons can help with this both within the physical classroom, and within collaborative virtual worlds used as "classrooms" for distance education. 4.1 SoundShark An example of this latter role for sound is provided by SoundShark, an auditory version of SharedARK (Smith, 1989). SharedARKis a collaborative version of ARK, the Altemate Reality Kit. Developed by Smith (1987), ARK is designed as a virtual physics Iabaratory for distance education. The "world" appears on the screen as a flat surface on which a number of 2.5D objects may be found. These objects may be picked up, carried, and even thrown using a mouse-controlled "hand." They may be linked to one another, and messages may be passed to them using "buttons." Using this system, a number of simple physical experiments may be performed. In addition, SharedARK allows the same world to beseen by a number of different people on their own computer sereens (and is usually used in conjunction with audio and video links that allow them to see and talk to one another). They may see each other's hands, rnanipolate objects together, and thus collaborate within this virtual world. SharedARK is a multiprocessing system, with the potential for several "machines" or self-sustaining processes to run simultaneously. In addition, it pro-vides a very large world to users, in that the space for interaction is many times larger than the screen (depending on available memory, it may cover literally acres of virtual space). This means that simultaneous users of the system may not be able to see each other (or more accurately, the hands that represent them), despite their being in the same "world" and potentially changing it in ways that might affect each other. To help collaboration in this large, complex world, we extended the SharedARK interface with auditory icons that indicate user interactions, provide background information about ongoing processes and modes, and support navigation. The result is called SoundShark (Gaver and Smith, 1990). Sounds are used to provide feedback as they were in the SonicFinder: Many user actions are accompanied by auditory icons which are parameterized to indicate attributes such as the size of rele-vant objects. In addition, ongoing processes make sounds that indicate their nature and continuing activity even if they are not visible on the screen. Modes of the system, such as the activation of "motion," which allows objects to move if they have a velocity, are indicated by low-volume, smooth background sounds. These auditory icons are helpful for individual users. More interesting, the sounds support collaboration among remote users. Because each sound can be heard tbrooghout the "world," collaborators can hear each other even if they can't see each other. In addition, the distance between a user's hand and the souree of a sound is indicated by the sounds' amplitude and by low-pass filtering. This not only seems to aid navigation (prompting us to develop "auditory landmarks," objects whose soul function was to play a repetitive sound that could aid orientation) but allows collaborators to hear the di stance and even direction of their partners. 4.1 The ARKola Simulation Our experiences with SoundShark suggested that auditory icons could help collabo-ration amongst distributed colleagues interacting in a virtual environment. To test this, we developed a special application within SoundShark that we used for observing people's use of the system. Our aim was to assess the usefulness of auditory icons for collaborators within the system as well as for individual users. The application we came up with is a model of a softdrink plant called the ARK.ola bottling factory (Figure 4; Gaver et al, 1991). It consistsof an assemb1y line of 9 machines which cook, battle, and cap cola, provide supplies, and keep track of financing. The plant was designed to be fairly difficult to run, with the rates of the machines requiring fine tuning and with machines occasionally "breaking down," necessitating the use of a "repair" button. In addition, we designed the plant to be too large to fit on the computer screen, so partleipants could only see about half the machines at any given time. Each of the machines makes sounds to indicate its function. For instance, the "nut dispenser" makes waoden impact sounds each time a nut is delivered to the cooker, the "heater" makes a whooshing flame-like sound, the "bottler" clangs and the "capper" clanks. In addition, the rate of each machine is indicated by the rate of repetition of the sounds it makes, and problems with the machines are indicated by a variety of alerting sounds such as breaking glass, overflowing liquid, and so forth. As with SoundShark, the sounds were designed not only to be useful for individ-ual users, but also to be helpful in coordinating partners running the plant. We tested this in a simple observational study. Six pairs of partleipants were asked to run the plant with the aim of making as much "money" as they could during an hour-loog session. Each pair ran the plant for two hours, one with and one without auditory feedback (with the order, of course, being counterbalanced). We observed their performance from a" control room" via video links as they ran the plant, and videoed their actlvities for later analysis. Our observations indicated that sounds were effective in helping individual users keep track of the many ongoing processes. The sounds allowed people to track the activity, rate, and functioning of normally running machines. Without sound, . . . . • . . ............... , . ': I I .. . . I I I I . Fig. 4. The ARKola bottling plant simulation (about one fourth actual size). Reetangles show the extent of the plant each user sees at a given time. 115 116 people often overlooked machines that were broken or that were not receiving enough supplies; with sound these problems were indicated either by the machine's sound ceasing (which was often ineffective) or by the varloos alert sounds. Perhaps most interesting, the auditory icons allowed people to hear the plant as an inte-grated complex process. The sounds merged tagether to produce an auditory tex-ture, much as the many sounds that make up the sound of an automobile do. Partleipants seemed to be sensitive to the overall texture of the factory sound, refer-ring to "the factory" more often than they did without sound. These observations support the idea, introduced inSection 3, that well-designed sounds can help new users learn how a system works. These observations also support the thesis of Section 4, that sounds are particularly useful in guiding explo-ration of systems by providing information about a relevant subset of events. Finally, sound seemed to add to the tangibility of the plant and increased partici-pants' engagement with the task. This became most evident with a pair of participants who had completed an hoor with sound and were working an hour without. From the video, their increasing boredom with the silent plant is obvious (as are several mistakes that they attribute to the lack of sound). Finally, one of the pair remarks "we could always make the noises ourselves ... " Out major findings, however, related to the role of sound in collaboration. In both the sound and no-sound conditions, participants tended to divide responsibility for the plant so that each could keep one area on the screen at all times. Without sound, this meant that a each had to rely on their partner's reports to tell what was happening in the invisible part. With sound, each could hear directly the status of the remote half of the plant. This led to greater collaboration between partners, with each pointing out problems to the other, discussing problems, and so forth. The ability to provide foreground information visually and background information using sound seemed to allow people to concentrate on their own tasks, while coor-dinating with their partners about theirs. It was an effective way of linking partici-pants in this model world without forcing them to be tagether all the time. This is likely to be a very useful feature for helping with the balance between self-guided work and coordination in educational systems of the future. 5 Guiding Exploration: EAR Still another challenge facing designers of educational software is in balancing between tightly-constrained teaching systems and systems that allow free explo-ration on the part of students. Traditional "drill and test" systems seem boring, have dubious efficacy, and make limited use of the potentlal power that computing offers. Newer "exploratory learning" systems, on the other hand, may be confusing and, moreover, make it difficult to coordinate the material that students actually learn as they wander through complex virtual environments. A middle ground between these extremes may be found in systems which support guitled exploration of complex environments. The idea is to constrain students' explorations of rich informational spaces to a set of trajectories that cover a subset of relevant information. For instance, fictional characters have been used to guide access to a bistorical database using the conventions of narrative flow (Laurel et al., 1990). Systems such as these offer a good deal of flexibility to teachers: Students may be assigned one particwar trajectory, permitted to chose amongst a number of altematives, or allowed simply to wander freely through the information space. Auditory icons can help create systems that offer guided exploration by allowing a subset of information to be presentedat a given time. From this point of view, soundscan be designed not to convey all possible information, but to present that which is necessary to highlight a partienlar task. For instanee in SoundShark different sounds might be used for lessons involving gravity (for which mass, distance and the like are important) than for those involving the coordination of complex processes (like ARKola, in which the relative rates of the machines was most important). Using sounds, different aspects of the "world" can be emphasised for different purposes. 5.1 Environmental Audio Reminders The key to using sounds that guide exploration is in designing them clearly repre-sent only the relevant information for a given task, as opposed to all the information available. As an example of this, consider the Environmental Audio Reminders (EAR) system we use at EuroPARC (Gaver, 1991). This system plays a variety of nonspeech audio cues to offices and common areas inside EuroP ARC to keep us informed about a variety of events around the building. EAR works in conjunction with the RAVE audio-video network (Gaver et al., 1992; Boxton & Moran, 1990), which connects all the offices at EuroP ARC with audio and video technologies using a computer-controlled switch, and Khronika (Lövstrand, 1991), an event server which uses a database of events in conjunction with software daemons to inform us of a wide range of planned and spontaneous, electtonic and professional events. EAR, then, consists of sounds triggered by Khronika when relevant events occur, which are routed using the RAVE system from a central server to any office in the building. This system is set up to play sounds that remind us about a range of events. For instance, when new email arrives, the sound of a stack of papers falling on tbe floor is heard. Wben somebody connects to my video camera, the sound of an opening door is heard just before the conneetion is made, and the sound of a closing door just after the conneetion is broken. Ten minutes before a meeting, the sound of murmuring voices slowly increasing in number and volume is played to my office, then the sound of a gave!. And when we decide to call it a day, one of us may play the "pub call" to interested colleagues, who then hear laughing, chatting voices in the background with the sound of a pint glass being fllled with real ale in the fore-ground. Many of the sounds we use in EAR may seem frivolous because they are cartoon-like stereotypes ofnaturally-occurring sounds. But it is precisely because they are stereotyped sounds that they are effective. More "serious" sounds- such as elec-ttonic beeps or sequences of tones - would be likely to be less easily remembered than these. In addition, we have taken some care in shaping the sounds to be unob-trusive. For instance, many of the sounds are very short; those that are longer have a relatively slow attack so that they enter the auditory ambience of the office subtly. Most of the sounds have relatively little high-frequency energy, and we try to avoid extremely noisy or abrupt sounds. So though the sounds we use are stereotypes, they are designed to fit into the existing office ambience rather than introding upon it. EAR serves as a useful example of many of the themes of this paper: They are effective because tbey are stereotyped sounds that are easily learned and remembered. 117 118 They promote coordination by mak:ing information available to the distributed memhers of our labs. The overall effect is to allow us to hear events as if they were just outside our offices, despite the fact that these events are either too distant to hear naturally or have not even occurred yet The point bere, however, is that while EAR creates an intuitively accessible audi-tory environment that complements and supplements our everyday one, it is not a slavish imitation of all the sounds we might potentially hear around our building. We don't play the sounds ofpeople entering and leaving the building, for instance, or those of people typing, moving things in their offices, etc., though in principle we could. These events do not appear relevant to the tasks that we are supporting, and thus would merely be distracting. Instead we choose the sounds we play to indicate an important subset of the ongoing activities in the building. Moreover, individual users may tailor the system, registerlog interest in some events (e.g. meetings), but not others (e.g., pub time). In this way, the system supports a guided trajectory through the information that might be available. It offers the abil-ity to rnaintaio awareness of remote events without distracting or confusing us with too much data. 6 Conclusions None of the systems described in this paper- KidPix, the SonicFinder, SoundShark and ARK.ola, or EAR- were explicitly designed for educational purposes. But all have something to say about how to make such systems easier to use, more motivating, more supportive of group work, and better able to guide students in their leaming about complex, rich environments. Sounds can make systems easier to leam and more enjoyable to use. They are useful in supporting coordination without enforcing togetherness, because they can allow background information to be conveyed using the auditory channel while foreground, task-specific information can be provided visually. Finally, by providing only a subset of information, it is possible to use auditory icons to guide exploration along trajectories that are useful and desirable. So while the idea of designing educational software to be more noisy may have seemed counter-intuitive at the start of this paper, I hope that it is now possible for readers to imagine a day when teachers will be pleading with their students to make more noise. References Buxton, W., and Moran, T. (1990). EuroPARC's integrated interactive intermedia facility (iiif): Early experiences. In Proceedings of the IFIP WG8.4 Conference on Multi-User Interfaces and Applications (Herakleion, Crete, September 1990). Cohen, J. (1993). Kirk here: Using genre sounds to monitor background activity. Adjunct Proceedings of INTERCH/'93 (Amsterdam, May 1993). ACM, New York. Gaver, W. W. (1989). The SonicFinder: An interface that uses auditory icons. Human-Computer Interaction. 4 (1 ). Gaver, W. W. (1991). Sound support for collaboration. Proceedings of the Second European Conference on Computer-Supported Collaborative Work (Amsterdam, September 1991) Kluwer, Dordrecht. Gaver, W. W. (1993a). How do we hear in the world? Explorations of ecological acoustics. Ecological Psychology (5) 4. Gaver, W. W. (1993b). What in the world do we hear? An ecological approach to auditory souree perception. Ecological Psychology (5) 1. Gaver, W. W., & Smith, R. B. (1990). Auditory icons in large-scale collaborative environments. Proceedings of Human-Computer Interaction - Interact'90 (Cambridge, U.K., August 1990) North Holland, Amsterdam. Gaver, W. W., Moran, T., Macl.ean, A., Lövstrand, L., Dourish, P., Carter, K., & Buxton, W. (1992). Realizing a video environment: EuroPAR C's RAVE system. Proceedings ofCH/'92 (Monterey, CA., May 1992). ACM, New York. Gaver, W. W., Smith, R. B., & O'Shea, T. (1991). Effective sounds in complex systems: The ARKola simulation. Proceedings of CHI 1991, New Or!eans, April 28 -May 2, 1991, ACM, New York. Hutchins, E. , Hollan, J. , & Norman, D. (1986). Direct manipulation interfaces. In D. A. Norman & S. W. Draper (Eds.), User centered system design: New perspectives on human-computer interaction .. Hillsdale, NJ: Lawrence Erlbaum. Laurel, B., Oren, T., and Don, A. (1990). Issues in multimedia interface design: Media integration and interface agents. Proceedings of CH/'90 (Seattle, April 1990) ACM: NewYork. Lövstrand, L. (1991). Being selectively aware with the Khronika system. In Proceedings of ECSCW'91 (Amsterdam, September 1991 ). Smith, R. (1987). The Alternate Reality Kit: an example of the tension between literalism and magie. Proceedings of CHI + GJ 1987 (Toronto, April 1987) ACM, New York. Smith, R. (1989). A prototype fotoristic technology for distance education. Proceedings of the NATO Advanced Workshop on New Directions in Educational Technology. (Nov. 1988, Cranfield, UK.) 119 Prosody in experimental dialogues Ronald Geluykens and Marc Swerts Institute for Perception Research (IPO), P.O. Box 513, Nl-5600 MB Eindhoven, The Netherlands E-mail: geluyken@heiipo5.bitnet; Fax: +32-3-220 44 20 It is widely recognized that prosody plays a predominant role in the organization of discourse. More spe ei fically, i t may be u:;ed both to signal information structure [topic boundaries] and to govern interaction [turn boundaries]. From the rec•!nt di scourse-analyti c 1 i te rature deal ing wi th prosody in conversa ti • >n, i t is not clear, however, how these two dimensions may intera•:t. Moreover, prosodie analyses have mainly been limited to the study of final pitch movements only, though one often has the impression that discourse units can also have more global prosodie correlates. Both these questions, i.e. (i) interaction of prosodie cues to topics and turns and (ii) local and global prosodie demarcation of discourse units, are addressed in this paper in an experimental manner. From five pairs of test subjects, three types of discourse were elicited, each consisting of a speaking and a listening task. In all conditions, speakers were not allowed to use lexica! or syntactic cues to clarify discourse structure, and therefore could only rely on prosodie devices. In condition 1, a monologue setting designed to test information structure independently from turn-taking considerations, speakers were instructed to describe series of concatenations of differently colored geometrical figures [e.g.' a red triangle'] in such a way that major breaks between series became apparent, while listeners had to try and detect these breaks and mark them on an answer sheet. Condition 2 involved enforced turn-taking independently from information structure: participants were asked to take over turns at predetermined points in their descriptions of a simple row of geometrical figures·. In condition 3, finally, both variables --topic and turn boundaries--were combined to create a more complex setting, as topical breaks could occur within turns. The use of prosody as a marker of discourse structure appears to depend largely on the type of discourse setting. Firstly, on a local level, the nature of final pitch movements is different across condi ti ons: i t is not the case tha t final i ty is always signalled by means of a falling contour. Secondly, on a more global level, several prosodie devices --accent shifts from noun to adjective, topline declination, and relative heights of pitch maxima within NPs-- are employed to signa! discourse structure. These tendencies increase as a function of the complexity of the speaking task; in other words, they are most outspoken in condition 3 . This indicates that speakers indeed employ prosody both on a local and a global level for structuring dialogue discourse, both from the point of view of topic organization and from the point of view of interaction. These acoustic results will be discussed at the workshop, tagether with their perceptual evaluation. 121 .. The Writing lnstruction Script Hebrew (WISH) System Ab.wac1 Jon W. M~Keebyl, Yael M. Moses and Rachelle S. H:!llcrl 1 Department ofElec~cal E:tgineering and Computer Scknee 2oepLU"tment of Classic ai Languages George Wa5hington University, Wa§hlngton, D C. 200S2, USA This paper will de:.dbe an effectiv,~ too! fcr the tesehing of hanowriting of liJngua5.::g thM utilize non-Romar. characrer sets b;:ing develop~d at George W~shingron C'niv~rsity The tooi urili;o~s a rdcrocomputt:r, videodisc pleye:-, ::J.nd n sty!us and a grephl·:s tablet. to en:1bk >tL:derJtS tn revi:v. lhe ~ar;d.,·r!tin~ ;roctss, te~: h.:ndwritin;; abîlides, while instmc!lng thë student on hendwrit:ng skil!s. Kevw_grrl;;: Multimedia, Computer Assl'>ted !nst:uction, Foreign Language 1.0 BllCkground In the early seventies two self-proc!abted successf'Jl CA! handwriting sy~te.:n:; for ~cn-Romar. languages were de-veloped. Abboud's sy~tem (197::!), designed to instruct students in rhe writ~ng of Arabic, consisted of a computer and a grease pend;. Students w:re instructed to write letter~ on tht. computer screen b; usk~ the grease penci!. After the student had completed this ~ask. thc smdent was presented !he correcr lerrer forrr.at~on on the screen requirlng the st.:dents to 11nalyz:: th:ir own hand· ... ·riting ski!:s. Through an empirical study, Abboud reponed that students taugin '..!:;ins four to eight hours of CAI instructien and fcur hcurs of das~ïoom instructien pe:icrmed ~~gnlfi~ant!y better tian ~!udents Hr..ïght ~hrough a prol1r.ammtd instruction courl'e o;: those nudents taught through the audlo-lin$Ui:l methcd. However, ii ~hculd be noted that the audio~lingual methods base its instri.l~tion on listeni;)& and )peak!n$ a::d \'ery !htle or no writing. l11 add!tion to the increased profid~ncy in handwriting •nechanics, studCritSo that used t.he CAI system had a better attitude about ciass work and wer!! more inrer;;sted t:l çontinuing th!!'ir Ar.abic lsngnnge education. prorid~ fetdbaék rë!n.ting te rh~ srud~nt:, inpuued letter. First. ::he norrna: hund\\'titir~g po~ltior! ~ons.is~s of t!le student resting his.!her arm on a deskwp s:1rface. a5 op~.-::.sed to writi~g a !i!tter oa a ;;omputer 5creen with hislher hand positioned fr.!ely in a!r. The second problem r~sults fro!I~ ~he stude.nts e,·aluating their own wri~ten letter. Even though th~ studer.ts are able to g&in son:e visual feeèbeck f:om examinin,g a v. ritten ietter against examples of tb:: actua! letter, their cvalua:ion.s rnay be ir:terp:eted in.;orrectly. fn aèdition, the ~omparison of a wrinen letterand a target lctt~r dCJes not pro•;ide the :m:óet.r wit:l in formation ~.~OLit the letter fannation mechanics. Thus, the student's evaluation of hislher performa."l.:e i;., at be~t. incomplete. The system by Chuang and Chen (1973) was designed to tead: the Miting techniquts, assist in the handwriting pro.:es;; stroke b)' stroke, e'creen (.Figure ij. During the brer~ctive writing stage, srudenrs· hand\Hitin,g aèi!ities are: examined ~nd .;;v!!!uated both b) stuéent self-inspectiün and by conputer genera:e.j evûlualions. Letter WrlHng Wr1te the Ct.11'81Ve Script fa the0 BbckLetter ,,.,. Figure 1. The Letter Writing Screen A5: the studer.t writes a let~er. the wriuen !etter's (x,y) points and its bc.undr.r)' box ar-:. ca!culated. From th;! boundary box, the letter is reslzed wlth rM::-ect to the target letter and th~n ovtr1aid on ;op of the target !ett~r through a iransforrnadcn of ~he (X,y) points. The vis u al .:va.iuation providr:d by everlll)' ing thc writtl!i1 letter on the target :i!tttr provldes immediate and constrUcûve feedback to the stl.ident. In addition to this immediate visual f~edbaclo::, the ;orrectness of the writt~n letter is determined end feedbfl.ck given to thë user in order to impro\e hislher handweitins skilb. Tv determine the con-ectness cf the writtèn l.:tter, it is necessary to perform a detailed evaluo.tion of the \Witten leaer b> examining its attribmes (Table 1). 125 ,. 126 Shape Proportional Size Position Stroke Direction Stroke Ord~r Prvponionàl Spadng Letter formlappearance The size in rela.tion to the other written letters. T'ne horizonral .and vertic al posltion uf the whole letter as well as componentsof the lett~:r. Di:ecticn of writing stroke. The order in which mokes are ~odtten. Since most Hebrew cursive letters nre one !itroke le:te~s. stn_,~c: order wlll a!so be u~ed to ref::: !o th~ connectivit.>' of a one-stroke letter. The sp::tcing benveen kners iind lines of iettets Table 1. Lettor Atuibutes By dic.k.i!lg on the desireë letter in the !~tter writlng sectian, the aser is able T0 review rhe attribute the ,~·riting of the letreL Th~ student can contin~e to the ni!Ä! lett<:r .:ftc.r $Jc:ce~;ful!y writing ~t lea~t 5 let~ers of the current letter. In addition to the ktttr writir.g section of the interactive writing st;>gt, chi!n~ is a practice se.;;t:on ..... ... ne praetiet section al!ow5 the st~.:dent to write :he cursive leners .• .,ithir: woïds to pr::1::tk:: his/her 'Nritit\g ski!ls. The cn!y constraint on the p;act:;:;~e s;::dor. is thc:t !.rud.;nt:; are only allo>v~J w p~acri.:t: !etters compieted iï1 r...'Je letter writing stage. Here, too, the wrirten targt:t letter is overbid e>n the target lette: and the :;rudent is able to revie\;,· the attribute eväluativn of :he V/fitten target ietter by sel~cting rhe d~lir::-·1 \vri~ten Jec:er. Letter Evaluati on EYII~Itbn Of: 0 Yam L&t1&r nug.êt t ane r lr:l S~p~ El9rl;a»d Fitlll'ld V Pçsiiera · ~131 n Spi.e& Mlln S:;:;tO@I ! J llsm.Y• i:IIIQ~e~ttan~ All Ü!Ji·~.r.~ cou~r r.r,......l0tf'l €ll 1 j Dlr&t:llön Cl~r,·1föa ~~~ On• 01~ Drdsr !SD'CI nex.:e~sary to pe!form a d~tailed evaluation of th~ written letter by e~;amining each anribr:~ of ti1e wrlt:er: letter. As ~h:: algcri:hm of the written letter wit:.'! re~pe.:t toeach Ietter &tttibute for me t.irget letter. In addition, a gdd~ is provided w the studer.t tO help lncrease the correc:ness in lëtter formation a.r.d f~ppearance . 2. I. I Thc...Hcllrcw Cursive Alpbabel ln o:der w idi'!ntify th~ attrlbute:S of (he Hebtew .;~mivc alphabet, it \vas nccessary to d.:l~:mir.e a Si!t of gen::ral ch.::ractcristics for Hebrew cursi ve let:ers as w~Il as \he letter attribut:s for each chd:'~cter cf :he Hebrew a!phabet. A detailed attribute listbg for each letter and iclen!ified !he optima! let~er us ,,.dl sas ac::eprable matches was created. The shape attribute d.:;te:miM a charact.ers po!lition. Upper Space Main Space Lower Space ~ -4 --r-~ Pigure '3. The Space, Quadrant. and Space und Quadrant System fo:- me Hc:brew Letters. 127 128 -· _ ..._ ·-· :.- ._IJ. • '- ·-2. I .2 Sbape Eyaluatjon The most diffic~o;lt of !!.11 letter nttributes to evaluate is the kttu'; ~h.ipe. For the \\'ISH :.)~tem, the visualization of the chsractM a3 wel! as a !iimple template matching algorithm are used te evaluate the correctne~s of the letter. Given !hut many of the letters of rhe Htbrew cursivl!: ~.lphabet art comprised of 31CS and str3ight lines it was possible to deveiop a m~n letter is compared to the target lisring with all differences noted in order to provide student feedback. ,. 2 1 6 Stroke Qcdcr Eyaluarion The St."'ke order of thc entered Ierter is determined by ordering the strokes as they are writter. by the student and compruing them tö th~ order of .strokes of thc cxpe:.:ted letter. Sine~ most letten or..l}' u:>e one stroke, stroke order is also used ro refer tO connect: vity. For letter5 that are nnly onc s;;ct.e, th~ character is con:.ide:ed to be disconn-ected if the student lifts the stylu~ bt>yond ~he height o! the pro).imity of thc pen styius and gra.phics tablet while writing. 2..1..1Progortiona! SpacinK Evaluation l'roportiur.al spaci11;5, the $pace between letters and Ii~es. is c.nl~· caku!a:: of a non -Roman tar.gu.agé hä:;; been descri bed. Currently, the pri:r1ary objectiv~ is ro create a CAI system to effectively tea"h the handwriring of Hebrew cursive :.crlpt characto;rs. Usability testing and educaric,nal impact studies are piarmed for the 199'3-94 academie ye!lr. !'v1odifications to the \\'riting Instll.lctior. Script Hebrew (WISH) systtm wil! be mad~ to anable 1t to eff~ctivel~ te:.ï.:-h ether non-Roman languages, such as Ancl~nt Gl'êek, Russian, Chin~se, Japanese, Korean, and Arabic, as wen. Re( erences Abboud. V.C. (1972, ~~brch). The computer ElS an inst:uctienal device for th~ Arabic w!iting ;,ystern Co.m;utcrs and the H1Jmaniües, Q,1_, 195-207. Brod! R~.t ( 1988), Fo1eign Lar.gu~ge Enrollments in tJS In.stitutions of Higher Edt.fcation. .~J)l:"L ~ul1e~h1 1{12), 39-44. Browr., RM. (December, 1965). On-line computer .re.:-ognitio~ óf handprimed characters. ~EE T~·;;nsl)ctiv:>~ Qll E!eqror:ic Computer<:, 750· '752. C;;.sey, R. & Nagy. G. (1966, Febn:ary). Recognitit'n of prit.td Chinese characrers. !EEE TranndC'li...Qll '!=:'e,'fr•1ni • E -'" , 91-101. Chuo.ng. H.Y., Chen, S {19i3, August). "Cctr.puter Aided Instruct.:on in Chir.ese Charac:ers". P•0ceerl''T' of :h~firo:t Incernachrsl .;vrr.,q.;;jnm gn r;gmr:uws r.nd ~. inp;:~1''~UI.fiH ·~vMem-., 599 - 6! 6. Govindan, V.K. & Shivaprasaà, A.P. (1990). Cha;acter recog:nition · A r~vi~w. I';!trf:rn Rt::J:ogniüon. fJ., 1, 671-683. Heller. R.S. & Mcses, Y.M (July i990). OT L'OT: A pilot program - An interactive Computf'r a1èed instructien tutoria1 fcr the writing of the Hebre\\ alphabet. A Pr>uN~ill w TB:\1 Co~llli.Qn George Washin~ton Univ~rsit). Wa!ihmgton. D. C. Srnith, W. ( 1989). M..Q~n Technolo~n· in fill~i~r Langu~!: Educ.a.r.iQn.;..ëtsoll~ns and Proiects. National Tex.tbook Company. Llncolnwood, Illinoi,!;. St.~llings, W.W. (1975). Approaches to Chin~se cha.racter recogni:lon. f~~ro Reco~nit;oo. 8. 87-98. Stcllingll, W.W. (1977). Chinese character recognition. In K.S. Fu (Ed.), CQ:nrauniqtlon Synta.ctk Patleen ~.iOitiow Ap~llwio~. (pp. 95-123). Berlin, Germany; Springer-Verlag. 129 130 Ward, J.R. & Blesser, B. (September 1985). Jnternctive recc~nltion of handprinted characters fcr computer input. I~EE Coml1.Yln GriJ.pjc<; and A~!icmions. 24 - 37. \Voolsey, Kristina Hooper. (Spring 1991). A discu:..si~1n with Jen ~t:Ke!'b~. R.1chei1e Heller, and Kristina Hocper Woolsey r~garding the \VISH system. The Learner's Partner: Foreign Language Learning and Real World Encounters Stacie L. Hibino Computer Science and Engineering, Uni versity of Michigan, l1 01 Beal Avenue, Ann Arbor, MI 48109-2110, USA The Learner's Partner is a pedagogical model using interactive multimedia for foreign language learning. This model is designed for individual student use (or for use by pairs of students) and focuses on several aspectsof language acquisition, including: viewing and listening, reading, writing, speaking, comprehension, and cultural understanding. A variety of actlvities are provided at increasing levels of interactivity and difficulty, thus providing some scaffolding to.the student while addressing individuallearning styles and needs. Video is at the heart of the model, providing cultural context, modeling, and real world encounters. Two types of video segments are used - real-life scenarios (e.g., buying a bus ticket) and short cultural reports. Activities for the scenarios focus primarily on comprehension and speaking skills,leading up to role playing. Activities for the cultural reports focus on comprehension and writing,leading up to student research reports. Applications using the Learner's Partner (LP) model have been developed in French, Spanish, and Chinese, for first and second year students. LP activities in Hebrew are also currently being developed. The applications have oot yet been incorporated into classroom use, but some preliminary pilot testing bas been done. My role in the LP bas not only involved design, but also the development of a template. The LP template incorporates authoring tools for each of the activities, thus allowing foreign language experts (i.e., end-user programmers) to easily develop LP applications while still having some room for creativity. This poster session will display and descri he each of the LP activities, as well as some of the authoring tools used by the developers. Acknowledgements Learner's Partner applications are being designed and developed by several memhers of Project FLAME (Foreign Language Applications in a Mul ti me di a Environment) at the University of Michigan. Besides myself, the design and development team include: Professor Edna Coffin (project director; Hebrew), Gonzalo Silverio (Spanish), Joanna Porvin (French), and Pamela Colquitt (Chinese). New Human-Computer Interaction Techniques Robert J.K. Jacob Human-Computer Interaction Lab, Naval Research Laboratory, Washington, D.C., U.S.A. Abstract. This chapter describes the area of human-computer interaction technique research in general and then describes research in several new types of interaction techniques under way at the Human-Computer Interaction Labaratory of the U.S. Naval Research Laboratory. 1. Introduetion Tufte [12] has described human-computer interaction as two powerful infonnation processors (human and computer) attempting to communicate with each other via a narrow-bandwidth, highly constrained interface. A fundamental goal of research in human-computer interaction is, therefore, to increase the useful bandwidth across that interface. A significant bottleneck in the effectiveness of educational systems as well as other interactive systems is this communication path between the user and the computer. Since the user side of this path is difficult to modify, it is the computer side that provides fertile ground for research in human-computer interaction. This chapter describes interaction technique research in general and then describes research in several new types of interaction techniques under way at the Human-Computer Interaction Labaratory of the U.S. Naval Research Laboratory. Interaction techniques provide a useful focus for human-computer interaction research because they are specific, yet not bound to a single application. An interaction technique is a way of using a physical input/output device to perfonn a generic task in a human-computer dialogue [1]. It represents an abstraction of some common class of interactive task, for example, choosing one of several objects shown on a display screen. Research in this area studies the primitive ele-ments of human-computer dialogues, which apply across a wide variety of individual applica-tions. The basic approach is to study new modes of communication that could be used for human-computer communication and develop devices and techniques to use such modes. The goal is to add new, high-bandwidth methods to the available store of input/output devices, interaction techniques, and generic dialogue components. Ideally, research in interaction tech-niques starts with studies of the characteristics of human communication channels and skilis and then works toward developing devices and techniques that communicate effectively to and from those channels. Often, though, the hardware developments come first, people simply attempt to build "whatever can be built," and then HCI researchers try to find uses for the resulting artifacts. 2. Eye Movement-Based Interaction Techniques One of the principal thrusts in interaction technique research at NRL bas been eye movements [5, 6, 9]. We have been interested indeveloping interaction techniques basedon eye movements as an input from user to computer. That is, the computer will identify the point on its display screen at which the user is looking and use that information as a part of its dialogue with the user. For example, if a display showed several icons, a user might request additional information about one of them. lnstead of requiring the user to indicate which icon was desired by pointing at it with a mouse or by entering its name with a keyboard, the computer can detennine which icon the user was looking at and give the infonnation on it immediately. 135 136 Our approach to this interaction medium is to try to make use of natura! eye movements. This work begins by studying the characteristics of natura! eye movements and then attempts to recognize appropriate patterns in the raw data obtainable from an oculometer, turn them into tokens with higher-level meaning, and design interaction techniques for them around the known characteristics of eye movements. A user interface based on eye movement inputs has the poten-tlal for faster and more effortless interaction than current interfaces, because people can move their eyes extremely rapidly and with little conscious effort. A simple thought experiment sug-gests the speed advantage: Befare you operate any mechanica! pointing device, you usually look at the destination to which you wish to move. Thus the eye movement is available as an indica-tion of your goal befare you could actuate any other input device. However, people are not accustomed to eperating devices in the world simply by rnaving their eyes. Our experience is that, at first, it is empowering to be able simply to look at what you want and have it happen, rather than having to look at it and then point and click it with the mouse. Befare long, though, it becomes like the Midas Touch. Everywhere you look, another cammand is activated; you cannot look anywhere without issuing a command. The challenge in building a useful eye movement interface is to avoid this Midas Touch problem. Carefully designed new interaction techniques are thus necessary to ensure that they are not only fast but that use eye input in a natura! and unobtrusive way. Our approach is to trytothink of eye posi-tien more as a piece of information available to a user-computer dialogue invalving a variety of input devices than as the intentional actuation of the principal input device. A further problem arises because people do not normally move their eyes in the same slow and deliberate way they operate conventional computer input devices. Eyes continually dart from point to point, in rapid and sudden "saccades." Even when a user thinks he or she is viewing a single object, the eyes do not remain still for long. It would therefore be inappropriate simply to plug in an eye tracker as a direct replacement for a mouse. Wherever possible, we therefore attempt to obtain information from the natura! movements of the user's eye while viewing the display, rather than requiring the user to make specific trained eye movements to actuate the sys-tem. We partition the problem of using eye movement data into two stages. First we process the raw data from the eye tracker in order to filter noise, recognize fixations, compensate for local · calibration errors, and generally try to reconstruct the user's more conscious intentions from the available information. This processing stage uses a model of eye motions (fixations separated by saccades) to drive a fixatien recognition algorithm that converts the continuous, somewhat noisy stream of raw eye position reports into discrete tekens that represem user's intentional fixations. The tokens are passed to our user interface management system, along with tokens generated by other input devices being used simultaneously, such as the keyboard or mouse. Next, we design genetic interaction techniques basedon these tokens as inputs. The first interaction technique we have developed is for object selection. The task is to select one object from among several displayed on the screen, for example, one of several file icons on a desktop. With a mouse, this is usually done by pointing at the object and then pressing a button. With the eye tracker, there is no natura! counterpart of the button press. We reject using a blink for a sig-nal because it detracts from the naturalness possible with an eye movement-based dialogue by requiring the user to think about when he or she blinks. We tested two alternatives. In one, the user looks at the desired object then presses a button on a keypad to indicate his or her choice. The second alternative uses dweil time-if the user continues to look at the object for a sufficiently long time, it is selected without further operations. At first this seemed like a good combination. In practice, however, the dweil time approach proved much more convenient. While a long dwell time might be used to ensure that an inadver-tent selection will not be made by simply "looking around" on the display, this mitigates the speed advantage of using eye movements for input and also reduces the responsiveness of the interface. To reduce dweil time, we make ·a further distinction. If the result of selecting the wrong object can be undone trivially (selection of a wrong object foliowed by a selection of the right object causes no actverse effect-the second selection instantaneously overrides the first), then a very short dweil time can be used. For example, if selecting an object causes a display of information about that object to appear and the information display can be changed instantane-ously, then the effect of selecting wrongobjectsis immediately undone as longastheuser even-tually reaches the right one. This approach, using a 150-250 ms. dweil time gives excellent results. The lag between eye movementand system response (required to reach the dweil time) is hardly detectable to the user, yet long enough to accumulare sufficient data for our fixation recog-nition and processing. The subjective feeling is of a highly responsive system, almost as though the system is executing the user's intentions before he or she expresses them. For situations where selecting an object is more difficult to undo, button confinnation is used rather than a longer dweil time. Other interaction techniques we have developed and are studying in our laboratory include: continuous display of attributes of eye-selected object (instead of explicit user commands to request display); moving object by eye selection, then press button down, "drag" object by mov-ing eye, release buttontostop dragging; moving object by eye selection, then drag with mouse; pull-down menu commands using dweil time to select or look away to cancel menu, plus optional accelerator button; forward and backward eye-controlled text scrolling. Eye movement-based interaction techniques exemplify an emerging new style of interac-tion, called non-command-based [10]. Previous interaction styles all await, receive, and respond to explicit commands from the user to the computer. In the non-command style, the computer passively monitors the user and responds as appropriate, rather than waiting for the user to issue specific commands. Because the inputs in this style of interface are often non-intentional, they must be interpreted carefully to avoid annoying the user with unwanted responses to inadvertent actions. Our research with eye movements provides an example of how these problems can be attacked. 3. Three-Dimensional Interaction Another area of interaction technique research at NRL has been an investigation of three degree of freedom input [7]. In studying interaction techniques, each new piece of hardware that appears raises the question "What tasks is this device good for, and how should it be incorporated into interface designs?" Such questions are typically answered specifically for each new device, based on the intuition and judgment of designers and, perhaps, on empirica! studies of that device. Our work in three degree-of-freedom input provides an example of how greater leverage can be achieved by answering such questions by reasoning from a more general predictive theoretica! framework, rather than in an ad hoc way. We begin by posing the question for the three-dimensional position tracker, such as the Polhemus 3SP ACE or Ascension Bird trackers. While directly answering the question ''What is a three-dimensional tracker good for?" we also try to shed light on the next level question, i.e., "How should you answer questions like 'What is a three-dimensional tracker good for?'" Con-cepts such as the logica! input device provide descriptive models for understanding input devices, but they tend to ignore the crucial pragmatic aspects of haptic input by treating devices that out-put the same information as equivalent, despite the different subjective qualities they present to the user. Taxonomies and other frameworles for understanding input devices have tended to hide these pragmatic qualities or else relegate them to a "miscellaneous" category, without further structure. 137 138 Instead, we draw on the theory of processing of perceptual structure in multidimensional space [2, 3]. The attributes of objects in multidimensional spaces can have different dominant perceptual structures. The nature of that structure, that is, the way in which the dimensions of the space combine perceptually, affectshow an observer perceives an object. We posit that this dis-tinetion between perceptual structures provides a key to understanding perfonnance of multidi-mensional input devices on multidimensional tasks. Hence two three-dimensional tasks may seem equivalent, but if they involve different types of perceptual spaces, they should be assigned to correspondingly different input devices. The three-dimensional position tracker can be viewed as a three-dimensional absolute-position mouse or data tablet; it provides continuous reports of its position in three-space relative to a user-defined origin. The device thus allows a user to input three coordinates or data values simultaneously and to input changes that cut across all three coordinate axes in a single operation. (A mouse or trackhall allows this in only two dimensions.) Such a device is obviously useful for pointing in three-space, but it is also applicable in many other situations that involve changing three values simultaneously. We considered two tasks that both involve three degrees of free-dom, i.e., that require adjusting three variables. For comparison with the three-dimensional tracker, we used a conventional mouse (for two of the three variables in the tasks) and then pro-vided a mode change button to turn the mouse temporarily into a one-dimensional slider for the third variable. A naive view of these two alternatives suggests that the three-dimensional tracker is a superset of the two-dimensional mouse, since it provides the same two outputs plus a third. Thus the three-dimensional tracker should always be used in place of a mouse (assuming ideal devices with equal costand equal accuracy), since it is always at least as good and somelimes better. Our intuition tells us that this is unlikely-but why? The goal of this research is to develop a firmer foundation from which to draw such judgments. To do this, we extend Gamer's theory of pro-cessing of perceptual structure [3], first developed with fixed images, to interactive graphical manipulation tasks and thereby use it to shed light on the selection of multidimensional input devices. Garner obseiVed that relationships between the attributes of an object can be perceived in two ways that differ in how well the component attributes remain identifiable. Some attributes are integrally related to one another-the values of these attributes combine to form a single composite perception in the abserver's mind, and each object is seen as a unitary whole; while other attributes are separably related-the attributes remain distinct, and the obseiVer does not integrate them, but sees an object as a coneetion of attributes. Our hypothesis is that the structure of the perceptual space of an interaction task should mirror that of the control space of its input device. To examine it, we considered two interactive tasks, one set within an integral space and one in a separable one, and two input devices, one with integral dimensions and one, separable. This yields a two by two experiment, with four condi-tions. We expect performance on each task to be superior in the condition where the device matches that taskin integrality/separability. That is, the interaction effect between choice oftask and choice of device should far exceed the main effects of task or device alone. For the integral three-attribute task in the experiment, the user manipulates the x-y location and the size of an object to match a target, since location and size tend to be perceived as integral attributes; for the separable task, the user manipulates the x-y location and color (lighmess or darkness of greyscale) of an object to match a target, since location and color are perceived separ-ably. The difference in perceptual structure between these two tasks is in the relationship of the third dimension (size or greyscale) to the first two (x and y location); in all cases, the x and y attributes are integral. For the integral device condition, we use a Polhemus tracker, which permits input of three integral values. For the separable condition, we use a conventional mouse, which pennits two integral values, to which we added a mode change to enable input of a third-separable-value. Our hypothesis prediets that the three degree of freedom input device will be superior to the two degree of freedom (plus mode change) device only when the task involves three integral values, rather than in all cases, as with the naive hypothesis mentioned above. Our experimental results strongly supported this hypothesis. We found that neither device is uniformly superior to the other in perfonnance. Instead, we find significantly better perfor-mance in the experimental conditions where the task and device are both integral or both separ-able and inferior performance in the other two conditions. These results support our extension of the theory of perceptual space to interaction techniques, which prediets that the integral task (size) will be performed better with the integral device (Polhemus) and that the separable task (greyscale) will be performed better with the separable device (mouse). How might these results be used in designing controls for zooming and panning of a geo-graphic display? Zooming and panning, taken together, involve three degrees of freedom. The most common design uses a mouse or trackhall for two-dimensional panning and a separate con-trol for zooming. We claim that a user typically does not really think of zooming or panning operations separably, but thinks rather of integral operations like "focus in on that area over there.'' The space is thus Euclidean, like that of the size task in the experiment, and, therefore, making the user do the two separately violates perceptual compatibility. It would be more natura! to permit a user to make a gesture that perfonns the overall operatien he or she had in mind, using an integral three-dimensional input device. The user moves the puck around in a volume directly in front of the display screen. Moving it in the x or y direction parallel to the display surface causes panning; moving it perpendicular to the display (directly toward or away from it) causes zooming. The user typically moves the puck in all three dimensions simultaneously, resulting in some combination of zooming and panning and directly reaches the view of interest We have demonstrated a mockup of this application. 4. Egocentric Projection Recent work at NRL has greatly extended the notion of a three degree of freedom panning and zooming device to the concept of egocentric projection (to be discussed further in final version of this chapter). 5. Dialogue Interaction Techniques Another direction in our research is the notion of dialogue interaction techniques [8, 11]. In a direct manipulation or graphical interface, each command or brief transaction exists as a nearly independent utterance, unconnected to previous and future ones from the same user. Real human communication rarely consistsof such individual, unconnected utterances, but rathereach utter-ance can draw on previous ones for its meaning. It may do so implicitly, embodied in a conversa-tional focus, state, or mode, or explicitly. Most research on the processes needed to conduct such dialogues has concentrated on naturallanguage, but some of them can be applied to any human-computer dialogue conducted in any language. A direct manipulation dialogue is conducted in a rich graphicallanguage using powerful and natura! input and output modalities. The user's side of the dialogue may consist almost entirely of pointing, gesturing, and pressing buttons, and the computer's, of animated pictorial analogues of real-world objects. A dialogue in such a language could nevertheless exhibit useful dialogue properties, such as following focus. For examp1e, a precise meaning can often be gleaned by combining imprecise actions in several modes, each of which would be ambiguous in isolation. We thus attempt to broaden the notion of interaction techniques in these two dimensions (multiple transactions and multiple modes). A useful property of dialogue that can be applied toa graphical interface is focus [4]. The graphical user interface could keep a history of the user's current focus, tracking brief 139 140 digressions, meta-conversations, major topic shifts, and other changes in focus. Unlike a linguis-tic interface, the graphical interface would use inputs from a combination of graphical or manipu-lative modes to determine focus. Pointing and dragging of displayed objects, user gestures and gazes as well as the objects of explicit queries or commands all provide input to determine and track focus. Moreover, focus would not be maintained as a single object, but rather a history of the course of the user-computer dialogue. It is necessary to track excursions or digressions in the dialogue so focus can be restored as necessary. In addition, it is helpful to track focus by categories. This allows the user to refer to ''the ship'' even though the current focus is another object. In that case, the recent history of focus would be searched to find a ship of the appropriate category. Finally, focus is not necessarily a concrete object; it may be a class or category of objects ("all blue ships") or a more abstract entity ("the previous command"). Human dialogue often combines inputs from several modes. Deixis often involves a point-ing gesture that does not precisely specify its object; the listener deduces the correct object from the context of the dialogue and, possibly, from integrating information from the hand gesture, the direction of the user's head, tone of his or her voice, and the like. A user could, similarly, give a cernmand and point in a general direction to indicate its object. The interface would disambigu-ate the pointing gesture based on the recent history of its dialogue with the user and, possibly, by combining other information about the user from physical sensors. An imprecise pointing gesture in the general direction of a displayed region of a map could be combined with the knowledge that the user's recent commands within that region referred principally to one of three specific locations (say, river R, island /, and hili H) within the region and the knowledge that the user had previously been looking primarily at islands displayed all over the map. By combining these three imprecise inputs, the interface could narrow the choice down so that (in this example) island I is the most likely object ofthe user's new command. We call these higher-level interaction elements dialogue interaction techniques, and we have begun designing and testing them in our laboratory. We are also developing a software architecture for handling these properties that span more than one transaction. It treats them as orthogonal to the usual lexica!, syntactic, and semantic partitioning of user interface software. Our first system demonstrales the use of a focus stack in an interactive graphics editor. In the future, we will expand to a richer representation of dialogue than a stack, to support a wider range of dialogue interaction techniques. 6. Conclusions This chapter has provided an overview of a variety of new human-computer interaction tech-niques we are studying and building at NRL. Interaction techniques like these, when applied to the design of specific interfaces, increase the useful bandwidth between user and computer. This seems to be the key bottleneck in improving the usefulness of all types of interactive computer systems, and particularly educational systems, which depend heavily on dialogues with their users. Acknowledgments I want to thank my colleagues at the Naval Research Laboratory, Dan McFarlane, Preston Mul-len, Manuel Perez, Linda Sibert, and Jim Templeman. who have conducted much of the research I have described in this chapter. This work was sponsored by the Office of Naval Research. References 1. J.D. Foley, A. van Dam, S.K. Feiner, and J.F. Hughes, Computer Graphics: Principles and Practice, Addison-Wesley, Reading, Mass. (1990). 2. W.R. Garner and G.L. Felfoldy, "Integrality of Stimulus Dimensions in Various Types of lnfonnation Processing,'' Cognitive Psychology 1 pp. 225-241 (1970). 3. W.R. Gamer, The Processing of lnformation and Structure, Lawrence Erlbaum, Potomac, Md. (1974). 4. B.J. Grosz, "Discourse," pp. 229-284 in Understanding Spoken Language, ed. D.E. Walker, Elsevier North-Holland, New York (1978). 5. R.J.K. Jacob. "What You Look At is What You Get: Eye Movement-Based Interaction Techniques," Proc. ACM CH/'90 Human Factors in Computing Systems Conference pp. 11-18, Addison-Wesley/ACM Press (1990). 6. R.J.K. Jacob. "The Use of Eye Movementsin Human-Computer Interaction Techniques: What You Look At is What You Get," ACM Transactions on lnformation Systems 9(3) pp. 152-169 (Apri11991). 7. R.J.K. Jacob and L.E. Sibert, ''The Perceptual Structure of Multidimensional Input Device Se1ection," Proc. ACM CH/'92 Human Factors in Computing Systems Conference pp. 211-218, Addison-Wesley/ACM Press (1992). 8. R.J.K. Jacob. "Natura! Dialogue in Modes Other Than Natura! Language," in Natura/ Dialogue and lnteractive Student Modelling, ed. R.-J. Beun, Springer-Verlag, Amsterdam (1993). (in press). 9. R.J.K. Jacob. "Eye Movement-Based Human-Computer Interaction Techniques: Toward Non-Command Interfaces," pp. 151-190 inAdvances in Human-Computer lnteraction, Vol. 4, ed. H.R. Hartson and D. Hix, Ablex Publishing Co., Norwood, N.J. (1993). 10. J. Nielsen, "Noncommand User Interfaces," Comm. ACM 36(4) pp. 83-99 (April1993). 11. M.A. Perez and J.L. Sibert, "Focus in Graphical User Interfaces," Proc. ACM International Workshop on Intelligent User Interfaces, Addison-Wesley/ACM Press, Orlando, Fla. (1993). 12. E.R. Tufte, "Visual Design of the User Interface," IBM Corporation, Annonk, N.Y. (1989). 141 Yelena Korni~~~rov~ Psychologic.:al rnst.ltut.e~ t.iE:.·V. Uk! t.\.i.ne Hu man -c om p1...1 ter <.: ommLin J.."ç: •" ti on i. n t he 1 n s. t.n .. 1c t .l çm ë:\ l ~ y ""Lf'm~~, compar·ed t.o tt·,.;t .l.n i.hê' or·ri.ln.:>ry/c.~r;~nerc.d f'llrpos:E.' man-mBchine 5yl5tem~ and to f,tudr:-nt-·teacher· jnt~l'"action in the tr .. achtiOnr;\1 .1nstn .. lt.."tion, ht.\s .i.ts owr• psyc·hc.ll<:tq·lr..r.:d jo)\\!:' I... ~-1 l. .i j!'\ ,- i t i ~",i' • In t~~t'·ms CJf 1':\C:tj.vit.y/tn::~~,- apprc • ."ll •. lt u,,~ qr.:,ld c:d ·functionit''9 o·f any 01'"dina1··y mdn·~machine ~~-y~tem is solvirtg of certélin p!"'nblem Jn pt-~rt.iculr.lqp~ .;;.r-.d th~ dialoguo interaction betwPen man ~nd tcmputer· is ~imed Alt Uu;:• tulfilmvnt. Di' tlü~ tBs.k. n·.~· ma.Ht oo.:d of in5tructional eystmm is not to solve particular subJe=t prnblt:'ilt~ but to t:,~e""t:e.:> cif.z'Sl.f'c:"bl~ .. r:henqes Jr; ., ;';tuder.t tllOl~C:"lf. T t1i 5 tea tu r·f~ c~ .~~~\se::; ;·urtt•~:-·.. cJi•a, \.i rn. t iuro!::. .in mAn-m~chine ~ommunic~tion in profQssjonal and lr\Sti··~!Ctl.C.in~~l CCJtttp>:t!'i WtiH'II iH"E: t\:> full,~w~. 1. ln tl''lt:> situaticm of .in;;t.ri.IC:t.i.Cltl the m<"n-·mHchi• .. ··e int.Pr-t'\c:-t..lon is i'l OH.Hm~ 1r..1r ~.tl..tç"h?n1's lf:i.lltfl.:l.l•l} tl~l.\Ct:ttm..=d objr?c:t1v~~:: .:md l..~n,,ds. ~. In efficient ingtru~tlonal system5 the subject of Clialogut:.• .i~- not re~.tr-.ictet1 to tt'lt'.' v~ubject cnnt.t.'11l uf the lc,?.:Hili.ng t.:'\!i~ csolvincJ act.ivity. lrï this ~J.i.tuatinn the t.ommurdc<"\tiorl bE.~tvJe-:•l.',n t:::ch.ï'nge ot r~mad·::, but .it ar·isee due to f?!>:i~b:•nce of \.1-10 pc,~dtlons~ twc.• vl<. .. ws of ttle 5<:~Me prt)l:",.lt:•tu. F'romotJ.C.11'\ o·f intcrnal dialOQ\..\C? pt"OVlt:let'l L"'Jy fr};lt:.ntlr.:•.l utc:•n me:,ç:;nne di.::~logliE:' i:. int1i.C".~t~ve o1' efttc1e.•ncy of ditdugllt-:o commun.l.c:atton ln :\.r•~Lr_~.u Lit:.11.:.l :,.-!!llem~. All t hh• êlbove··mf't'lt.lorted "fe.•atW'"f1S e.·rtbl.\t;.' fr u~t~ tt~<; h Cli:her· and arE' c hl\J'" ac tt:>r l $ t:J.c 1 ol"' man ··mt:\t..: tlillt::' 1....\JIIIIfl\..llt J ,; t'l t. i on .i ro the c:c,ntf'~:t of .lr\$tn.tction. lnî?Y sl'lo1.1ld be t:c~l.t:~ll .i.r1lu i.I.C:COLtrtt while de?.siqnin9 i\nd eval\.lë•t.tng .1rtstn.~~~t.it.lll~l sofb.Jè:\l"'f.~. Üf.'~r.].gnerf, st·,c'ILtld cHrc•c:t the.t.r· c•11or tl:'J 1':\t c~nd l.I!H:> ?.ll the tet·.tmfJlog.ies .r.~vaile.bll? tor emlwru:i11q the.~ .i.nteorn.;,l CJi a 1 u~~~t~ ~J f' ll n:~: l'Jd.1.1dr.:n t • ,. l -----···---- 143! -------------------------~' 144 '·"' ~I .•. t:l,lllt•nr .it ll.4 c unrrtrt \lrl c: i'\ 1..1 V(' <>S.·~lé!C i. of .1 r•c.:. tr-uc t :i C.tflf:'. 1 S )' ~; t.t"fllS t. 0 1 .i Yf2 !; OIYlr!\LI'î 1 C: :::\1 .J.. t111 tJG• tHC• \?Tl ~;i: Ud L'n t at Hl t r,:•C:H" hF..:'r" ~ r.Hl'-·: t<·;n r,?J lC.it t.he ioll contn-:.11.1.if'HJ tl1;·· ~; .• t·u<">i"lnn .:.\nd thr~ .\t->.HI'ill\!1 prc"~c·:J: ... Hc.:-tt~tJnd~-> te ch;:,nql:".' tJi ve11 ê.'\ dJ ·f f~:'n'·'rJt on.:·. CJr1f:' u'f thr· po~.r~,j t.t:i(]lili'l'·='· u'f .lro'o".tr-t\C"tJon,:•,l r:;uff'•"l·'l.r"P <:;;hcu1rl bf~ ",;:•r-y c.:u-efu1 jr. c:r·tlt!r· Ln ,:ifl q C o::;n -f t:r t. ë: t.• 1 G i:\ tfi;Qs;.pt.c• I" f:.' f Q t' c;; t.~.t<.iC"''f1 t S • Ref"erences l"''.:üut::;;,J,t, f_.l--1. ~~H,;:\Pl·H·'t:.Cl~.~in, f.Lf-'..< f·.Q!11'tLf:i\j:t(ifh~,~ [.H.~.: ,ihl<'lïl!.);~ 0 {")~>)'''·.Cc-:\tOt\tf)l~~ l."~1Cî'f~r·lt·\ ~ ~"w,. F'. itl.: fk.>i>·.orn::sr·~~· .. rE'r.I~~·~E' ncHr:rn• yn;·<'HH!c;·~n ;pi v•r(·it.·~·tcH:l /\~51Tr: fHd·~w=·T LJ.O. t·~~;..H>•P. ~ f:~·llH1J.1:1. üJt .. \)fla 1"Jf.~?. Some benefits and problems of adaptive action-oriented user-interfaces Pertti Saariluoma Michael Miettinen University of Helsinki Department of Psychology Fabianinkatu 28 00100 Helsinki 10 A presentation to be given in NATO ASI: Basics of Man-Machine Communication for the Design of Educational Systems Eindhoven August 16.8.-24.8. 1993. 147 148 Abstract Human actions are chunked. Low-level action components are organized into hierarchical systems and, normally, people do oot think but rather use the highest level concepts. A user-interface which takes into account the hierarchical and chunked organization of human actions, we eaU action-oriented. In an action-oriented interface, all action components are bidden bebind general action labels and when working with an action-oriented interface, the actions are called the action labels. A way of realizing action-oriented user-interface is to let the interface monitor human behavior and adaptively modify itself. In this paper, we discuss three experiments we conducted to study some psychological problems in the use of supervised adaptive interface which follows the action-oriented principles. In the experiments adaptive interface was shown to be a viabie method of eneauraging the use of action-oriented interface; computer-cuing in order to promote the subjects' use of action-oriented facilities. 1. Introduetion People normally chunk their actions into wholes (Chase and Sirnon 1973, Milier 1956, Ericsson and Kintsch 1991). Por example, when taldog sarnething we do not think about moving shoulders and elbow joints, wrists and fingers but instead, we just take the object. Taking is an action and all the required subprocesses are subordinated under this one action label in OUT minds. Information processing, which is build on chunked actions, could he termed actionbased. Human thinking and communication is action-based. The main goal is important and the operative details are as far as possible bidden bebind the high-level action concepts. We believe that the chunked character of human actions should he respected in designing user-interfaces and we call a user-interface which is close to this ideal, action-oriented. Instead of forcing users into manual and mentally loading process control, an action-oriented user-interface requires nothing but information about the user's intended actions. The name of the action, however complex, should he enough to get a computer to carry out the action with minimal, ifany, human involvement Two conditions are required: Firstly, the concept of action oriented user-interface is polar. Interfaces are neither totally action-oriented nor standard, but more or less action-oriented. Secondly, an action-oriented interface is different from the notion of adaptive user-interface (Murray 1991, Wetzenstein, Ollenschläger and Wandke 1990). Action-oriented user interfaces need not he adaptive because they can he realized without any adaptation. Neither needs an adaptive user-interface to he action-oriented. However, adaptive interfaces provide interesting teehoical opportunities for the designing of action-oriented user-interfaces. The teehoical accomplishment of the chunking required in action-oriented interfaces is easy in principle easy, because it can he realized by writing macros. In this way it is possible to improve action-orienting in most standard interfaces. However, the range of users' needs, the varlation of the programs and the programming environments make the practical designing of action-oriented interfaces difficult. An ideal action-oriented interface presopposes knowledge about users' and user-groups' abilities, mental models, and intentions, and this information is, for the most part inaccessible in the planning and manufacture of human-computer interfaces. The design of action-oriented interfaces can also he enhanced by the users who can modify the interfaces to make them more suitable for their abilities and personal needs. This process can he fostered by providing suitable software to aid the modification. Machine learning and adaptive interfaces offer promising perspectives for interactive modification (Murray 1991, Wetzenstein-Ollenschläger and Wandke 1990). The interfaces themselves may learn to register users' needs and abilities by monitoring their behavior. They may either modify themselves or at least provide easy-to-use methods for supervising the modification. Since computers cannot possibly know the users' intentions, this supervised modification seems to he preferabie to unsupervised interface modification. However, oUT main interests are not teehoical but psychological. We are attempting to onderstand the psychological processes involved in woricing with action-oriented interfaces. A priori, we assume that one of the most difficult challenges for psychology in developing action-oriented interfaces will he caused by leaming. lndividuals' work-pattems develop in the course of time and training, and hence their expectations conceming the user-interfaces change. The users should also learn to use the facilities supplied by the interface. They should know how to use their existing knowledge to make the interface best answer their needs. 149 150 2. First experiment 2.1 Introduetion Our main experimental task was menu navigation. This task was selected because it is known that novices prefer menus but, with increasing expertise may find them less convenient (Card 1984, Paap and Roske-Hofstrand 1988, Vandierendonck, van Hoe, de Soete 1988). Consequently, the problems of learning and interface adaptiviness seemed to be more relevant in menu navigation than in, e.g., programming. Of course, this does not meao that we think menu navigation is the only task in which the idea of action-oriented interface is relevant We consider the concept of action-oriented interface as a general and versatile concept which can be discussed in practically any interface context. The user's knowledge is an essential factor of his behavior in a menu environment This knowledge may be divided into task-knowledge and environmental knowledge. The former refers to the content specific knowledge about goals and contentsof a current computational task and the Jatter, knowledge about the behavior of the interface. Some of users are accidental and do not have much information about the task or the interface while others are experts in both. To abserve the effects of these factors in our experiment, the familiarity of the task-environment was varied. A second factor which could affect users, is the task complexity. If tasks are simple, it would seem natoral to assume that current practice would be sufficient. The benefits of the action-oriented interfaces should become greater when the task demands increase. This is why the task complexity was varied. The main variabie was macro construction. Human action in a menu environment, which does not allow the use of macros, was compared with an environment which allows macro operations. We predicted that the use of rnacros would greatly improve subjects' performance and, consequently, the navigation task times would decrease in the macro environment compared to the non-macro environment 2.2Metbod 2.2.1 Subjects Thirty two subjects were used. They were computer-science students and cognitive science students at Helsinki University. All students had a good basic knowledge of computers and applications software. All were rnales except one. The age of the subjects ranged between 22-26 years. The subjects were divided into four equally-sized groups. 2.2.2 Materials The experiments were run with a standard 386SX PC with VGA monitor. A menu navigation program called ACONA I (Action Oriented Navigation Program) with macro facility was written with Pascal. A CO NA I is a basic menu environment. It presents a number of options for the user and the users' task in navigating is to select from various options by pressing the correspondingly numbered key. Navigation takes place by moving from one menu toanother until the target is reached. When the target has been found, the system backtracks to the beginning and presents the next task. The task number and the number of tasks in the experiment are presented to the subjects in the middle of the first row of screen. The target is shown in the middle of the seventh row. The current menu system node is presented underlined in the middle of the tenth row and the menu items indicating the next menus below it are located under it. In this and the second experiment, the number of the alternatives in each menu was four. The maximum depth of the search tree was five and thus the total number of nodes was 161. Macros are made by pressing the spacebar while in the initial menu, thus marking the ftrst element of the macro when the target menu is found, the last element of the macro is defined by pressing the spacebar a second time . After the definition of the last element, the interface ask "Do you want to make a macro from the first element (e.g. Copenhagen) to the last element (e.g. Helsinki) (Y/N)?". If the answer is negative, no macro is made. If the answer is positive, the interface asks the subject name to the macro. The maximum number of characters in a name was 8. Self naming was selected because the literature of command narnes supports the usie of mnemonic narnes and the literature of cue-validity has convincingly shown the superiority of self-naming (Barnard and Grudin 1988, Mäntylä 1986). From this point on, the macro can be called up by pressing the esc-key. Then the program prints a macro index which lists the narnes of all the macros, numbered in alphabetical order. To call up a macro, the subject presses the correspondingly numbered key. The call up of a macro moves the cursor to the macro endpoint menu. In this way, rnacros allow the chunking of the menu navigation processes. 2.2.3 Procedure and design In these experiments the subjects were told that they should act as salesmen or saleswomen. The subjects' task was to move from Copenhagen to another city or town as quickly as possible. Copenhagen was the root node in all subtasks and in all conditions. The menu system presented city narnes as options or menu items. Moving from one city or town to natorally required normal menu navigation. The depth of the search was varled by presenting target cities with either the depth of two or foor. The difficulty of the task was also varled by using in one situation, very familiar cities, e.g. Amsterdam, London, Berlin, etc., or by using, in another situation, less well known towns to oor Finnish subjects e.g. Erwitte, Hulia, Piney, etc. The major variabie was the macro option. Half of the subjects had the opportunity of defming and using rnacros but the other half had no macro option available. The subjects were tested individually. They made 32 navigation tasks in all. The only within-subject variabie was the depth. Familiarity with cities and the macro option were between-subject variables. The tasks were presented in eight groups of foor tasks of the same depth, i.e. 2 or 4. To control the learning process, the presentation was ordered so that aftera task group of depth 2 came a group of depth 4 and vice versa. The presentation was counterbalanced by presenting a half of the subjects with depth 2 first and half of the subjects with depth 4 first. This was done to avoid more than foor sequences of the same depth tasks. 2.3. Results The average processing times of the subjects in different conditions are presented in Figore 1. Seconds 200r------------------------------------------. 100 NF2 NU2 MF2 MU2 NF4 NU4 MF4 MU4 - Series 1 1111111 Series 2 1111111 Series 3 I:2TI Series 4 N=No macro. M=Macro. F=Famlllar ct ties. U=Untamlllar towns. 2=Depth 2, 4=Depth 4 Fig. 1. The average processing times. The main effect of task-time training, i.e., the subtask number F(3,224)=197.44, p<.001; menu tree depth, F(1,224)=4400.58, p<.001; and the familiarity of the contents, F(1,224)=8100.18, p<.001 were significant but the macro facility had no significant effect, F(1,224)=1.40, DS. 151 152 Also, the interactions of training and depth, F(3,224)=40.92, p<.001; training and familiarity, F(3,224)=54.24, p<.001; familiarity and depth, F(1,224)=2234.25, p<.001 as well as the three-way interaction between training, familiarity and depth, F(3,224)=80,74, p<.001 were all significant. The frequency of the constructed rnacros as a consequence of tasks solved is presenled in Table I. Table 1. Frequencies of constructed and used rnacros during the time of experiment by all subjects in the successive presentations of subtasks. FAMIUAR CITIES UNFAMIUAR TOWNS Leaming Macros Macros Leaming Macros Macros (Series) defmed used (Series) defmed used S1 1 0 S1 0 0 S2 9 3 S2 2 1 S3 15 6 S3 4 3 S4 17 8 S4 5 4 2.4. Discussion The familiarity hypothesis was confrrmed. Subjects were able to use their geographical knowledge in menu navigation. They made much more sensible search plans in familiar tasks than in unfamiliar. This means that domain knowledge is an essential variabie in menu navigation. The depth effect or task-complexity effect was also substantial, as one would expect. The deeper a path, the more difficult it is for the subjects to find the correct route. The interaction between task complexity and familiarity is understandable. The more unfamiliar the interface, the more blind and unorganized the search (Card 1984, Vandierendonck, van Hoe, de Soete 1988). This interaction bas an important consequence when the action oriented interface concept is considered. The structural decisions concerning the interface modification are not independent from the domaio knowledge structures. People seem to use domaio knowledge analogies in constructing mental models about the struct~ of the interface. To our surprise, our main hypothesis failed. Subjects did not navigate significally faster when they had the opportunity to make macros. They simply did not try to profit from this facility, which can be seen from the frequency of macros. All the subjects knew how rnacros could be constructed. They made a number of rnacros but seldom used them. Especially in unfamiliar conditions, both macro construction and macro use was modest Since macro construction required some extra time and planning, the subjects rejected it, as shown by the low frequency of use, and therefore did not benefit from the macro facility in their task solutions. This indica~es problems for our idea of user-modified and action-oriented interface, because it presopposes wide macro use. 3. Second experiment 3.1. Introduetion In the first experiment, the macro facility was of practically no benefit. People were simply not interested in using it. The reason may either be that the subjects were not willing to use macro procedures because they thought that the costs of macro construction surpassed the benefits of its use or it may be that they simply did not take into account the use of this possibility. To test these hypotheses for the lack of use of the macro option, we decided to use computer cuing. It is well-documented in the cognitive literature that cues after beneficia! effects in problem solving (Maier 1930, 1931). Also, Wetzenstein-Ollenschläger and Wandke (1990) have shown that subjects, in using an adaptive interface, may be encouraged to select more complex-action levels by computer cuing. If the reason for the low use of rnacros is overlooking this facility, computer cuing should be an effective cure for the problem. If the reason is conscious avoidanee of macro options, cuing cannot provide much help. 3.2Method 3.2.1 Subjects Thirty two male students with similar backgrounds and ages as in the fiJ'St experiment were used. None of them had participated in the frrst experiment The subjects were divided in four equallysized groups. 3.2.2 Materials ACONA I was modified for this experiment by adding it a facility which actively suggested macros. The new program was called ACONA II. ACONA 11 monitors the paths subjects frequently use. If a path is used four times, the program interrupts, reminds subjects that they have used this route several times and asks if the subjects would like to make a macro. After a "YIN" answer, A CON A 11 operates precisely like A CO NA I. In ACONA 11, no indication of the macro elements are needed and it defines the frrst and last element by the frequency of visits. If in the current path, some of the cities have a visit frequency higher than three, the program takes it as an element and backtracks until a city of the same visit frequency is found. Macro defines the route between those two ei ties. However, since the program never makes the same suggestions twice, the backtracking is continued until a menu element of the same frequency is found. If no suitable menu element can be found between root and the currently active element, no suggestion is made. 3.2.3 Procedure and design The procedure was the same as in the frrst experiment Design familiarity versus unfamiliarity was the between-subjects variabie and depth was again the within-subjects variable. 3.3. Results The main results are presented in Figure 2. NF2 NU2 MF2 MU2 NF4 NU4 MF4 MU4 - Series 1 R Series 2 R Series 3 EIJ Series 4 N=No macro, M=Macro. F=Famlllar cl ties, U=Unfamlllar towns, 2=Depth 2, 4=Depth 4 Fig. 2. The average processing times. The following main effects and interactions were significant: task-time trammg, F(3, 224)=175.87, p<.001; depth F(1, 224)=2791.29, p<.001; familiarity, F(l,224)=5302.26, p<.001; training x macro F(3, 224)=8.64, p<.001; training x depth F(3, 224)=26.17, p<.001; training x familiarity, F(3, 224)=44.74, p<.001; familiarity x depth, F(l, 224)=1528.54, 153 154 p<.OOl, as well as the three-way interaction between training, depth and familiarity, F(3, 224)=67.71, p) • » At 14:45 additional officers secure the scene of the crime and detailed investigations within the scene take place. Miss Rachel Black is interviewed and she reports last seeing her mother alive at 13:00 hrs earlier that same afternoon. Analysis of Mrs Black' s personal diary show that she was expecting a visit from her fmancial advisor, Mr Sterling, at 14:00HRS that same afternoon. » Examinations at the scene ofthe crime show that a valuable stamp collection and a large sum of cash, which Mrs Black was hoping to invest with her financial consultant, is missing. » House to House enquires in the local area reveal that an elderly next door neighbor of the victim, Mrs Withers, sawa very ta1l young man with a beard visitMrs Black' shouse between 13:30HR.S and 14:30HRS. » Other residents in the street report that a door to door salesperson, Mr Dodgey, called on the houses in East street between lO:OOHR.S and 14:00HRS. » A merober of the public whowas walking their dog reported seeing a maroon hatchback drive at high speed down the heavily shaded Ash Grove from South Street at 14: 15HR.S on the afternoon in question. » One ofMr Dodgey' s costomers of that day, at number 55 East Street reports buying a yard brush from him at 14:00HRS. 167 168 » The incident room is set up at AverageTown Pollee Station and the detectives set up interviews with Miss Rachel Black, Mr Bogus, Mr Dodgey and Mr Sterling. WE CAN NOW SUMMARISE THE DATA ABOUT EACH INDIVIDU AL INTERVIEWED. Miss Rachel Black: 21 Years old. 5 foot 5 inches taU. Student at AverageTown Art College. States that she left her Mum to go shopping on the aftenloon in question. Mr Bogus : 30 Y ears old. 5 foot 6 inches tall with long beard and hair. Refuses to answer any questions. Mr Dodgey : 35 Y ears old. 5 foot 10 inches tall. Sideburns and medium hair. Reported that he was doing bis rounds that day but that he never got as far as Mrs Blacks at number 50. States that he parleed bis Red Sierra XR4i outside number 70 East Street during bis sales round and after com-pletlog bis sales at 14:: 18HRS he drove away down East Street heading in an easterly direction. Mr Sterling : 45 Years old. 5 foot 8 inches taU. Large Moustache and sllght sideburns. Receding hair. States he visited Mrs Black as arranged but was 20 minutes late (14:20HRS) got no reply and proceed to bis next appointment in nearby North Street. 4.2.DEMONSTRATING THE PROBLEMS INHERENT WITBIN THE SIX STAGES OF CRIMINAL INTELLIGENCE ANAL YSIS. In the preceding simplified description of a crime we have attempted to incorporate features of the data which will allow us to show the problems facing the pollee service when they try to recreate and analyse the events which were associated with a crime. Before we show some of the ad vaneed computer supported tools and techniques which we hope to evaluate in oor project we will briefly summarise some of the problems faced by investigators using current methods and techniques. SIZE OF ACTUALAND POTENTlALDATA SETS MAY BE OVERWHELMING. In our mini-crime we dellbemtely decided to present a small number of events, places and individuals. However we hope that even this small number of data items will have allowed the reader to start to appreciate the difficulty of accumtely constructinga simulation ofthe events associated with a crime. Example from the Mini Crime. Our crime contains only ajew dozen data items, whüe real data sets may approach 20,000 items of injormation. RELEVANCE OF THE DATA IS UNKNOWN. Not only is the data set large but the investigators have no way of knowing in advance which items of data are relevant. If you appeal to the publlc for information about events associated with an approximate time or place you will inevitably get large amounts of data which bas llttle or no relevanee to the crime being investigatèd. Example from the Mini Crime. There is no way ojknowing ijthe sighting ojMr Bogus, a well known conjidence trickster, is at all relevant to the crime under investigation. DATA SET IS INCOMPLETE. No matter how well an investigation is conducted the data model of the crime will always be incomplete. This problem is made worse because it is only with hindsight that the investigators know for eertaio where the most important gaps in their data exist. Example from the Mini Crime. Large gapsexist in our knowledge of events at the scene ofthe crime, 50 East Street, AverageTown between 13:30 and 14:30 HRS on the day ofthe crime. DATA SET IS INACCURATE. If two individuals reeall a shared event it is unlikely that their accounts will match each others exactly - unless they have been rehearsed or prompted in some way. It is even more unlikely that their recollections will perfectly match those which actually took place. Quite unintentionally individual biases and interpretations will colour descriptions of events. These inaccuracies can produce conflicts aod errors within aoy criminat intelligence event simulation. Examples from the Mini Crime. It is uncertain how accurately the elderly neighbour, Mrs Withers, described the visitor who she saw visiting the victim.lfMrs Withers was extremely short, or had developed a slight stoop, she may have overestimated the visitors height. An additional problem is what Mrs Withers described as a beard may bedescribed by another person as 'sidebums and a moustache'. Another example from our mini crime is the colour and de scription of the car which was se en driving at high speed down the heavily shaded Ash Grove. Environmenta/light levels wil/ injluence the perception ofcolour. DATA SET IS AMBIGUOUS. As a result of the inevitable inaccuracies held within a typical data set there may be areas of the data based crime simulation which bas two or more possible mini-scenarios or interpretations. These ambiguities can only be ciarifled by gaiDing more defmitive data about those data items. In the worst case these ambiguities can lead to many quite different crime scenarios which, until more or better intelligence is available, must be regarded as being equally likely. Example from the Mini Crime. The combination ofwitness descriptions makes it difficult to know ifthe visitor described by the elderly Mrs Withers was the jinancial consultant Mr Sterling, the doortodoor salesperson Mr Dodgey, the conjidence trickster Mr Bogus, or another individual as yet unknown to the investigation. DATA SET MAY BE MISLEADING. A series of intentional or unintentional errors in the data provided to the form the data based crime simulation may make the resulting simulation misleading. This can lead to wasted investigations or in the worst case, a miscarriage ofjustice. Example from mini crime. Mr Dodgey's statement specijied that he had not completed his doortodoor round in East street asfar as the victim, Mrs Black's, at number 50. However one ofMr Dodgey's customers ofthat day lived at number 55 East street beyond Mrs Blacks, and Mr Dodgey's car was parked outside number 70 East Streel. This suggests that either Mr Dodgey or the customer at number 55 has made an inaccurate statement. INVESTIGATIONS SUBJECT UMITED RESOURCES. Another problem which faces the pollee serviceintheir criminal investigations is that oflimited resources. In some cases this cao meao that the officers in charge of ao investigation have to select one of many possible avenues of investigation. 169 170 Example from mini crime. We have not actually stated what resource constraints apply to our mini crime but it is likely that the investigating ofjicers would have limited resources available to assist them in their investigations. 4.3.EXAMPLES OF POSSmLE TOOLS AND TECHNIQUES TO BE EV ALUATED IN THE MYCROFT PROJECT. Having, very briefly, considered some of the problems which face the pollee service in their tmenviable tasks of criminal intelligence analysis we can now devote the remaiDder of this paper to providing selected examples of the varloos computer based tools and techniques which we intend to evaluate in the processof the Mycroft project. DATA GATBERING & DATA ACCUMULATION The Mycroft system will require that data items are categorised into various types. In ourmini crime these categones would include; a list of identified people, a list of unidentified people, a list of places, a list of items, a list of animals involved, and a list of vehicles involved. LIST OF IDENTIFIED PEOPLE. » MrsBlack » Miss Rachel Black » Fred Bogus » Mrs Withers » MrDodgey » MrWoof-Walker » Mr Buy-Anything LIST OF UNIDENTIFIED PEOPLE. » Visitor to Mrs Black. Male with facial hair who is taller than Mrs Withers. LIST OF PLACES. » Ash Grove, Average Town » BT Phone Box, South Street, Average Town » 48 East Street, Average Town » 50 East Street, Average Town » 55 East Street, Average Town » 70 East Street, Average Town LIST OF ITEMS. » Mrs Black's PersonalDiary » Valuable Stamp Coneetion » A Large Sum of cash LIST OF ANIMALS INVOL VED. » Large Dog. LIST OF VEHICLES INVOL VED. » Maroon Hatchback » Red Ford Sierra '-H_O_LM_e_s_N_u_m_b_e_r ____ __~ID Unldentllled HH4365835 _ I Attitude unknown I Prevlous Crlmlnal Actlvlty None I Known Assoclates Aódress lS~e~x ~~----'II/V;J4 se I Ethnlc Orlgln _ _ _ _Northern European ~=====-=====:::-:=: IHelght IIWelght ._s_f_o_o_t_a_~_· n_c_h_e_s _ _j 13 . 0 Stone I Bulld Stocky IHalr Balding I Accent Southern IHa~r Colour _ L~ght Brown Dlstlngulshlng Features Beard POST COTTAGE, AVERAGETOWN, AlO 6AS Figure 3 Person Data Entry Screen DATAENTRY. Each data category bas its own set of computer based data entry templates (see figures three & four) which in addition to aiding accurate and efficient data entry also build up a table of logica! flags which are used by the embedded expert system in its various knowledge based operations. Through a detailed six month study (2) we created and evaluated a prototype of these logical tables using flags such as temporal & spatlal proximity to the crime, likely accuracy of the entry (witness validity), witness characteristics, uniqueness of the item, number of Type of Building Telephone Box Locatlon of interest 0Flat 0 Whole Floor lAllas INameofSite SOUTH STREET Site Aódress I ~~er of Site ewholeSite 0 Area In Room 67, SOUTH STREET, AVERAGE TOWN. AlOO 0Unknown 2SL 0Room Prevlous Crlmlnal Actlvlty ll Reason For Interest Soft Drugs In area of Interest Figure 4 Data Entry for a Place times accessed or cross-referenced and the age of the data. HOLMES Number HH236288 I 171 172 Each flag holds a number which rates this data item on the associated se ale Figure 5 Example Logical Data Flags ge of THE TRANSFORMATION OF DATA TO INFORMATION By means of these logical flags, and the stored data, the intelligenceanalystscan begin toassess the importance and relevanee of the gathered data. lt is hoped that the Mycroft system will concentrate the attention of the intelligence analyst to those items of data which provide valuable information about the crime. THE TRANSFORMATION OF INFORMATION TO KNOWLEDGE. The Mycroft system will support varloos novel ways of simulating the known events that are associated with the crime. The easiestmethod of demonstraling the potential of these techniques is to show how the Mycroft system might assist in varloos example mini-crime problems we discussed in the preceding section (above). HIGHUGHTING ONLY THE RELEVANT. By specifying a spatial background within the area of the crime a digitised map of the area is presented. The analyst cao then specify the time period which he wishes to investigate and only data elements which occurred witbiD the specified spatial and temporal limits will be displayed. It is equally possible to view the same set of information with temporal and spatial axis exchanged, thus allowing unique perspectives on the data. HIGHLIGHTING GAPS IN THE KNOWLEDGE BASE. Using the different information views, which we have just descri bed, the Mycroft system will also highlight those areas in the knowledge base which contain too little infotmation. HIGHUGHTING INACCURACIES. Since each data item appears as an icon witbiD the current view inaccuracies are more easily visible. In addition the expert system will Figure 6. An Information 'Window' on the Raw Data search for and highlight any conflicts which may be within the data set. For example in our mini crime Mr Dodgey' s statement that he did oot reach as far as number 50 East Street would be highlighted by the system showing bis icon visiting number 55. HIGlll..IGHTING AND HELPING TO COPE WITH AMBIGUITY. The Mycroft system would highlight the possible problems surrounding Mrs Withers' description of the visitor to Mrs Black. Mrs Withers' description could match any of the male suspects. However Mycroft's knowledge base would recognise that Mrs Withers is short and would therefore tend to over estimate height. Another example is where Mr Woof-Walker described a maroon coloured hatchback travelling at speed down the shaded Ash Grove. Mycroft would highlight the possibility that because the car was so close (temporally and spatially) to Mr Dodgey's departure, the car in question could be Mr Dodgey's Sierra travellog in a Westerly direction. THE TRANSFORMATION OF KNOWLEDGE TO DEC/SIONS & REAL-WORW ACTIONS. HELPING TO MATCH RESOURCES TO THE INVESTIGATION. The Mycroft system will record the actions which are generated and tbe resources used in the course of the investigation. It is hoped that this feature will allow the investigating officers greater control and feedback in how their investigation is progressing. BACKGROUND PROCESSING. In addition to the various analysis supportfeatures webave alreadymentioned the knowledge base embedded witbiD tbe Mycroft system will be capable of independent background processing using expert systems derived from the experience of expert detectives, tbe use of bistorical crime and offender proftles. Over periods when tbe system is not being used to its full processing power it will be able to conduct its own analysis of the data sets witb the goal of generating a set of recommendations which will be ranked on what is judged to be their potential outcome. S.CONCLUSIONS. The Collaborators Expectancies. Law Enforcement. The successful condusion of tbe project will leave the pollee with a tooi that will have major benefit in the investigation of serious crimes. Forther tbe force will have developed considerable experience in creating "expert" systems. It is intended that these areas be capitalized toforther expand the role ofMycroft into otber areasof pollee work. For the techniques it employs could equally be used in formulating organizational pollcy and otber areas where tbe relationships between entities need to be simulated or modeled, and where intelligent knowledge based systems could be employed. Academie. The academie side of tbe Mycroft project will forma con tribution to the advancementofknowledge in the following specialistareasof computer science : objectoriented design & programming, knowledge ellcitation in complex and vague domains, simulation of human decision making on 'complex' and 'fuzzy' data sets, design and creation of an IKBS for crimina! intelligence worlc, human computer interface design & evaluation, technology attitudes, the impact of the introduetion of new technology and methods : botb in the pollee force and in society. Commercial. McDonnell Douglas see tbe Mycroft project as having tbe potential for developing into an ongoing collaborative research programme between the existing project partners. It is hoped that tbis collaboration would continue to develop through subsequent generations of tbe Mycroft system and its IKBS rule base. Government The scientists witbin SSG are interested in tbe development and enhancement of existing and new intelligence analysis metbods and tools. The continued development ofMycroft beyond the stages outlined in this document would ensure that tbe very latest tools, methods and techniques were available to the UK' s criminal intelligence taskforce. S.l.CONCLUSION. We have invested a considerable amount of effort and resources into what are new and challenging areas of human-computer-interaction and artificial intelligence. So far tbe Mycroft project bas established a new and unique metbod of intelligent multi-media information and knowledge representation and presentation. This promises to provide a truly adaptive metbod for knowledge representation and manipulation. We invite other parties to join us in this exciting area of inquiry. 173 174 6.REFERENCES. 1- Hardy, P. (1991) "lnformation Vs Knowledge: An Alternative View of Major Incident Rooms" Intemal Paper : Sussex Police. 2- Holland, L. (1991). "An Investigation ofKnowledge Elicitation Techniques in CrimiDal Intelligence Settings" Unpublished MSc Dissertation, University of Portsmouth. 3 - Oldfield, R.W. (1988) "The Application Of CrimiDal Intelligence Analysis Techniques To Major Crime Investigation - An Evaluation Study". Scientific Research And Development Branch. Home Office Publication 30/88. 4 -Woods, A.J ., Jenkinson, R.E. (1991) "Intelligence and Operational Systems HOLMES Front End Demonstrator Evaluation Report " Scientific Support Group Report. Home Office Pollee National Computer Organisation Publication 1/91. 5- Morgan, K., Holland, L., Hardy, P., Casey, P., Quinn, T., Mead, R., Roberts, K., Cadwell, I., Oldfield, R. (1993) "A Description of the Possible Role of.Computer Enhanced CrimiDal Intelligence Data Analysis" Paper Presenled to the British Psychological Society's Division of Criminalogical and Legal Psychology Annual Conference, Harrogate, 1st March 1993. 6- Morgan, K., Hardy, P., Casey, J., Holland, L., Quinn, T., Mead, R., Roberts, K., Cadwell, 1., & Oldfield, R. (1992) "Implementation and Design Issues in Cooperative Knowledge Based Systems for CrimiDal Intelligence Analysis : The Mycroft Perspective". Paper presenled to the Cooperating Knowledge Based Systems Special Interest Group (CKBS) Meeting at Queen Mary & Westfield College, University ofLondon. December 9th 1992. Learning in Neural Networks PaulMunro Department of Information Science, University of Pittsburgh, Pittsburgh PA 15260 USA Abstract. Until recently, the overwhelmingly dominant mode of machine compu-tation has been absolutely deterrninistic. Neural networles provide an alternative ap-proach to computation that is inspired by neurobiological principles. These sys-tems can learn categones from examples, and generalize their learning, such that novel examples can be classified. Correct classificarion depends on a variety of fac-tors, including the goodness of the sample used for training and the consistency of the test item with the staristics of the training set Neural network training proce-dures are more closely related to statistica! regression techniques than they are to mainstream AI. Analyses of the networles both during and after training show re-markable similariries to human learning, and may give insight to the principles un-derlying human information processing. 1 Background The biologica! principles that enable higher level cognition are just beginning to be understood. While the neural components that make up our brain are so much slower than the electronic devices found in computers, they are much more numer-ous, and thus process inforrnarion in parallel; this massive parallelism gives rise to forrns of computation that are very different than those perforrned on electtonic computers, which process instrucrions one at a time in sequence. Parallel distribut-ed processing has been found to account for several cognitive phenomena (see Rumelhart and McClelland; 1986) from perception (e.g., optical illusions) to higher levels (e.g., language and problem sol ving). 1.1 Neural Network Fundamentals The theory behind parallel distributed processing borrows its essenrial concepts from neurobiology, but a host of aspects considered vital by neurobiologists are ig-nored (to the chagrin of many). Of course, no singular description describes all neu-ral network models, and so the following description should not be interpreted as a strict definition. A neural network is defined as a structure consisring of nodes joined by "one-way" connections (i.e., a directed graph). Each node receives signals along connections that terrninate on it, and transmits a signal on connections that it originates. The transmiued signal is a function of the received signals and some in-temal parameters. A small network is shown on the leftof Figure 1.1, and a gener-al node is shown on the right, where the output is a funcrion of the inputs, s, t, u, etc., and some internal parameters, A, B, C, etc. 179 180 r = f(s, t, u, ... , A, B, C, ... ) s u Fig. 1.1. A small neural network (left) and a schematic of one node (right). The intemal parameters of the nodes are typically weight factors assigned to the node's input signals. So in Figure 1.1, the unit might have a response like r=As+Bt+Cu. More typically, this weighted sum would be computed and then passed into a threshold-like function, like r=O if (As+Bt+Cu) q > not-q > not-p. To model this data they calculated the expected information gain of each card with a = b = 0.1. That is, they assumed that the properties described in p and q are rare in the population. A similar assumption was made by Klayman and Ha ( 1987) in accounting for related data on Wason's (1960) 2-4-6 task. On this assumption the order in expected information gain is: E(l,(p)) > E(l, (q)) > E{l,(not- q)) > E(l,(not- p)) That is, if subjects base their card selections on the expected information gain of the cards then the observed order in the frequency of card selections is exactly as Oaksford and Chater's (1993) model of optima! data selection predicts. Oaksford and Chater (1993) also show how their model generalises to all the main pattems of results in the selection task, including the manipulation described above of introducing negations into the task rules. In sum, if the selection task is regarded as a problem of probabilistic optima! data selection then the putative biases observed in this task disappear. 5 Condusions and Some Possible lmplications for Man-Machine Communication 5.1 Summary I began this chapter by claiming that much of the bias literature needs to be re-interpreted in the light of the growing number of normative probabilistic re-analyses of the data upon which claims of bias have been based. These analyses, 227 228 as the flrst example of Bimbaum (1983) reveals, were often contemporaneous with the heyday of the "heuristic and biases" tradition. However, these analyses were largely set aside in the rush to explain these biases in process terms. Of course these accounts simply ignored the fact that they were now assigning apparently irrational functions to the processes they invoked. Recently, however, there bas been a re-emergence of concern with rationality and rational analysis (see, e.g., Anderson, 1991; and the collection edited by Manktelow & Over, 1993) and with it I suspect that we are witnessing a degenerative problem shift (Lakatos, 1970) for workin the area of beuristics and biases. Lopes (1991) bas observed that the rhetorica! emphasis of the beuristics and biases tradition was on Thagard' s 1 above, i.e., on human dumbness. She points out that this de-emphasised the importance of rationality in cognitive explanation. In witnessing a problem shift back towards rational function we should be wary that the psychological insights of the beuristics and biases account are not similarly abandoned. As Gigerenzee and Murray (1987, see also, Gigerenzer, Hoftrage & Kleinbölting, 1991) observe there bas always been an essential subjective, psychological aspect to probability theory. As we have seen assumptions about which priors to employ ortheir effects on the decision criterion, for example, are not decideable within probability theory. In arriving at psychological theories of reasoning in any domaio there will be a need for essentially psychological accounts of how these values are arrived at and how people tractably implement their probabilistic competence. Nevertheless, following Marr (1982), flrst identifying the rational functions that the human cognitive system needs to compute is again being seen as a feasible and necessary frrst step in specifying an adequate cognitive theory. 5.2 Man-Machine Communication While I am not competent to judge whether these developments will have immediate ramiflcations within man-machine communication, I cao outline an area where these considerations may be relevant the apparent success of grapbics interfaces like the highly usabie Macintosh interface. I make the assumption that all interfaces are deterministic systems. That is, the actual rules that determine what happens given user input are deterministic, i.e., as long as the system is functioning normally, a given input, key press sequence, mouse click in a menu locatio~ etc., yields a single speciflc action with probability 1. Now at some metaphysical level of description the world may be a deterministic system. Certainly classica! mechanics which for the bulk of our everyday macroscopie needs is all we need to predict the physical world, regards the world deterministically. However, this does oot meao that in acquiring knowledge of the macroscopie world human beings model it deterministically. Irremediable ignorance of the disposition of a complex system whose next state we wish to predict often precludes deterministic preti;~•ion. Instead scientists must settie for probabilistic theories in order to make adequate predictions. Similarly, when confronted with a complex user interface novice users may model the syst.em not deterministically but pröbabilistically. This provides a possible explanation for the success of interfaces like the Macintosh. Deterministic command line interfaces are all or nothing affairs, either you know the appropriate command or you don' t. The organisation of access is also very flat in that if you don' t know the specific command there are no higher level exploratory actions you can perform in order to find out. This kind of organisation is only suited to rote learning of the appropriate command language and rules. This contrasts with the Macintosh interface, where there are often many ways of performing the same action. Moreover, there are a set of high level actions, like pulling down menus, clicking on labelled buttons, scrolling etc. that enable the user to interact with the computer in an exploratory manner. This is analogous to our normal inductive interactloos with the world. In acquiring practical skilts the ability to rnanipolate and explore bits of our world permits us to learn much faster. It seems possible, therefore, that these interfaces succeed because they exploit the same abilities that allow us to team so effectively about the world in order to inductively explore the interface. In sum, novices may model their interactloos probabilistically in accordance with their natoral competences; the very competences that are now being revealed in the probabilisitic retreat from bias. References Anderson, J. R. (1991). Is human cognition adaptive? Behaviaral and Brain Sciences, 14, 471-517. Beyth-Marom, R. (1982). Perception of correlation reexamined. Memory&: Cognition, 10, 511-519. Birnbaum, M. H. (1983). Base rates in Bayesian inference: Signal detection analysis of the cab problem. American Jou11Uil of Psychology, 96, 85-94. Cheng, P.W., & Novick, L. R. (1990). Aprobabilistic contrastmodel of causal induction. Jou11Uil of Personality and Social Psychology, 58, 545-567. Evans, J. St B. T. (1989). Bias in hwnan reasoning: Causes and consequences. Hillsdale, NJ: Erlbaum. Evans, J. St B. T., & Lynch, J. S. (1973). Matching bias in the selection task. British Jou11Uil of Psychology, 64, 391-397. Evans, J. St B. T., Newstead, S. E., & Byrne, R. M. J. (1993). Human Reasoning. Hillsdale, N.J.: Lawrence Erlbaum Associates. Gigerenzer, G., Hell, W., & Blank, H. (1988). Presentation and content: The use of base rates as a continuous variable. Joumal of Experimental Psychology: Human Perception and Performance, 14, 513-525. Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilisitic mental models: A Bronswieldan theory of confidence. Psychological Review, 98,506-528. Gigerenzer, G., & Murray, D. J. (1987). Cognition as intuitive statistics. Hillsdale, N.J.: Lawrence Eerlbaum Associates. Jaspars, J. M.F., Hewstone, M.R. C., & Fincham, F. D. (1983). Attribution theory and research: The state of the art In J. M.F. Jaspars, F. D. Fincham, & M. R. C. 229 230 Hewstone (Eds.), AnTibution theory: Essays and experiments, (pp. 3-36). Orlando, FL: Academie Press. Jobnson-Laird, P. N., & Wason, P.C. (1970). A theoretica! analysis of insight into a reasoning task. Cognitive Psychology, 1, 134-148. Kahneman, D., & Tversky, A. (1972). On prediction and judgement. OR/ Research Monograph, 12. Kahneman, D., & Tversky, A. (1982). The simulation heuristic. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), iudgement under uncertainty: Reuristics and biases, Cambridge: Cambridge University Press. Kelley, H.H. (1967). Attribution theory in social psychology. In Levine D. (Ed.), Nebras/ca Symposium on Motivation Vol. 15, (pp. 192-238). Lincoln: University of Nebraska Press. Klayman, J., & Ha, Y. (1987). Confrrmation, disconfrrmation and information in hypothesis testing. Psychological Review, 94, 211-228. Lakatos, I. (1970). Falsification and the methodology of scientific research programmes. In I. Lakatos, & A. Musgrave (eds.), Criticism and the growth ofknowledge, (pp. 91-196). Cambridge: Cambridge University Press. Manktelow, K. I., & Over D. E. (Eds.) (1993). Rationality, London: Routledge. Marr, D. (1982). Vision. San Fransisco, CA: W. H. Freeman & Co. McArthur, L. Z. (1972). The how and what of why: Some determinants and consequecnces of caual attribution. Joumal of Personality and Social Psychology, 22, 171-193. Norman, D. A., & Draper, S. W. (1986). User-centred System Design. Hillsdale, N.J.: Lawrence Erlbaum Associates. Oaksford M, & Chater N. (1993). A rational analysis of the selection taskas optimal data selection. Teehoical Report No. CCN-UWB-93-05: Centre for Cognitive Neuroscience, University of Wales, Bangor. (In submission). Oaksford, M., & Stenning, K. (1992). Reasoning with conditionals containing negated constituents. Joumal of Experimental Psychology: Learning, Memory & Cognition, 18, 835-854. Popper, K. R. (1959). The logic ofscientijic discovery. London: Hutchinson. Thagard, P. (1988). Computational philosophy ofscience. Cambridge, MA: MIT Press. Tversky, A., & Kahneman, D. (1980). Causal schemas injudgements under uncertainty. In Fishbein M (Ed.), Progressin social psychology, Hillsdale, N.J.: Lawrence Erlbaum Associates. Wason, P.C. (1960). On the failure to elminate hypotheses in a conceptual task. Quarterly Joumal of Experimental Psychology, 12, 129-140. Wason, P.C. (1966). Reasoning. In B. Foss (ed.), New horizonsin psychology, Harmondsworth, Middlesex: Penguin. Wason, P.C. (1968). Reasoning about a rule. Quarterly Joumal of Experimental Psychology, 20, 273-281. An X Window Based G UI for the Dynamic Simulation of the Chemica! Recovery Cycle of a Paper Pulp Mill F. Pais1 , B. Gay2 and A. Portugal1 1 Department of Chemica! Engineering, University of Coimbra, Largo Marquês de Pombal, 3000-Coimbra, Portugal 2 Department of Computer Science and Applied Mathematics, Aston University, Birmingham B4 7ET, U.K. Abstract. This con tribution describes an interactive software system (XProc-Sim) for easy and efficient analysis of the behaviour in transient state of the chemica! recovery cycle of a pulp mill. It is formed by two independent but coordinated components: a graphical user interface (GUl), written in C, which was developed using the X Window system together with Sun's XView toolkit, and simulation modules written in FORTRAN77, which may run on the same or on a different machine. Process representation, control of the integration performed by the simulation programs, data and perturbation in-puts, and graphical representation of results are performed on a colour UNIX Sun SPARCstation2 + grapbics accelerator using the mousefmenu driven in-terface. The keyboard is used to input data only (numeric values or character strings). Significant ditrerences between previous systems and the software described bere are XProcSim's use of the X Window system for the GUl development, which ensures portability to a number of hardware platforms. Remote, networked X-based display systems .may be used, and there is also the possibility of running the simulation program on a remote machine. As the complexity of mathematica} roodels increases, this is a critica} ditrerence for an on-line system as the execution time- and therefore the response time- · may become prohibitive without the use of powerful machines, which do not necessarily possess grapbics display capabilities. The distributed structure makes it possible to take the best possible advantage of existing machines. Keywords. Graphical User Interface (GUl), Distributed Systems, Process Simulation, Chemica} Recovery Cycle, Pulp Mill 1 Introduetion Process simulation is one of the most widely used tools for the design, opti-mization or simply the study, in steady or transient states, of in dustrial units and plants. lf the analysis of process dynamics under conditions of start-up, grade changes, emergency outages and shutdowns is desired, then dynamic simulators are necessary (Schewk and others,1991). Our primary objective was to develop a package for the computer simulation of the transient state 234 of a paper pulp mil!. Rather than descrihing mathematically the full set of industrial equipment involved such as pumps, valves, etc., we wanted to develop reliable dynamic models for the main units in the cycle and study their interaction. This is not a minor task as we are interested in the detailed behaviour of every unit which led, in some cases, to complex mathematica) models of large dimeosion which present salution difficulties and whose sim-ulation is time consuming (e.g., the lime kilo model alone is composed of a set of hyperbolic partial differential equations which, after spatial discretiza-tion and using the metbod of lines, is converted to hundreds of stiff ordinary differential equations). On the other hand, the migration of simulation tools from "simulation experts" into the hands of practical process engineers will be accelerated by the availability of user-friendly interfaces (Schewk and oth-ers,1991). In order to manage the amount of data involved and results pro-duced, as well as to easily define or alter all the simulation parameters and impose the desired perturbations, it soon became apparent that a powerful user interface was needed. A number of GUI's have recently been developed for systems that range from genetics applications (Drury and others,1992) to economie analysis (Jensen and King,1992) and processing speetral data (King and Horlick,1992). The development of a GUl, which would enable us to display graphically the results of the simulation, was therefore a very important feature for the usability of our simulation programs. 2 On graphical user interfaces "Software that functions but is difficult or confusing to use rarely actually gets to do anything"(Rimmer,1992). Basedon software ergonomie investigations, the condusion that graphical user interfaces are the interactive man/machine communication of the future is now widely accepted (Encarnaçäo and oth-ers,1991). Recent studies have shown that users in a GUl environment work faster, more accurately and with lower frustration and fatigue levels than users in a text-based environment. Considerable increases in both productiv-ity and accuracy of completed work are achieved using GUI's (Peddie,1992). There is, however, a lack of standardization and the number of existing sys-tems is high. Examples are X Window for UNIX workstations, MS Windows and Presentation Manager for IBM Personal Computers, the Maclntosh soft-ware for Apple computers, GEM for the Atari and Intuition for the Amiga (Nicholls,1990; Pangalos,1992). In the UNIX-based world the de facto stan-dard, developed by MIT (Scheifler and Gettys,1986), is usually known as X Window. It is nota GUl, but rather a common windowing system across net-works connecting machines from different vendors, as well as a foundation on which to build graphical user interfaces (Peddie,1992). X can display output windows from several applications at once in one screen, and these programs can run simultaneously on different machines. The machines do not have to be the same type or run the same operating system (Moore,1990). Since the X Window system is in the public domain and not particular to any hard-ware platform or operating system, and its distribution is notsubject to the commercial interests of a single supplier, it may well become the standard windowing environment of the 1990's for all sorts of computers. Any ma-chine that supports multitasking and interprocess communication and bas a C compiler can support X (Moore,1990). One of the goals of X was to impose no style of interface on its users. It relies, in practical terms, on the development of graphical user interface tooikits - sets of widgets, or build-ing blocks, such as menus, buttons, scrollbars, etc. - each with a consistent appearance and behaviour, to assist the application developer do bis work. Initially developed as user interfaces were the Open Look GUl, by AT&T and Sun, and the Motif GUl by Open Software Foundation (OSF) and a number of softand hardware manufacturers (DEC, Hewlett-Packard, Microsoft, etc.). These user interfaces were later complemented by toolkits, e.g. XView devel-oped by Sun, and tend to become the standards for the UNIX world. Both of them possess a high degree of applications portability. In our work, we used a colour UNIX Sun SPARCstation2 + grapbics accelerator workstation. Although it need not be so, in this case it functioned as bath the elient and the server of the X Window architecture (for details, see Reiss and Radin, 1992). We also used Sun's XView toolkit, tagether with the Open Look in-terface and window manager as developed by Sun and AT&T. A tooikit is an indispensable tooi for sa ving time, since it avoids lengthy sets of low level intructions to create a simple object and guarantees consistency across all objects - that is, the same look and fee!. Although functional consistency still bas to be ensured by the interface designer, appearance consistency is ensured by the correct use of such a toolkit. The word object bas been used bere for the first time. To simplify developing applications, X tooikits employ concepts used in object-oriented programming. This is quite a feat as the tooikit is written in and for the C language, which does not directly support objects (Miller,1990). 3 Basic Features of XProcSim XProcSim's interface was designed from scratch so a number of ways of achieving the desired objectives was usually available. Although "design is a creative process and hence there are few absolutes that must be adhered to" (Rubin,1988) consideration was given at every stage to the human factors guidelines which are, at present, playing an important role in the new science of user interface design. Two important characteristics of the interface we are developing are the fact it is menu-based and, primarily, a colour applica-tion. The advantages of menu-based user interfaces over cammand language ones are well known (Kantorowitz and Sudarsky,1989): no need to learn the cammand language which allows for productivity in a very short time, pos-sibility of exploring the operations provided simply by browsing through the 235 236 menus and, if an adequate help system is installed, no need for a manual in most cases. The mouse is used for all the selection operations, whereas the keyboard is used to enter numeric values or character strings only. On the other hand, colour is an excellent means of improving the compre-hensibility of information displays such as pie and bar charts (Rubin,1988). However, use of colour must he judicious to avoid visual interaction that func-tions as communication-damaging noise and can produce strong after-effects. According to cartographic principles, background dull colours (muted, gray-ish or neutra!) are more effective, allowing the smaller, bright areas to stand out with greater vividness (Thfte,1992). As to the palette of colours that should appear on the screen, a way to avoid "colour junk" is to use colours found in nature, chosen from widely spaeed points in the colour spectrum (Thfte,l992; Rubin,1988). Rubin also states that the use of colour in a man-machine interface should he as consistent as possible with everyday usage. These basic principles were foliowed by XProcSim, where the default back-ground colour is consistent through all the windows (light grey). Wherever possible, colour is used to convey information about the physical system itself. Units where green liquor is processed are coded green, units where lime is the main reactant are coded yellow, and so forth. In the lime kiln diagram, for example, colour goes from yellow to red as temperature increases. However, a commercial package should never he totally dependent on the use of colour since potential users would he significantly restricted and compatibility of the interface with monochrome displays is now under development. Another basic feature of XProcSim is the fact that all its windows are scalabie which allows the user to resize them at will. No limitation has been imposed on the maximum number of windows which can he displayed simultaneously. Windows can at any time he reduced to their iconic state - icons have been attributed to all of them - so that avoidanee of a duttered screen is left to the user. The structure of the interface is presented in Fig. I. Basically, the objectives we aimed at were: * Graphical representation of the process and individual units * Organiza-tion and easy access to simulation data* Graphical representation of results * Possibility of executing the simulation program on remote machines and full control of its execution status * Easy access to integration data* On-line help system *Error warning and recovery* Possibility of executing a selected set of UNIX commands through menu options provided by the interface. 3.1 Graphical representation of the process A very simplified flowsheet of the process, in the form of a block diagram, is displayed on request and gives acess to all the data for all the units. This diagram is formed by unit and stream type elements. Selecting one of these will display a pop-up menu containing all the information items available for that unitor stream. These include schematical representations of all the units l Unit Menu: UnH Diagram Physlcal and Chemica! Data Unit Dlmenslonl Operallng Conditlont lnlemal Profile I (Une and bar chart plo\a) MainWindow lnltlallze Syltem Ullltue1 -Help Change Numerical Parametera Integrale unlll... Stop Inlegralion QuH Reaurne Inlegralion Oult Inlegralion Rutart Daemon for lniUallzallon In Remote Machine Reatart Oaemon for Slmulallon In Remote Machine Check Slalul of Background Proceuea Llat Flle1 Klll Proceuea Crealed In Remote Machine Creale Terminal Emulalor 1 Stream Menu: Flow rale Compoaltlon State Varlablel Fig. 1: Main options available through the interface. 237 238 Model Observe Mathematica! Numerical Evaluate Model ~ Solutlon Graph f---+ Plot t I I I Fig. 2: Information Flow Model {Adapted from Nielsen,l991). (which we hope will be of interest to the novice user), possibility of displaying instant internal profiles, design characteristics of the units, physico-chemical proper ties, etc .. 3.2 Organization and easy access to simulation data No commercial physical properties database was used. However, all the val-ues used in the simulation of a unit, initially stored in files, are available and can be changed through the interface using data panels. Among these data we include the already referenced dimensions of the unit and physico-chemical parameters such as heat transfer coefficients, activation energies and Arrhenius constants for the chemical reactions, etc .. 3.3 Graphical representation of results . Graphical representation of simulation results corresponds to four stages (see Fig.2). Two interactive modules, one for line charts and one for bar charts, have been developed to perform the final task of creating an actual image on the screen. The Iine chart is used to represent either profiles along space for a given time or variation of a variabie along time. Because it would be impossible to prepare charts with all possible combinations of variables, only a Iimited number have been selected and are automatically prepared for graphical display. However, both the Iine and bar chart modules can load data files so the more experienced user can generate his own data files conesponding to whatever variables he wishes to represent graphically. Both modules present a range of options to the user such as colour of all the elements, text font size, Iine width, colour and style (for the line chart), bar colour and fill pattern (for the bar chart), automatic or manual selection of axes range, etc .. Local Machine sun Sparc2gx Model 4f75 1 - Interface modules 2 - Programming language: C 3 - c compiler XWindow and XVIew llbrar1es RPC Remote Machine Sequent Symmetry 1 - C/Fortran77 Interface modules Fortran77 module~ 2 - Programming languages: c and Fortran77 3 - C and Fortran77 compilers Fig. 3: C/FORTRAN77 structure. 3.4 Possibility of running the simulation program in a remote machine As the complexity of mathematica} models increases, so does usually execu-tion time. If on-line interaction is desired, then for practical reasans very powerful machines may become necessary. The benefits of parallel process-ing for complex systems have become obvious in the last few years (Shan-dle,1990). The simulation program is at present being executed in a Sequent Symmetry UNIX machine, so execution times can reach critical values. Com-munication of the interface with the remote simulation program is done in two ways: 3.4.1 Successful RPC calls Wh enever possible, direct transfer of parameters is made. The RPC (Remote Procedure Call) package is used to link the local C interface routines to a remote C routine which in turn calls the remote FORTRAN77 routine (see Fig.3). It may he noted that one objective of this work is to re-use existing FORTRAN77 simulation code rather than translate it into C. For this purpose a C/FORTRAN77 interface was developed. Current parameters are transferred as arguments to the remote C routine 239 240 using the UDP transfer mechanism. This made it necessary to develop XDR (EXternal Data Representation) routines which transfarm data particular toa specific machine into a machine-independent format. A limitation of 8 Kbytes exists which may in the future make it necessary to use other transfer mechanisms such as TCP which bas no limit on the amount of data sent. 3.4.2 Signal handling routines lnitialization (reading all the default parameters) is done via an RPC call which will in principle have a successful return; sending the simulation pa-rameters to the simulation program is done in the same way. However, ex-ecution time of the simulation program can he quite long and is usually unpredictable. In order to avoid blocking the interface until the remote rou-tine returns, a short timeout for the RPC call is specified using the lower layer of the RPC package. An unsuccessful return occurs and is ignored, while execution continues in the remote machine. To compensate for the loss of the initia} elient/server link, flags are written to a log file whenever a change occurs in the execution status of the simulation program. This change can he the availability of new results, in which case writing the flag is straightforward, or events such as abnormal termination (e.g., due to arith-metic exceptions) of the simulation programs. This led to the need for the instaBation of a series of system signa} handling routines, which catch signals generated by the system and write the appropriate flag to the log file. The interface possesses a routine that periodically checks the log file and takes apropriate actions according to the flag value, such as displaying a dialog box to alert the user to the crash of the simulation programs. This methodology makes it possible to know what is happening to the simulation programs run-ning in background while the interface remains free for the analysis of results produced so far. System signals can he generated either by the system- and intercepted and handled by the signa} handling routines- or by the user (e.g., to stop, pause or resume the integration) and sent to the remote program. A diagram summarizing the GUI/remote simulation program communication types is presented in Fig.4. A possible installation problem resides in the fact that the C/FORI'RAN77 interface (the C routines that "wrap" the FORTRAN77 routines) is compiler-dependent. If the simulation programs are run on another machine, it may he necessary to alter the souree code of these routines. 3.5 Easy access to integration data Data panels relative to the integration process only have been provided. The items consist of all the parameters needed by the LSODI integrator (Hind-marsh,l980). In case another integrator is used, the corresponding data panel must he changed as well. Local Machine XWindow + XVIew Remote machine Menu Opllon: lnltlallzallon _ _____,r lnltlallzatlon Server (C) lnlllallzallon RoUlinea (Fortra.n77) Succeuful RPC call Read default valuea from files Menu Opllon: Inlegration +--------------Slmuiatien Server (C) Slmuiatien RoUlinea (Fortran77) Unsucceuful RPC return (link la loat) Wrlte slmuiatien reaulla to files Ruults are read and prepared for dlaplay Wr11e naga to log file (new dala avallable, normal or abnormal termlnatlon, etc.) Interface routine checks log file perlodlcally: atatua of executlon la known Fig. 4: GUI/simula.tion programs communica.tion types. 241 242 3.6 lmplementation of an effective on-line help system Any kind of help can bring a dramatic impravement in user performance. Allowing users to control wben to initiate a help request is better in terms of speed of task and error frequency (Rubin,1988). Context-sensitive help (i.e., help information that is related to the partienlar point the user bas reached) bas been selected. Altbough it is quite simple to create a help button in every window, on pressing wbich a text read-only window is displayed, the optima) structure of such a help system still bas to be seriously tbougbt tbrough. For example, where should help pertaining to transitory pull-down menus be placed? Should it be a layered system- that is, more than one level of help? The final structure of the help system is still under study. 3. 7 Error warning and recovery In tbe case that errors are made by tbe user on the interface side, error detection is simple and performed by checking tbe value returned by the function. If an error value is returned, a dialog box is displayed. In the case that errors occur on the simulation side, the strategy bas been described above. Dialog boxes are displayed in any case. 3.8 Possibility of executing a selected set of UNIX com-mands through menu options Including some UNIX commands as menu options was thought to be a conve-nient feature. The aim was to provide a very simple interface to the operating system in order to perform operations such as listing files or checking the pro-cesses currently being executed. All the already referenced commands to be sent to remote programs such as halting execution or resuming it are UNIX commands as well, in the form of system calls. Last, if the user wants direct access to the operating system, a terminal emulator window can also he cre-ated by the interface. This can be useful if a more experienced user wants to have full and unrestricted access to the operating system. The keyboard will then of course be used to input commands as in any cammand tooi. 4 Conclusions XView. tagether witb the X Window libraries have been shown to provide a wide range of grapbics capabilities and a useful set of general purpose widgets. XProcSim is a GUl for the grapbical representation and management of data pertaining to a specific set of simulation programs. lts future development is to create a front end for process flowsheeting and/or dynamic simulation programs in gener al. Although out of the scope of tbe present work, selecting icons by tbe cursor and moving tbem to the desired location in a "drag and drop" operation would he an easy way to build the process diagram. The general guideline, as far as GUI's are concerned, seems to be that there are no strict guidelines. lt is however generally accepted that, to be fully ef-fective, a GUl must he consistent, easy to learn by the novice user, provide shortcuts for the experienced user, obey ergonomie and human factors (such as adequate width and depth of menus) and even aesthetic considerations, etc .... At this point of our work, it is obvious that user feedback is deeply needed in order to refine the prototype. This kind of practical information is probably, apart from theoretical considerations, the ultimate assessment of interface utility. Acknowlegments. We are grateful to the Commission of the European Communities, under the Science Program grant contract No.B-SC1/900596, for the finantial support that made this work possible. References Drury, H.A., Clark, K. W., Hermes, R. E., Feser, J. M., Thomas Jr., L. J., Donis-Keller, H. (1992) A graphical user interface for quantitative imag-ing and analysis of electrophoretic gels and autoradiograms, BioTechni-ques, 12, 892 Encarnaçäo, J ., Cote-Muiioz, J ., Eckardt, D. (1991) User interfaces to sup-port the design process, Computers in Industry, 17,317 Hindmarsh, A. C. (1980) Lsode and Lsodi: two new initial value ordinary differential equation solvers, ACM Signum Newletters, 15, 10 Jensen, D. L., King, A. J. (1992) Frontier: a graphical interface for portfolio optimization in a piecewise linear-quadratic risk framework, IBM Systems Journal, 31, 62 Kantorowitz, E., Sudarsky, 0. (1989) The adaptable user interface, Commu-nications of the ACM, 32, 1352 King, G. B., Horlick, G. (1992) An advanced user interface for processing and displaying interferometric and speetral data: Spectroplot, Spectro-chimica Acta, 47B, E353 Miller, J. D. (1990) An Open Look at UNIX, M & T Publising, Inc., and Prentice-Hall International (UK) Limited Moore, D. (1990) The migration of the X Window system, BYTE IBM Spe-cial Edition, Fall, 183 Nicholls, B. (1990) Looking at the graphical user interface, BYTE IBM Spe-cial Edition, Fall, 161 Nielson, G. M. (1991) Visualization in scientific and engineering computa-tion, Computer, September, 58 Pangalos, G. J. (1992) Standardization ofthe user interface, Computer Stan-dards & Interfaces, 14, 223 Peddie, J. (1992) Graphical user interfaces and grapbics standards, McGraw-Hill Inc. 243 244 Reiss, L., Radin, J. {1992) X Window inside & out, Osborne McGraw-Hill Rimmer, S. {1992) Graphical user interface programming, WindcrestfMcGraw-Hill Rubin, T. (1988) User interface design for computer systems, Ellis Horwood Limited Scheifler, R. W., Gettys, J. (1986) The X Window System, ACM Trans. Graphics, 5, 79 Schewk, C. F., Leaver, E. W., Schindler, H. E. (1991) The roles for process system simulation in the design, testing and operation of processes, proc-ess control and advanced control systems, J. Pulp and Paper Canada, 92, 53 Shandle, J. (1990) Reai-time simulation speeds HDTV designs, Electronics, March, 68 Thfte, E. (1992) The user interface: the point of competition, Bulletin of the American Society for Information Science, JunefJuly, 15 A METHOD OF EVALUATING THE USABILITY OF A PROTOTYPE USER INTERFACE FOR CBT COURSEWARE Dr. Garry Patterson. Dept. of information Systems, The University of Ulster, Jordanstown, Co. Antrim. BT37 OQB. Narthem Ireland. Abstract This paper describes the evaluation process for assessing the usability of a prototype user interface for Computer-Based training (CBT) courseware. The MlD AS (Multi-media lnteractiveDesign Aided System) model presents a methodology forthedesign, implementation and evaluation .of multimedia CBT courseware. Results of the usability of the prototype application are presented which are based upon the evaluation criteria within the model. The evaluation results are analysed totest the performance of the prototype and their implications discussed. The implementation of the model prescntcd is centeredon the following application- the training of pilots and engineers on the braking system of the British Aerospace Jetstream Aircraft. Keywords Human-Computer Interface, Computer Based Training, Courseware, Multimedia, Evaluating Usability, Authoring. Introduetion The research project was conducted under the general perspecti ve of the definition of methods and tools for the evaluation of an interface based on human factors criteria. The goal was to design an interface for use by CBT courseware users in an cffective leaming environment. This study prescnts one such method; the validation of human factors criteria proposed within the MlD AS framework. There are numerous different human factors methods currenüy available for the evaluation of interfaces. They each have advantages and drawbacks, the major problem however is that none of them allows a complete evaluation of an interface. Scveral classifications of such methods have been described ([Christie'90], [Karat'88], [Maguire'89] and [Senach'90]). Three main categoties of methods cao be distin-guished depending on their orientation; theoretica! models, expert judgement or the users. The latter approach is the one that will be stressed and used in this paper. 247 248 1 Evaluation Methods based on Users In this category, evaluation methods are based on various user related data. The data may consist of users' performance data collected during user interaction with existing orprototyped interfaces (behavioural data) or from cognitive indications (understand-ing and memory). The datamayalso consist of subjective information extracted from users' attitudes and opinions about the interface. Extracting and monitoring user data is performed with several tools that are more or less intrusive. Such techniques include on site observation, audio-video recording, physiological recording or concurrent verbalisations. These can influence users' behaviour. Conversely, questionnaires, subsequent verbalizations, individual or group interviews havelessof an influence on users' performance. The use of anEvaluationBooklet,presented the userwith a practical methad in the farm of a checklist The metbod adopted is discussed in Sections 2 & 3. 2 The Evaluation within the methodology of MlD AS 2.1 Overview of the Method The metbod is a practical tool, inthefarm of a checklist. It is basedon a set of software ergonornies criteria, which a well-dcsigncd user interface should aim to meet The checklist consistsof sets of specific questions aimed at assessing usability. These provide a standardized and systematic means of enabling those evaluating a CBT system interface. 2.2. The Evaluation Checklist Each of the frrst ten sections is based on a criterion, which a well-designed user interface should aim to meet. The criterion is described at the beginning ofthe sec ti on, and a numberof checklistquestions follow, which aim to idcntify whether the interface meets the criterion. The criteria are not in any order of importance within the checklist The ten criteria are summarised in Table 2.1. Wh en answering the checklist question, the evaluator ofthe CBT courseware ticks one of four alternative op ti ons. Of these, 'always' is the most favourable reply, and 'never' is the least favourable. The other two options are 'most of the time' and 'some of the time'. Research ([Hoinville'80], [0ppenheim'79]) has shown, that if a 'neutra!' category was included as a fifth option, many subjects simply tick this option, whereas with the structure presented the subject has to think more carefully of the relevanee of the answer to each specific question. There is space beside each question for the evaluator to make and comments of relevanee to the question and to their answer. At the end of each section, space is provided for the evaluator to add any other comments, good or bad, of relevanee to the issues raised within the sec ti on. This is foliowed by a five-point rating scale, ranging from 'very satisfactory' to 'very unsatisfactory', which enables the evaluator to give a general assessment of the interface in terms of the criterion and questions within the sec ti on. Table 2.1 Evaluation Criteria Section Heading Criteria Statement (Summary) Visual Clarity Information displayed on any CBT courseware screen should be clear, well-organized, unaambiguous and easy to rcad Auditory Presentation The music or voice commentary for any CBT screen should be clear, structured, unambiguous and easy to follow Consistency The way the CBT courseware looks and works should be consistent at all times Compatibility The way the CBT courseware looks and works should compatible with user conventions and expectations Infonnative Feedback Users of the CBT courseware should be given clear, infonnative feedback on where they are in the training course, what actions they have taken and whether they have been successful Explicitness The way the CBT courseware works and is structured should be clear to the user Appropriate Functionality The CBT courseware should meet the needs and requircments of users when carrying out tasks Flexibility and Control The CBT courseware interface should be flexible in structure, in the way infonnation is presenled and in tenns of what the user can do, and to allow them to fcel in control of the system Error Prevention & The CBT courseware should be designed to Correction minimize the possibility of user errors; users should be able to check their inputs and to correct errors User Guidance & Support Informative, easy-to-use andrelevant guidance and support should be provided in the CBT courseware 249 250 Two further criteria are included, which relate to the general usability of the CBT courseware. Section 11 is specifically concemed with usability problems that the evaluator encountered when carrying out the tasks. For each question, the evaluator indicates whether there were major, minor or no problems. Section 12 consists of general questions on system usability. The questions are 'open', allowing the evaluator to express opinions of various factors, i nel u ding the bestand worst aspects of the CB T courseware interface system. 2.3 Sourees of the Checklist Questions The checklists have been developed with reference toa number of sources. A number of questions from the 'Usability: Screening Questionnaire' in ([Clegg'88]), and the format for these sections have been adapted from this source. In addition, early versions of Clegg provided the stimulusfora number of questions within Sections 1 and 10. Othersourcesforthechecklistinclude([Ravden '89]. [Smith '86]. [Gardner'87] and [Shneiderman '87]). 2.4 How should the Checklist be Used? A key feature of the method, and a prerequisite for using the checklist, is that the evaluator uses the system tasks which the CBT courseware has been designed to perform,as partoftheevaluation. This takes place beforecompleting thechecklist, and represents the first step in the evaluation. Tasks used in the evaluation should be representative of the work which is to be carricd out using the systcm, and should test as much of the system, and as many of its functions, as possible. Such tasks require careful construction and are extremely important to the validity of the user interface evaluation. Th ere are a num her of reasons for use of representative tasks: [i] Tasks which are realistic and representative of the work for which the system has been designed provide the most effective way of demon-strating the system 's functionality. [ii] This approach enables those evaluating the interface to see it not simply as a series of sereens and actions, but as part of the application system as a whole. [iii] By carrying out tasks, evaluator's can he exposed to as many aspects of the user interface as possible, which is necessary if they are to comment usefully, and in detail, on specific features, problems, strengths and deficiencies of the CBT courseware interface. [iv] Many significant problems and difficulties of the coursewareare only revealed when carrying out tasks. [ v] In some cases, there may be importantaspectsof usability which can only be captured by using the system (e.g. inflexibility of a menu structure when the task requires rapid movement between different parts of the system.) In addition to the above points, important information can be gathered by observing an evaluator's performance when carrying out tasksin an evaluation. Observations and recording of task performance is extremely valuable in identifying those difficulties which evaluators experience when interacting with the system. As a supplement to the completed checklists, this information can enable, for example, investigation of critical confusions, common mistakes within the CBT interface. Examinatien of differences between evaluator responses to checklist questions, by referring to the task data; examinatien of task data for supporting evidence of an evaluator's comments about the interface; and comparison of evaluator groups according to their task performance arealso issues which can be drawn from the learning environment. 2.5 Who should be involved in the Evaluation Process? The metbod enables a variety of different people, with different background and expertise, to evaluate the same uscr interface. A particular benefit of this is that it enables end-users, such as the CBT students and trainers and designers who will use the system to access its usability befare it is implementcd into a full courseware package. The degree of end-user involvement which is possible will depend, toa large ex tent, on the context of the evaluation. Factors such as the general availability of end-users, the scale and budget for the evaluation and the degree of novelty and complexity of the application may all influence the scope of user involvement In addition to representa-tive end-users and the CBT courseware designers, other evaluators such as software engineers not involved in the design and developmentof the interface being evaluated, who can provide an objective viewpoint from a more technica! perspective; or cognitive psychologists and those in hu man factor groups, who can provide know led ge and understanding of the potential psychological implications of different designs. 2.6 When can the Metbod be Used? The methad can be used to evaluate usability both during and after design and development of the user interface. Wh ere prototyping tools are available, it can enable early evaluation of an interface, or if alternative interfaces, so providing feedback at early stages in the design process. In this way, different configurations can be tested, and necessary improvements made, beforereaching a fully operational state ready for implementation. At the later stages of dcvelopment, it is more difficult to make major modifications to the interface. The metbod can be of particular value in the later stages of development, where the 251 252 system is sufficienûy developed to enable realistic tasks to be carried out Similarly it can be used to evaluate a system befare full implementation. This may be particularly valuable to an industry who intend to purebase from courseware developers that requires amendment, modification, or impravement befare implcmentation. The methad can also be used in evaluating an application package with a view to purchase. Here tasks can help to assess whether the system meets the required functionality, and the evaluation can highlight good and bad features, aspects which are unsatisfactory, and those which are excellent. The metbod can also be used to evaluate alternatives for purchase, as it can allow a comparison between them. In certain situations the checklist may not be sufficient to cover all the important usability issues which should be assessed in a particular evaluation. For example, where an interface relies upon speech input, some checklist questions will not be entirely appropriate and others may be added. The same may be truc for systems which do not employ mainstream, relatively conventional user interfaces. It may be possible to augment or 'tailor' the checklist in these cases. Great care must be taken if augmenting the checklist, in order to preserve its balance. 2.7 Potential Financial Benefits of Using the Metbod CBT user interface evaluation using this metbod may incur some immediate financial costs; for example, where the development process is lengthened, or through the involvement of end-users and others in the evaluation. However, the importance of user interface design and of usability should not be underestimated, and its evaluation should be expccted to take the time and effort worthy of its importance. In the longer term, the benefits which are likely to resultfrom the evaluation will outweigh the casts for CBT courseware developers and companies which use their skills. Benefits may include: - reduced training time for end-users; - reduced support casts, due to fewer, and less significant difficulties; - reduced need for amendments, modifications and revisions after implementation~ . - where relevant, incrcased sales, as a more usable, wcll-dcsigncd and acceptable training environment is provided; - a greater willingness among end-users to accept thc training system and to use it effectively; - a greater awareness among those developing CBT training courseware application systems of the requirements for 'user-centred' design. In Section 3 the evaluation methad is used to assess the usability of the prototype application user interface. In the light of experimentation, recommendations for changes to the prototype are made. Section 4 presents the results from the Evaluation Procedure. 3 The Evaluation 3.1 Goal of the Evaluation The goal is to assess the usability of a user interface for CBT courseware. The method, and in particular the checklist, enables an interface to be evaluated by a variety of people with differing expertise and backgrounds; including, for example, interface designers, other technica! experts and, most importantly, representative end-users, whomayor will actually use the system in practice. The methad does not aim to sol ve problems, or to enable a quantitative assessment of usability. It provides a means of identifying problem areas, and the extracting of information concemingproblems, difficulties, weaknesses and areas for improvement. 3.2 Metbod 3.2.1 Subjects One hondred and twenty subjects participated in the study. Th ere we re 15 pilots whose experience in flying ranged from two to twenty-four years (M = 7.5; S.D. = 6.5), and the remaining participants were the student and staff population registered with The University of Ulster. Male and female subjects participated. The ages of the subjects ranged from 18 to40 years (M = 23.7, S.D. = 4.7). 3.2.2 Equipment The equipment used in the evaluation includcd a list ofEvaluation Criteria that were designed to assess the usability of the user interface for the CBT courseware. Section 2.1 presented an overview of the method, and Table 2.1 summarised the criteria. The CBTcoursewaredeliveryplaûorm wasanAppleMacintoshllfx with 12"highresolution coloor monitor. Interaction was via a keyboard and mouse. 3.2.3 Procedure Each subject participated in one individual evaluation session. The evaluation consisled of a leaming phase and identification tasks in intcractive mode. The subjects then completcd the Evaluation Booklet. In thc lcaming phase subjects were invited to study the prototype application relating to The Braking System of the British Aerospace Jetstream 32 Aircraft. They were encouraged to work through the introduction, statement of objectives, operation of the brake system and component location modules. However, the subjects could at any stage return to the main menu screen, and execute the courseware in any order. Th is would also assess the systems' usability. The identification tasks foliowed the completion of the leaming phase. During these tasks, subjects were presented with problems relating to answering questions on the location 253 254 and opera ti on of the braking system. No time limit was imposed on either phase. Times rangedfrom 25 -40 mins. (M = 32.5, S .D. = 6.5) forthe pilots; 29- 72 mins. (M = 50.5, S.D. 13.6) for the engineering students; and 40- 93 mins. (M = 65.0, S.D. = 11.9) for the rest of the subjects. Following the CBT lcsson the subjccts wcre requircd to fill in the Evaluation Booklet. This process took approximately 60 mins. to complete. The subjects were encouraged to fill this in immediately after the hands-on scssion, and to bc as critica! of the user interface (if nccessary.) Spaces for writtcn comments were provided bcside each checklist qucstion. 3.2.4 Data Collection and Analysis Two types of data were collected. Firstly, the grading ofthc checklist questions, which was in the form of aD in the appropriate box which best describes the subjeet's answer toeach question, or 'N/A' if the qucstion was not appropriate to the courseware. Sccondly, the subject may commcnt on any issue rclating to cach spccific qucstion in the appropriate column. The objectives of the data analysis were thrccfold:-Firstly, to assess the usability of the prototype application. To achieve this, global scores were calculatcd for each criteria question from the sample responses. A correct response was defined as either a 0. 'N/ A', or 'don't know' in one of the appropriate columns on the checklist grid. If a checklist question was left blank, or two responses given, this was defined as a missing value. In the data anal ysis using SPS S, these val u es were eliminatcd, thus reducing the sample size. Sccondly, to (a)identifyproblemareasand weakncsscs within thcprototypeapplication user interface; and (b) make rccommcndations to enhance a multimedia UID for CBT courseware. Thirdly, using a factor analysis statistica! technique, to identify key variables and subsequently design an Evaluation Booklct to assist CBT courseware designers in assessing usability of a UID fortheir end-users at the prototype stage of development. An explanation of the analytica] technique - Principal Components Analysis is contained in Section 3.2.5 3.2.5 Analytica! Technique- Principal ComponentsAnalysis Factor techniques aim to produce new dimcnsions bascd on combinations of correlated varia bles, which clear theanalysisof confusingmulticollinearity ([SPSS '90], [Nie'75], [Johnston'78]). Given a set of n relatcd variables, factor techniques transform n variabie co-ordinale space into new m variabie co-ordinale space where the new m variables (which are called 'factors' or 'components', depending on the particular form ofthe analysis used) are linear functions ofthe original n varia bles, which are now zero-correlated among themselves, that is, orthogonal. These new linear functions can be powerlul predietors of previously hidden relationships among the original vari-ables. Common Factor Analysis (CFA) and Principal Component Analysis (PCA) differ essentially in the assumptions they make about the data, and therefore in the interpretations they permit CF A is concemed only with the analysis of that portion of the total varianee that correlates with other variables - the 'common reliable variance'([SPSS'90]). The common reliable variance, however, has to be estimatcd from thatdata, which means that CFA is partofinferential statistics, and must therefore make rigid data assumptions ([Maxwell'79], [Clark'73]). In contrast, PCA is simply a mathematica/ procedureforvariance manipulation, and as suchmakes no assumptions about the distribution of the data and no attempt to assess statistica/ significance. Moreover, in PCA the tot a/ varianee is analysed, and the components may be thought of as simply a parsimonious tiescription of the data. The data obtained from the PCA technique in this study will be used to identify key variables for the design of the reviscd evaluation boakietand is obtaincd from exccuting the statistica! package SPSS 4.0 on the Apple Macintosh computer environment. To assist in arriving at an acceptable number of factors for the representation of the data, it is helpful to examine the percentage of total varianee explaincd by each. The total varianee is the sum of the varianee of each variable. For simplicity, all variables and factors are expresscd in standardized farm, with a mean of 0 and a standard deviation of l. In section one of the evaluation booklet, Visual Ciarit y, there are 14 variables, vel, vc2, ... ,vc14 and cach is standardized to have a varianee of 1, the total varianee is 14 in this example. The total varianee explained byeach factor is listcd in the column labelled Eigenvalue. The next column contains the percentage of the total varianee attributable toeach factor. For example, the lincar combination formed by Factor 2 has a varianee of 1.43, which is 10.2% of the total varianee of 14. The last column, the cumulative percentage, indicates the percentage of varianee attributable to that factorand those that preeede it in the table. No te, that although variabie narnes and factors are displayed on the same line, there is no correspondence between the lines in the two halvesof the table. The first two columns provide information about tlie· individual variables, while the last four columns describe the factors. The table shows that 55% of the total varianee is attributable to the first four factors. Theremaining nine factors tagether account for only 45% of the variance. Thus, a model with four factors may be adequate to repcesent the data. Several procedures have been proposed for determining the numbcr of factors to use in a model. One criterion suggests that only factors that account for varianee greatcr than one {thc eigenvalue is grcater than onc) should bc includcd. Factors with a varianee less than one are no better than a single variable, since each variabic has a varj~ce of 1 ([Tucker'69]). 255 256 4 Results from the Evaluation Procedure 4.1 The User Interface ofthe Prototype Application Table 4.1 provides a summary of the overall ratings for Sections 1 to 10 of the Evaluation Booldet. These global scores provide a good indication of the general performance of the user interface for the prototype application. The results lead us to conclude that the design of the interface for theBraking S ystem module provided a very satisfactory environment for leaming. In five of the ten sections the range of values were between (l.~x< 2.0) and the other five between (2.~<3.0). The overall mean fortheten sections being Table 4.1 Summary of Overall Ratings for the Evaluation Criteria Criteria Readings N MV x SD. Section 1 - Visual Clarity 120 0 1.55 0.71 ~ection 2 - Auditory Presentation 120 0 1.77 0.88 ~ection 3 - Consistency 118 2 1.61 0.71 ~ection4 - Compatibilty 118 2 1.81 0.76 ~ection 5 - Informative Feedback 120 0 2.15 0.95 ~ection 6 - Explicitness 118 2 2.01 0.81 ~ection 7 - Appropriate Functionalilty 119 1 1.95 0.82 ~ection 8 - Flexibility and Control 116 4 2.41 0.93 ~ection 9 - Error Prevention & Correction 112 8 2.25 0.94 Section 10 - User Guidance & Support 85 35 2.67 1.33 Rating Scale 1 2 3 4 5 X-+ I I I I I Very Moderately Neutral Moderately Very Satisfactory Satisfactory Unsatisfactory Unsatisfactory The lowest rating scores were identified in Section 10. In this category, U ser Guidance and Support. only 85 subjects completed this section. The reasans for this can be identified by examining the questions presented in the Evaluation Booklet Section 10. The only user support provided in the courseware was audio and visual feedback if incorrectanswers were given by the subjects to the questions. There wasnoHelp menu integrated into the braking system to provide on-line assistance for the aircraft terminology or hardcopy outputto supplement the CBT materiaL The subjects did not have a copy of the training manual to complement the CBT instruction. The data from this section supports these claims. The hooklet used in the evaluation process is a gencric document for UID evaluation and thus by including options which were not included in the prototype design, can be remedied at the implcmcntation stage of development The use of a multimedia plaûorm proved a very successful user environment for lcaming. 63% of the subjects commented on the effective use of multimedia to present the training material, with a further 10% supporting the specific use of grapbics and animation. The Jack of an on-line Help facility is a lso noted as one of the worst features of the systcm. 33% included comments, spccifically mentioning the Jack of feedback from the system. A further 24% commcnted on a lack of explanation of the direction icons. In the written comments, the subjects however noted that once they became familiar with their functionality, they did notpose a problcm. It is very eneauraging to note that in general terros some 63.5% of all subjects had no difficulty in either onderstanding the system or in finding any irritating system problems. In recommendations suggested by the subjccts for improvement, only 23.3% included the ioclusion of a Help facility. Over 55% concluded that the CBT courseware represented a good prototype application. A further 10% suggested that the provision of rnaving-video illustrations would have enhanced the lcaming environment. Noevidcnceexisted to suggest thatthedifferentsubjectgroups found the userinterface environment more complex to use or the training material any more difficult to understand. One reason for this is that no assumed knowledge was required for an onderstanding of the braking systcm. The pilots commented on the effective use of multimedia in designinga realistic training environment. In the past, this had only been available when the pilots progressed from the classroom instruction to the Flight Training Devices, and even then this was a static environment. Only when the pilots progressed to training in the Full Flight Simulator was interactivity a part of the lcaming experience. 4.2 The Revised Evaluation Booklet The Revised Evaluation Booklet has been designed to assist designers and end-users in assessing the usability of a UID at the prototype stage of development The questions included in each sec ti on are identified from the analysis of performing PCA on the subjects data. The coefficients (factor loadings) in the Factor 1 column of the FACTOR MATRIX. Factors with large coefficients (0.55, where Sis a set of autornaton states, where the initia! state s0 and the final state sn are 261 262 distinguished; D is a set of inputs; E is a set of outputs and h : S x D .. S x E is a function that calculates autornaton output e and next state s according to the current state s and autornaton input d (user's action). Let us consider autornaton state classes. Autornaton can: - answer a user's question and wait for the next question; - ask a question and wait fora user's action; - answer a question, intercept initiative, ask user a question, and wait for user's action. We shall designate thern accordingly s~ s2 and s3 • The final state Sn rernains unrnapped. We can note that S = S 1 U S 2 U S 3 U sn. We shall introduce sorne sets: Q- set of user's questions, A- set of user's answers, Q ' - set of autornaton questions and A ' - set of autornaton answers. There are null answer a0 in A '. For each sk we shall define special sets: Qk - set of discernible user's questions, and A k - set of discernible user's answers. Let there are exist input d0 which cannot be recognized as a discernible user's question or answer at any step: n n d0 ~ U Qk, d0 ~ U Ak, and d0 eD. The autornaton set of inputs A:=1 k=1 consists of possible user questions and answers (D -= Q U A ) , and its set of outputs contains it's own answers and questions, that it asks intercepting initiative in dialogue ( E ... Q • U A • ) . Let us exarnine output function /. Assurne that autornaton is in the sk state. lf user's action d is recognized as question (d e Q~, then autornaton can praeeed by answering the question, or intercepting initiative and asking the user a question of it's own. lf a user's action is recognized as an answer (d e A~, then autornaton will ask the user a new question or answer the most recent unanswered question. Let us exarnine transition function g. Assurne that automaten is in the Sk state. If SJc e S 1 and the user's actiondis not recognized as a discemible question or answer, then the g ( SJc , d) state belongs to S 2 • If SJc e S 1 and the user's action is recognized as a question, then the g (sic, d) state belengs to any class of states (excluding final state sn ). If sk e s 2 U s3 and the user's action is not recognized or recognized as a discemible answe~. then the g ( sk , d) state belengs to S 2 • lf sk e s2 U s3 and the user's action d is recognized as discernible question, then the g ( sk , d) state belongs to any kind of stat es (including the final state sn ) • Let us consider the autornaton cycle.lf current autornaton state sk e S 2 , autornaton will ask a question. If sk e S 1 U S 3, autornaton will begin step with visualization of the answer which was obtained on the previous step. If sir. e s3 , after the answer is visualized autornaton will intercept initiative and ask question q e Q · connected with the current state. Following any of the given situations, autornaton will wait for the user's action. In any state autornaton must recognize sorne of the user's actions as questions. The accepted user's action is either recognized as a question frorn Q kor answer frorn A Ir., or it rernains unrecognized. According to accepted action autornaton produces an output e =I (skI d) • 1f the action rernained unrecognizedl then I ( sk , d) = I ( sk, d 0) • Than autornaton transits to the new state g ( sk 1 d) . Autornaton starts frorn the state s0 • Autornaton can start dialogue as an active partner - by asking the user a question or 1 as passive partner, waiting for the user1S question. In the first case s0 e S 2 • In the secend case we consider that s0 e S 1 and that autornaton will start this step with visualization of the null answer llo· When autornaton transits to the sn state, it stops. Autornaton models are interesting because it is simple to implement such models on a variety of conternporary computers. 3 Netmodel The state-transition diagram of the above mentioned autornaton can be mapped to a network G = < S 1 r > with the node set S and the relationship r that sets up a correspondence between node s; and the set of nodes r ( s;) which are at the ends of the edges starting frorn this node. All edges starting from each node are nurnbered. Let us change transition function g slightly in such way that g : S x D .. N. Transition frorn the current node will occur to the node ending the edge with nurnber g ( sir. , d) • We shall not represent the final node sn- The dialogue systern will finish its work when g ( sk 1 d) > card r ( sir. ) I in other wordsl when g ( sk , d ) is greater than number of the edges starting from the current node. We shall name thesetof network and step function the dialogue scenario. 4 Net model implementation Net model is useful as a formal basis for the universa} dialogue interpreter which can conduct dialogue according to a stared dialogue scenario. Function h can be implemented by the internat facilities of the dialogue interpreter or by use of external processes. Answer to the user's question and/ or transition code can be calculated by a separate executable module invoked by systern. 263 264 Database or knowledge base can be connected with such a dialogue system. It can contain some facts, data or rules on how to analyze the user's questions or how to synthesize answers. Then, some steps of dialogue can be connected with database (knowledge base) modifications. In order to implement dialogue scenario generator based on above mentioned net model, weneed to choose interpretation of sets S , Q and A. We may consider any key press, mouse click or mouse movement as a user's action. However, such interpretation would lead to an unnecessarily detailed dialogue scenario consisting of an unmanageably large number of nodes. To reduce complexity of the dialogue scenario networks we can use their recursiveness. We can replace some subgraphs withnodes and, in that way, increase the "granularity" of nodes and possible user questions and answers. We can substitute nodes for subgraphs satisfying the following condition: all edges entering subgraph enter the same node. We shall name such subgraphs substituted by nodes scenario units. Usual user interface objects, such as entry fields, menus, dialogue boxes with pushbuttons, radio buttons and check boxes, help, may be considered as standard scenario units. Conception of the scenario units allows us to develop dialogue scenarios by means of incremental refinement. We can provide the reusability of scenario units: the same units can be used instead of the different nodes or even in the different dialogue scenarios. The dialogue partner model was implemented in the Dialogue Processor project. The Dialogue Processor consists of a Dialogue Generator to generate dialogue scenarios, and a Dialogue Interpreter to interpret these scenarios. The Dialogue Generator stores a generated scenario as a metafile - a stream of objects. The scenario determines permissible transitions (net edges). Each dialogue step (net node) can include passive audio and video objects, active video objects and description of external process that the step invokes. Passivè objects are used to express a question or an answer for the user's question. Active video objects, are used to give the user an opportunity to answer a question or to ask bis own question. Passive video objects are panels with static text or browsers for texts, tables or bit maps. Animated passive video objects and audio objects add an expressive force to generated dialogues. Active video objects are standard dialogue panels - menus, choice lists, or dialogue boxes with entry fields and choice fields of different kinds. 5 Summary The suggested answering dialogue partner model is methodological base of the Dialogue Processor - the tooi to generate dialogue systems. The Dialogue Processor uses object-oriented technology and stores dialogue scenarios as streams of objects. Further development of object-oriented databases will allow starage in one database dialogue objects and application objects, and production of more complex dialogue systems. The suggested model can be extended to describe dialogue partner training. lf we assume that step function, sets of net nodes and edges, or sets of discerning questions and answers depends on time, we shall obtain model of a non-stationary dialogue system. lf we add system actions that modify the above mentioned dialogue system elements, we shall get a model of dialogue partner that can be taught. 265 Intelligent Assistance for Simulator-Based Training Eva L. Ragneroalm Department of Computer Science, University of Linköping, S-581 83 Linköping, Sweden E-mail: elu@ida.liu.se This poster presents an ongoing project aiming at the design of a prototype system intended to provide intelligent assistance for simulator-based training of process-control operators in the domain of paper and pulp manufacture. The purpose of the training in our case is to increase the operator's onderstanding of the process and to operationalize bis knowledge in a diagnostic context It is intended to supplement classroom insttuction and on-the-job training. One difference between skilled process operators and novice operators is the abil-ity to diagnose errors efficiently. Skilled operators follow a diagnostic procedure that appears to be transferrable (Schafstaal1991). Teaching this diagnostic procedure to novice operators in order to operationalize their knowledge would involve presenting a fault situation on a simulator and fol-lowing the operator through the steps of the procedure. The problem is that many of the steps are not normally verbalized or otherwise made explicit We are consider-ing getting around this by engaging the user in a dialogue using a Learning Com-panion (Chan, 1990). This dialogue must be easy for the student to engage in, and must not interrupt his thinking in a distracting manner, since concentration is important when learning in a dynamic environment (Munro, Fehling and Towne, 1985). One benefit of the learning oompanion as a dialogue technique would be the possi-bility of actively analysing the student's onderstanding of the system to be learned, while still preserving the exploratory nature of learning with a simulator. A nice side-effect is that in forcing the student to express his reasoning, one increases the learning potential. The dialogue also makes the learning situation more similar to reality, in which diagnosis takes place as a cooperative effort among the operators in the control room. 267 DIAWGUE SPECIFICATION LANGUAGE AND TOOLS FOR AN INFORMATION SYSTEM OVER THE TELEPHONE Andrés Santos, José Colás, Juan Lestani Dpto. Ingenierfa Electrónica- E.T.S.I.Telecomunicación Univ. Politécnica Madrid - Spain 1. INTRODUCI'ION Speech technology is already in the stage of producing applications. An important area is based on services to provide information through the telephone without intervention of a human operator (see tigure 1). Today there are several companies that provide limited information using touch-tone telephones. These systems have at least two basic problems: the user can access to the information in a restricted and artificial way (menu-driven with codified commands) [1-3) and touch-tone telephones are not available everywhere (in some countries, like Spain, the percentage of touch-tone telephones is very small, around 5 % ). The success of automatic information systems depends highly on the use of speech technologies. Speech is the natural way to communieale among people of every culture. Any useful system needs speech recognition and speech production capabilities to operate through a telephone line. Above these modules there should be another one to control the dialogue flow so that the user gets the information: this module can be called the dialogue manager [4-6) (see tigure 2). Among other functions, it controts the interactions between the system and the user: it receives requests, decides whether more specific data from the user are needed, gives the information if available, etc.. To perform these tasks, it also has to interact with the application, using some access functions. Considering man-machine communication, there are two main restrictions: one is due to the capacities of the speech recognizer; the other to the possibilities of the system to understand and generate speech: a) There are several kinds of recognizers depending on their vocabulary and complexity, from the simplest that can only recognize isolated digits. More complex systems would have possibilities to recognize continuons speech with large vocabularies. Obviously these differences have a big impact on the performance of the whole system [7, 8). b) Depending on the system's ability to understand the user's speech, the dialogue can be menu-driven or unrestricted. In the first case, the user can 269 270 APDUC~TION S:ö= 0)", - positive ciosure noted "+" : A+ means: "from once up ton times A (n integer> 1)", - empty strin~ noted "t" : analogous to the empty set in set theory, is a character string of length zero, - omission noted "[ ]" : [A] means: A I E. As stated before, the BNF formalismis a particular context-free grammar. Widely used nowadays in formal specification of languages, its notations are the following : - the meta-linguistic variables are written between "<" and ">"separators, - the terminal symbols are written with no special separator, - the "::=" symbol means "is composed of': it is the well-known denvation operator, - the disjunction, noted" I", is only used in the right hand side of the production rules, because we manipulate here context-free grammars, - the conjunction is implicit and associated to the usual writing order (left to right) of terminaland non-terminal symbols. Stages in the formal definition of the VKE language The actuallanguage design process using the combined BNF and RE formalisms, from formal specification to use, has these complementary parts : initial static definition, then dynamic description at use by the expert. Thus to turn specification into action, we have to follow two steps. Static definition of the VKE language by a FG The BNF formalism is seen as a Knowledge Representation formallanguage. By combining them, we give the BNF productions of our static VKE language. The associated FG is composed of a finitesetof productions (its cardinality is 16), and starts from the axiom. In the following description, terminal symbols are indicated in lowercase, non-terminals between brackets. The rules are numbered for convenience of reference. No ordering is in fact implied. Readings have been inserted to facilitate locating sections of the grammar. The resulting sections are to be used as a rough guide only. Tenninal symbols : any string in lowercase, Non-tenninat symbols : any variabie between inferior and superior meta-symbols, Start symbol: VK ELICITATION, Meta-symbols : BNF constructars and operators, Production rules : Start symbo/ 1. ::= (. )+ Space and time 2. ::=[

Basics of man-machine communication for the design of educational systems : NATO Advanced Study Institute, August 16-26, 1993, Eindhoven, The Netherlands

Abstract

Similar works

Full text

Available Versions

Pure OAI Repository