This paper presents an implementation of a decision support system to help tutors think about ways of using dialogue to support learning. The approach adopted has been to develop a software toolkit around a knowledgebase of dialogue methods, to assist tutors in the reflection required during the planning and design of dialogue to support learning

Cook, John

Oliver, Martin

Smith, Matt

English

Arrow@TUDublin

The ITB Journal Volume 3 Issue 2 Article 5 2002 Learning Through Dialogue (LTD) - a Toolkit to Support the Process of Planning for Effective Use of Dialogue in Learning Matt Smith John Cook Martin Oliver Follow this and additional works at: https://arrow.tudublin.ie/itbj  Part of the Educational Methods Commons Recommended Citation Smith, Matt; Cook, John; and Oliver, Martin (2002) "Learning Through Dialogue (LTD) - a Toolkit to Support the Process of Planning for Effective Use of Dialogue in Learning," The ITB Journal: Vol. 3: Iss. 2, Article 5. doi:10.21427/D7MD0X Available at: https://arrow.tudublin.ie/itbj/vol3/iss2/5 This Article is brought to you for free and open access by the Journals Published Through Arrow at ARROW@TU Dublin. It has been accepted for inclusion in The ITB Journal by an authorized administrator of ARROW@TU Dublin. For more information, please contact yvonne.desmond@tudublin.ie, arrow.admin@tudublin.ie, brian.widdis@tudublin.ie. This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 License ITB JournalDecember 2002                                                                                                                                                         Page 34Learning Through Dialogue (LTD)- A toolkit to support the process of planning for effectiveuse of dialogue in learningMatt Smith1, John Cook2 & Martin Oliver31 School of Informatics and Engineering, Institute of Technology at Blanchardstown, Dublin 152 Learning and Technology Research Institute,London Metropolitan University, UK3 Higher Education Research and Development Unit,University College London, London UKContact email: matt.smith@itb.ieAbstractThis paper presents an implementation of a decision support system to help tutors thinkabout ways of using dialogue to support learning. The approach adopted has been todevelop a software toolkit around a knowledgebase of dialogue methods, to assist tutors inthe reflection required during the planning and design of dialogue to support learning.KeywordsPedagogical issues, teaching/learning strategies, improving classroom teaching.1. IntroductionThe effective use of dialogue to support learning in Higher Education is complex anddifficult. For educational practitioners with limited experience of dialogue methods, the taskof designing (or redesigning) such teaching can be problematic and time consuming. Theresearch described in this paper describes one strategy to support tutors in pedagogical use ofdialogues. This project also works to contribute to wider issues about the developmentprocess for decision support toolkits, and the improvement of understanding about relativebenefits of using dialogue to support different kinds of learning in different domains.First the paper explores issues about the use of dialogue to support learning, followed byissues about software toolkits in general and their development. The paper then presents thedesign and development stages of  LTD  the Learning Through Dialogue Toolkit" itself,from initial design, through evaluation of a paper-based prototype, to full softwareimplementation.ITB JournalDecember 2002                                                                                                                                                         Page 352. The use of dialogue to support learningThere are many techniques for using dialogue to support learning. New methods and moderninterpretations of older ones have been encouraged by increased availability of Internettechnologies and systems to support asynchronous communication. Examples of dialoguemethods to support learning include Socratic approaches to repeated questioning, such asimplemented the WHY computer-based system (Stevens, Collins & Goldin, 1982), andmodern interpretations of Socratic dialogue, such as the theory of inquiry teaching proposedby Collins & Stevens (1991). Dialogue can be structured through formal rules for turn taking,such as the dialogue games described by Levin & Moore (1977). Others have exploredapprenticeship models of the community of inquiry approach (the TAPS project, Derry 1992)and learning assistants to support collaborative dialogues (the MetaMuse system, Cook2001).There is certainly no single correct approach that will meet all pedagogic aims, and foreducational practitioners with limited experience of dialogue methods, the task of designing(or redesigning) such teaching can be problematic and time consuming. Designing acomputer-based system to support the planning for effective use of dialogue in learning is anon-trivial task, which we have tackled through the development of LTD, a software toolkit.3. Toolkits to support decision makingA toolkit is a software system to support a design process (in our case, the design ofteaching that effectively incorporates dialogue methods). Toolkits support decision makingbased around an expert model of the design process  they provide a structure for decisionmaking and can make recommendations, based on goodness of fit between descriptions ofproblems elicited from the user and a knowledgebase of possible solutions. Oliver & Conole(1999) describe toolkits in more detail.For the toolkit we describe in this paper we have followed the methodology for toolkit design(Conole & Oliver, in press), which can be summarised as follows:ITB JournalDecember 2002                                                                                                                                                         Page 36(A) Identification of a suitable theoretical framework for design(B) Toolkit specification (how can the range of options available at each stage betranslated into a practical but flexible form of guidance for non-experts?)(C) Toolkit refinement: evaluating prototypes (how useful and flexible is the toolkit?)(D) Inclusion of user-defined features(E) The development of shared resourcesFor the LTD toolkit the first two stages (A and B), and the first iteration of stage C have beendescribed in detail previously (see Cook & Oliver, in press), and are only briefly summarisedhere. The focus of this paper is in presenting details a second iteration of the prototypingstep (C)  the development of a full software prototype of the LTD toolkit.4. (A) Identification of a suitable theoretical framework for designNo existing model could be found that described the process of comparing and contrastingdifferent discursive formats for education. Therefore relevant case studies were consideredand the following 5-stage model was derived: Identification of learning need Elicitation of learning objectives Elicitation of detailed description of task and context The filtering of options and recommendation of suitable approaches Selection, investigation and adoption of a suitable approach by the user5. (B) Toolkit specification: how can the range of options available ateach stage be translated into a practical but flexible form ofguidance for non-experts?A prototype toolkit was designed around the five decision-making steps identified in thetheoretical framework for the design task (steps 1 to 5 above). Descriptions are provided tothe user at each decision-making step, and the user is required to perform an activity/make achoice, while able to interrogate options and knowledgebase contents. The toolkit approachrequires the organisation of information in layers, allowing users to engage deeply withcontent/decisions they find important, and able to quickly skip over, or bypass completely,steps with less relevance to their particular teaching requirements.ITB JournalDecember 2002                                                                                                                                                         Page 37A knowledgebase was designed to contain details of different kinds of dialogue methods(such as court of law, structured debates, and Socratic dialogue). To assist users in finding agood fit between their teaching aims and available dialogue methods a knowledge spacewas developed to differentiate between features of different dialogue methods. Thisknowledge space comprised the following set of criteria identified as useful fordistinguishing between dialogue methods: Whether or not it is important to reach a right answer. Whether the method emphasises collaboration or competition. The duration required for a dialogue of this type. The numbers of participants required. Issues of power relationships within the discussion.These descriptors were adopted on their pragmatic value in discriminating betweenalternatives in a way that is likely to identify useful approaches. Since these descriptors arenot necessarily independent of each other, nor exhaustive etc., part of the evaluation of ourtoolkit will require validation of this knowledge space.The structure of the dialogue methods knowledgebase was based on this set of descriptors.Part of the design of the interactive aspects of the prototype toolkit involved asking the userto rate each of their learning objectives using this same mapping used to represent methods inthe knowledgebase. For the toolkit design the questions asked and ranges of answerspermitted were as follows: Is there a right/wrong answer? (Options: there is an absolute answer, there are criteria forright answers, very open/free.) Should the dialogue promote competition or collaboration? (Options: there will be clearlyidentified winners and losers, the dialogue will be highly competitive, some peoplewill do better than others, success will only be judged in terms of the group as a whole,credit will be given for listening to others and drawing on evidence.) What will the duration be? (Options: hours, days, months.) What will the group size and level of tutor support be? (One interval, e.g. 25-30students, one integer.) Issues of power (who defines rules and roles?) (Options: defined and assigned by thetutor, negotiated between tutor and participants, determined by participants, noformalisation of power/roles.)ITB JournalDecember 2002                                                                                                                                                         Page 386. (C) Toolkit refinement: evaluating prototypesA first, paper prototype was developed and evaluated with users through the "Wizard ofOz" approach (see Cook & Oliver, in press). This paper-based prototype toolkit was testedwith teachers/tutors, and evaluated to assess its suitability, ease of use, flexibility andrelevance. The result was that the following decisions were made to refine the toolkit design,based upon the results of the prototype evaluation: reduce number of steps by requiring the user to focus on a single learning objective at atime (thus there are now 4 steps, with the new step 2 combining the original steps 2Elicitation of learning objectives and 3 Elicitation of detailed description of task andcontext) provide a range of detailed examples for each user input, so the user understandsappropriate type of entry  Other design decisions implemented in the refined system include: allow non-linear navigation  so user can visit decision steps in any order, trying outdifferent choices (what if scenario experimentation) and seeing how recommendationschange knowledgebase entries rank criteria on a Real number scale from +1.0  to 1.0, thesystem returns goodness of fit to the user in terms of an average match between userchoices and knowledgebase entries in the range +100.0% to 100.0% (so a simplemeasure of closeness of match of alternatives is available, rather than simply the rankposition of alternatives)7. Second iteration of step (C) Toolkit refinement: evaluating prototypesBuilding upon the findings of the evaluation of the paper-based prototype, a full softwareprototype has been developed and is the focus of this paper. The following diagram (Figure1) presents a simple overview of the toolkit and interactions:Figure 1: Overview of LTD toolkit.ITB JournalDecember 2002                                                                                                                                                         Page 39Figure 2 illustrates a screenshot of the toolkit home page:Figure 2: The LTD home pageNavigation is performed by clicking on the image map at the top of the window, or on thebuttons on either side of the image map.7.1 Step 1 – Learning contextStep 1 involves the entry of details about the learning context. The screenshot below (Figure3) illustrates entry of details for Step 1.Figure 3: Learning context (Step 1).ITB JournalDecember 2002                                                                                                                                                         Page 407.2 Step 2 – Learning objective ratingStep 2 supports the user in describing in detail features of the learning outcome that theteaching/learning is to support. Textual descriptions of the learning objective, and how itsachievement is to be judged are provided by the user. The user then rates their learningobjective in terms of the knowledge map that encodes the knowledgebase entries. Examplesof two of the questions, and structured choices for each have been implemented are asfollows:B: Degree of encouragement of collaboration "(doesn't matter)", "Individual competition required", "Individual competition encouraged", "Distinct contributions to group encouraged", "Collaboration within group encouraged"E: Definition of power roles "(doesn't matter)", "Tutor assigns roles & rules", "Tutor and Participants negotiate roles &rules", "Participants assign roles & rules", "There is no formalisation of roles & rules"The figure below is a screen shot of example entries and choices for Step 2.Figure 4: Details of Learning Outcome (Step 2).7.3 Step 3 – Method selectionStep 3 is where the toolkit ranks each method in the knowledgebase against the rating choicesmade by the user for the learning objective. Rankings can range from +100% to 100% -these figures are based on the mean of the closeness of the match between each criterionentry for a dialogue method and the actual ratings entered by the user. An example of theITB JournalDecember 2002                                                                                                                                                         Page 41presentation of ranked dialogue methods is illustrated in the following screenshot (at the timeof writing only court of law has a full knowledgebase entry, the knowledgebase is beingpopulated with a range of dialogue methods over the next few months).Figure 5: Method Selection (Step 3).A simple explanation facility is provided, providing details of the percentage match of eachcriterion entered by the user with the selected knowledgebase method. This is illustrated inFigure 6 below.Figure 6: Excerpt from explanation of selected method score.ITB JournalDecember 2002                                                                                                                                                         Page 427.4 Step 4 – Summary ReportStep 4 collates all the entries by the user as a summary report for printing or saving to file.An example screenshot from the toolkit presenting a summary report is as follows:Figure 7: Summary Report (Step 4).9. Conclusions and further workThe notion of principled design was one of five current strategic issues identified by Harasim(2001). In this paper we have described a software toolkit that allows practitioners to usedialogue in learning in an educationally driven way. We feel that LTD provides asimplification of the complex process of effectively incorporating dialogue into learning. Thetool is flexible in that it allows users to skip over sections they feel irrelevant and engagemore deeply with content they find important. The non-linear navigation supports easyidentification of alternatives and the browsing of their knowledgebase entries.The contribution of this paper is a report on the implementation of the LTD softwareprototype. The LTD system will, we hope, provide a useful decision support tool forteachers/tutors in all sectors of learning  enabling them to plan their approach toincorporating dialogue in their own learning context.ITB JournalDecember 2002                                                                                                                                                         Page 43Future work will test our claims with user evaluations of the software prototype. We will thenprogress to the final two steps (D and E) in the toolkit design methodology.10. Location of the toolkit and invitation to contributeThe toolkit can be found at the following URL.www.itb.ie/staff/mattsmith/The toolkit is implemented in HTML and Javascript.ReferencesCollins, A. & Stevens, A. L. (1991). A Cognitive theory of Inquiry Teaching. In P.Goodyear (Ed.), Teaching Knowledge and Intelligent Tutoring, 203-230. Norwood, NJ:Ablex.Conole, G. & Oliver, M. (in press). Embedding Theory into Practice Using Toolkits.Journal of Interactive Media in Education.Cook, J. (2001). Bridging the Gap Between Empirical Data on Open-Ended TutorialInteractions and Computational Models. International Journal of Artificial Intelligence inEducation, 12, 85-99.Cook, J. & Oliver, M. (in press). Designing a toolkit to support dialogue in learning.Computers and Education.Derry, S. (1992). Metacognitive Models of Learning and Instructional Systems Design. InM. Jones & P. Winne, (Eds.), Adaptive Learning Environments: foundations and frontier.Hamburg: Springer-Verlag.Harasim, L. (2001). The Future of Learning (Keynote Lecture). In CAL 2001 Learningacross the ages–looking back and looking forwards, Abstract Book, p. 39. ElsevierScience.Oliver, M. & Conole, G. (1999) From Theory to Practice: A Model and Project Structurefor Toolkit Development. BP ELT Report No. 12, University of North London.Stevens, A. L., Collins, A. & Goldin, S. E. (1982). Misconceptions in StudentsUnderstanding. In D. H. Sleeman & J. S. Brown (Eds.), Intelligent Tutoring Systems, 13-24. London: Academic Press.

Learning Through Dialogue (LTD) - a Toolkit to Support the Process of Planning for Effective Use of Dialogue in Learning

https://arrow.tudublin.ie/cgi/viewcontent.cgi?article=1041&amp;context=itbj

Learning Through Dialogue (LTD) - a Toolkit to Support the Process of Planning for Effective Use of Dialogue in Learning

Abstract

Similar works

Full text

Available Versions

Arrow@TUDublin