Evans, Gilda A.

Gibson, Anne L.

English

Biodiversity Informatics

D TOOLS OF THE TRADE What's In It for Me?: Evaluating Software and Video In this paper, the authors provide a temploJe for improvement of academic software as an evaluating the usefulness of software and video evolutionary social process; it is by discussing the materials in foreign language teaching and criteria of good software over time that academic learning. The evaluation criteria are drawn computing will be improved, that computers will from a variety of sources, including some from come to have their greatest impact." outside the field of foreign language pedagogy. In filling out the template, these criteria are to Rationale be considered. T his article synthesizes and attempts to make cohesive. the many disparate criteria for the evaluation of foreign . language teaching materials incor-porating video and computer. As Hakansson (1981) said, '~ll individuals will develop their own criteria, weighing items differently depend-ing on the setting, the audience, and the educa-tional goals to be met." Individuality notwithstanding, there is a need for consistency in evaluation. For example, what meaning should one inject into "would not recommend this movie for beginners," or "difficult to follow this computer program."? One reviewer might judge a movie or computer program on its artistic value while another reviewer judges them in terms of linguistic merit for the classroom. Both reviewers may essentially end up by saying '~n excellent film (computer program); I highly recommend it." Do we want evaluations of classroom teaching materials to be based on unsubstantiated com-ments or the criteria of personal taste and bias? Or should our goal be to initiate a move toward national guidelines-not to be confused with rigid lists-that would lead to better informed professionals? As Kozma (1989, p. 19) observed: "Establishing ... a crystallized list may not be our ultimate goal ... it may be best to think of the 54 Winter 1989-1990 In a recent graduate class on technology in the foreign language classroom, we assigned product reviews: Students were asked to fill out weekly reviews on both software programs and video-tapes in any language. The purpose of this assignment was three-fold: Students would become better informed about materials available in today's market; they would become more knowledgeable about the laboratory teaching and learning resources at our institution; and, they would hone their word processing skills. We believe that one of the basic services a language learning center can provide is a precise list of --'materials and information about what users can expect when using such materials. Titles of works, in and of themselves, are not sufficient and can often be misleading. For instance, "Corrida de Toros" is not about bullfights as one might expect. Commercial reasons frequently create program titles; such title labels often give very little information about actual content. A Prototype Evaluation Tool The evaluation tool designed to organize program information in a usable format contains some rubrics that are self-explanatory, e.g., language, level, and program type; other rubrics call for multiple responses because they are open-ended, e.g., lesson goal, strengths, and drawbacks. As we designed it, the evaluation tool is a template with guidelines for use on the Macintosh computer which spells out computer Journal of Educational Techniques and Technologies instructions needed by the user to fill out the template. One advantage of using a computer template-as opposed to filling out an evaluation on printed forms-is that the computer permits each template field (Language, Lesson Goal, Strengths, etc.) to be expanded to fit the amount of information the reviewer may wish to provide, be it one line or ten. Furthermore, information that is arranged in repetitive blocks is helpful to the user who can consult the template at a glance and scan for particular information in a recurring and fixed place, e.g., "Drawbacks" always appears at the end of the template. The open-ended categories or criteria of the templates can be challenging. Fortunately, there are many books and articles (Ahmad, et al., 1985; Altman, 1989; Hope, et al., 1984; Last, 1984; Schleger, 1988; & Underwood, 1984) in which experts discuss such criteria; it is from these discussions, in part, that we prepared guidelines for our graduate students. In addition, we organized the categories according to teachers' information needs, e.g., placing "Language" and "Level" first and then proceeding to other details about the items. The Template PRODUCT NAME LAB NUMBER HARDWARE REQUIRED Language/Level Lesson Goal Strengths PRODUCT NAME PUBLISHER/DISTRIBUTOR LAB NUMBER (of the holding as catalogued at your institution) without this kind of reference, language learners and teachers will be unable to locate the reviewed item HARDWARE REQUIRED A. For Software (usually this information can be found on the software package) Mac Plus, e.g., no hard drive, no external drive Mac Plus, 1Mb of RAM, hard drive preferred or external drive Mac SE, Mac II, etc. ffiM/clone, e.g., 512k or above Apple IIc Apple nGS Compaq Commodore I I Type Drawbacks B. For Video Multistandard VCR VHS Beta Three quarter inch, half inch NTSC or PAL LANGUAGE Specify the target language for which the program is designed LEVEL Specify if appropriate for first, second, third, fourth year (or above) and whether High School or College. This levelling may be done using a classification system familiar at one's institution, e.g., French 1441, or better yet, by ACTFL proficiency levelling, e.g., novice-mid, inter-mediate-high, etc. TYPE (use "S" for software, "V" for video) Winter 1989-1990 55 Journal of Educational Techniques and Technologies A. Software Program Types • "Thtorial" for introducing new material • "Drill and Practice" for mastery of already presented materials • "Game" with scoring, timing and competition elements • "Holistic practice" with contextualized practice activities • "Modeling" for how to perform a language task • "Discovery" for making linguistic generalizations and inferences • "Simulation" for experimenting with language use such as style or dialects • ')\dventure reading" such as in a murder mystery • "Reading Annotations" for information that is available on demand • "Idea processor" for preparing outlines • "Word processor" for writing • "On-line thesaurus" • "Spelling checker" • "Textual analysis" for literary analysis B. Video programs, type, and duration • Video segments made for FL pedagogical purposes • Commercials • Cartoons • TV programs such as: soap operas talk shows news music/variety • Feature films art films documentaries • Travelogues LESSON GOAL This section needs to be answered by evaluators in as much detail as possible; there is no way, however, to set up absolute standards for this section. We recommend that catalogue descriptions from vendors be a starting point; these can then be expanded or reduced. It is also helpful to include the skill(s) addressed by the program: listening, speaking, reading, writing, grammatical analysis, development of vocabulary and cultural aWdl'eness. Smith (1987) gives details on activities suitable for computer or video; they are divided by skill and level. 56 Winter 1989-1990 STRENGTHS/DRAWBACKS For this section of the evaluation, we encourage the reviewer to consider all of the following pedagogical and technical/pragmatic concerns. The reviewer should select the applicable points and give a brief rationale in the template summary. PEDAGOGY A. For software and video • Does it seem to skip from topic to topic • Is it pedagogically sound and worthwhile • Is length appropriate for mastery • Is there accuracy of language (often, the technologically sophisticated programs are deficient in sound language content. Certain new computer programs, e.g., HyperCard, allow non-technical people to author their own software and thus take advantage of their language expertise.) • At what level does program operate: word, phrase, sentence, paragraph, or discourse • What is the nature of the language: colloquial, literary, dialectical • Is it culturally authentic B. If software • Are methods inductive or deductive, e.g., present grammar patterns inductively without attempting formulation of rules • Are there checkpoints or tests in the program • Is practice meaningful or mechanical • Does it provide hints toward correct answer • Does it provide explanations for why correct answers are correct • Does it anticipate incorrect answers and provide explanations • Is it catalytic promoting group work • Is there instant and appropriate feedback • Is length appropriate for mastery C. Ifvideo • Is it subtitled • What type(s) of cultural situations are depicted • Is there anything special in film for culture or language learning, e.g., good body language • Does it contain short stand-alone sequences or can it be easily segmented • Is the speech rate understandable for a beginner, intermediate, advanced student Journal of Educational Techniques and Technologies • Does the non-verbal stimuli help clarify Usefulness of Product Reviews the verbal flow PRAGMATICS A. For Software and Video • Are the newly learned items readily usable in class, in conversation with natives, to understand a new movie, etc. • Is there effective use of color • Is there a teacher's manual • Is there a student workbook • Does the program capture attention • How many copies does program copyright allow B. For software • Are there clear directions for use • Is program easy to use, i.e., is the learning curve short • Is help available • Is there adequate documentation • Are computer screens uncluttered • Does it allow creative responses • Is it interactive • Are rewards available • Is there any student control • Is there branching of segments • What type of response is required by the program • Are there record-keeping options available to teacher, i.e., program keeps track of score, etc. • Is program making good use of the computer, i.e., real advantage of computer is the capacity to make decisions quickly, carry out operations precisely and in several layers simultaneously • Can student exit program at any point • Can student skip an item • Does program run when wrong answers are entered • Does program run when incorrect keys are touched mistakenly • Are clear prompts provided to students • Does program proceed automatically regardless of responses, i.e., are there built-in time limits C. Ifvideo • Is audio quality good • Is resolution good • Are charts and pictures included for clarification • If subtitling is present, is lettering easy to read The authors recommend the product-review assignment for graduate or upper-level students in a methodology/technology course on a recur-ring basis because it serves to better inform them of the ever-changing materials, and simultane-ously provides current evaluations of lab holdings. Our graduate students-largely high school teachers-commented that this was one of the most useful assignments for them profes-sionally because it helped them look with a more critical and reliable view at software and video programs. Consequently, they felt better prepared to make recommendations for purchase of lab materials at their own schools. The reaction of language students, teachers, and laboratory staff at the University of Texas at Arlington to the reviews has been very positive. Users cite the ease of use of the format-based on the template and its recurring descriptors-as the reason for the "well-thumbed" appearance of the review compilation. Maintaining quality control of the evaluations is the major challenge; we addressed this challenge in two ways. The authors-French and Spanish language teachers, respectively-had personal knowledge of many of the materials reviewed and were able to judge the accuracy of the reviews. The hours spent in compiling the reviews made it possible for the authors to study each review and reject those considered inadequate because they furnished little information or were clearly inaccurate. Where the reviewers differed about the value of an item, we tried to include all shades of opinions. We also obtained a number of reviews from native speakers of Spanish and German which were helpful as comparisons. Such multiple reviews of selected items will again be possible in 1990 when graduate students in our projected technology course will again be assigned to evaluate laboratory materials. We hope this frequent usage by graduate students-most of whom are language teachers-will lead to enhancement of the guidelines and the template. The authors welcome reactions and suggestions from readers for using or modifying the template. Winter 1989-1990 57 Journal of Educational Techniques and Technologies Conclusion Evaluating software and video for language teaching promises to be a necessary and on-going reality. As the proliferation of software and video programs continues, product evaluations that adhere to some standardized guidelines are not only needed but also very helpful. Standardized evaluation procedures enable coordinators and media laboratory staff to provide better information to users and language teachers who wish to integrate the laboratory experience with classroom learning and teaching materials. References Ahmad, K., Corbett, G., Rogers, M. & Sussex, R. (1985). Computers, language learning, and language teaching. Cambridge: Cambridge University Press. Altman, R. (1989). The video connection. Boston: Houghton Miflin, Company. 58 Winter 1989-1990 Hakansson, J. (1981). How to evaluate educational courseware. The Journal of Courseware Review, 1(1). Hope, G., Taylor, H., & Pusack, J. (1984). Using computers in teaching foreign languages. Orlando, Florida: Harcourt Brace Javanovich. Kozma, R.B. (1989). Reviewing academic software. Syllabus. 5(April), p. 19. Last, R. (1984). Language teaching and the microcomputer. Oxford: Basil Blackwell. Schleger, P. (1988). Video review evaluation and rap session. Dallas, Texas: ASTD National Conference. Smith, W.E (Ed.) (1987). Modem media inforeign language Applications. Lincolnwood, IL: National Textbook Co. Underwood, J. (1984). linguistics, computers, and the langauge teacher. Rowley, MA: Newbury House Publishers. JoE.ToTo Contributor Profile Gilda A. Evans and Anne L. Gibson are at the University of Texas at Arlington. Interested readers may write to them at the following address: Department of Foreign Languages and Linguistics, Box 19557, Arlington, TX 76019-0557. 

What's in it for Me? Evaluating Software and Video

The University of Kansas: Journals@KU

https://journals.ku.edu/IALLT/article/download/9363/8841/

What's in it for Me? Evaluating Software and Video

Abstract

Similar works

Full text

Available Versions

Biodiversity Informatics

The University of Kansas: Journals@KU