Interaction by means of computer-mediated communications (CMC) is widel

Ally, Mohamed

Cookson, Peter

Crawford, Gail

Fahy, Patrick  J.

Keller, Verna

Prosser, Frank

English

University of Calgary Journal Hosting

The Alberta Journal of Educational Research Vol. XLVI, No. 1, Spring 2000,85-88 Patrick J. Fahy, Gail Crawford, Mohamed Ally, Peter Cookson, Verna Keller and Frank Prosser Athabasca University The Development and Testing of a Tool for Analysis of Computer Mediated Conferencing Transcripts Background Interaction by means of computer-mediated communications (CMC) is widely discussed in the distance education literature (Abrami & Bures, 1996; Anand & Haughey, 1997; Bates, 1995; Dede, 1996; Jonassen, 1998; Maier, Barnett, Warner & Brunner, 1996; Nixon & Salmon, 1996). However, much of the literature is more arbitrarily prescriptive or merely anecdotal than empirical (Gunawar-dena, Lowe, & Anderson, 1997). It appeared that, despite several possible candidates, a notable barrier to more rigorous research was the lack of a recognized C M C analysis tool. Process Beginning in late 1998 a team of faculty and students of Athabasca University's Master of Distance Education (MDE) program reviewed the literature on C M C analysis, comprising some 224 publications, seeking models or tools that might provide reliable analysis of C M C interactions. A total of 10 possible models or analytic approaches were identified (Fahy et al., 1999). A comparative review of the models was then conducted by the team, and two models (Bullen, 1997; Zhu, 1996) were chosen for further study based on their perceived (a) ease of use, (b) reliability, (c) validity, (d) theoretical support, and (e) cross-discipline utility. Features of the Analytic Models Selected for Further Research The Bullen and Zhu instruments focus on different aspects of C M C interaction. Zhu's (1996) work, based on Vygotsky's theory of proximal development, focuses on social negotiation, collaborative sense-making, and mentoring as ways of Patrick J. Fahy (patf@athabascau.ca) is an associate professor, Master of Distance Education (MDE) program, Athabasca University, 1 University Drive, Athabasca, AB, T9S 3A3, tel: 780-675-6216 or fax: 780-675-6170. Gail Crawford (gail@athabascau.ca) is an associate professor and acting director, MDE program, Athabasca University, 1 University Drive, Athabasca, AB, T9S3A3, tel: 780-675-6238. Mohamed Ally (mohameda@athabascau.ca) is an associate professor, MDE program, Athabasca University, 1 University Drive, Athabasca, AB, T9S 3A3, tel: 780-675-6406. Peter Cookson (peterc@athabascau.ca) is associate Vice-President, Academic, Athabasca University, 1 University Drive, Athabasca, AB, T9S 3A3, tel: 780-675-6262. Verna Keller (vkeller@watservl.UWaterloo.ca) is a graduate of the MDE program. She can be reached at tel: 519-885-1211 ext. 3132. Frank Prosser (fprosser@sd69.bc.ca) is a student in the MDE program, a full time primary teacher, and is currently part of a design team for both an online high school course (BC First Nations 12) and multimedia resources. He can be reached at Box 491, Errington, BC, V0R1V0, or tel: 250-248-9399. 85 P.]. Fahy, G. Crawford, M. Ally, P. Cookson, V. Keller, and F. Prosser improving participants' problem-solving capabilities. The model classifies postings as vertical or horizontal: • Vertical interactions seek an answer from a more capable or better informed member rather than contributing to or constructing knowledge collabora-tively. • Horizontal interactions assume there is no authoritative or correct answer, and members must therefore interact to construct an acceptable com-promise answer (p. 824). Z h u applied her instrument to conference transcripts and concluded there were two likely processes for knowledge construction in an electronic commu-nications environment: • Construction of knowledge by a group influenced, motivated, and facilitated by discussion and interaction among peers. • Simple assimilation of information proposed by others, perhaps with some per-sonal editing or adaptation, but without significant interpersonal interac-tion (p. 840). Bullen's (1997) C M C analysis tool focuses on the relation between critical thinking and participation. The classification categories are: 1. Seeking clarification: attempting to appraise and understand the exact na-ture of the problem, issue or dilemma, including acknowledging and at-tempting to understand different points of view on an issue. 2. Assessing evidence: judging the credibility of sources of information and ob-servations and the quality of the data offered as evidence. 3. Making and judging inferences: assessing the soundness of inductive and de-ductive inferences and value judgments used in making decisions. 4. Using appropriate strategies and tactics: using strategies as appropriate guides in critical thinking. The instrument was developed and tested on sample transcripts from a graduate course in distance education, which were analyzed by two re-searchers independently. The researchers reported high levels of agreement (approximately 90%) after analyzing two sets of transcripts. Further Modification and Piloting of the Zhu Instrument Although both the Bullen and the Z h u models produced promising results, Zhu's (1996) model was chosen by the research team for further development. Changes to the Z h u analytic tool resulted in a new tool called the "transcript analysis tool" (or M D E TAT) by the Athabasca University research team. The M D E T A T uses the following classification categories: 1. Vertical questioning. Emphasis is on the acquisition of data or information, wi th the question addressed to the person viewed as most likely to pos-sess the "correct" answer. 2. Horizontal questioning. A s in Zhu's formulation, the purpose is to initiate or invite a dialogue. Horizontal questions anticipate collaboration and discus-sion to produce an acceptable answer or compromise solution or to in-crease consensus. N o "correct" answer is assumed necessarily to exist. 3. Statements. These contain no self-revelation and usually do not invite dialogue. The speaker, like a lecturer, provides information or correction 86 Analysis of Computer-Mediated Conferencing Transcripts to an audience assumed to be uninformed or in error. A "correct" answer is implied, and the speaker believes he or she possesses it. 4. Reflections. The speaker reveals his or her internal conflicts, values, beliefs, reasoning processes, misgivings, and doubts and provides other insights into his or her personal, individual , and usually invisible thinking proces-ses. In reflections the speaker assumes listeners are interested in these per-sonal revelations, are empathetic, and w i l l respond with acceptance and support. Replies in kind are welcome, as are horizontal questions, scaffold-ing, and other "accepting" responses. 5. Scaffolding. The speaker invites others to comment. Scaffolding comments include those that call on or name others, refer to others' views, or address shared group experiences. The M D E T A T was tested in a graduate course in distance education, where transcripts were analyzed by two researchers independently. The interrater reliability between the two raters was high (94%). Following this first applica-tion, the tool was modified slightly and tested again by a third researcher on a new sample transcript. The third researcher and the two original researchers independently analyzed another transcript, with an interrater reliability among the three researchers of 84%. Finally, after further revisions, a third transcript was analyzed to determine ease of use and reliability. The re-searchers concluded that the tool was easy to learn to use, and reliability for the third set of transcript was 70% (Fahy et al., 1999). Conclusion We continue to investigate and apply the M D E T A T . (One of the student members of the group has recently completed a thesis, in part based on this work; Keller, 1999). Future efforts w i l l focus on making the tool more reliable and using it to analyze more lengthy and complex transcripts. A m o n g the questions we have posed for continuing investigation are: • What types of interactions are associated with greater or lesser levels of participation and satisfaction as reported by participants? • H o w does moderator conference behavior affect participants? • H o w does participant interaction affect the conferencing behavior of other participants? References A b r a m i , P., & Bures, E. (1996). Computer-supported collaborative learning and distance education. American Journal of Distance Education, 10(2), 37-42. A n a n d , D . , & Haughey, M . (1997). Instructors' orientations toward computer-mediated learning environments. Journal of Distance Education, 12(1/2), 127-152. Bates, A . W . (1995). Technology, open learning and distance education. N e w York: Routledge. Bullen, M . (1997). A case study of participation and critical thinking in a university level course delivered by computer conferencing. Unpubl ished doctoral dissertation. University of British Columbia . Available :http: / /courses.cstudies .ubc.ca/~bullen/Diss/thesis .doc Dede, C . (1996). The evolution of distance education: Emerging technologies and distributed learning. American Journal of Distance Education, 10(2), 4-36. Fahy, P., C r a w f o r d , G . , A l l y , M . , Cookson, P., Keller, V . , & Prosser, F. (1999). The development and testing of a computer conferencing transcript analysis tool. Paper presented at the I C D E Conference, Vienna. Gunawardena, C , L o w e , C . & Anderson, T. (1997). Analys is of a global on-line debate and the development of an interaction analysis model for examining social construction of knowledge i n computer conferencing. Journal of Educational Computing Research, 17,395-429. 87 P.J. Fahy, G. Crawford, M . Ally, P. Cookson, V. Keller, and F. Prosser Jonassen, D . (1998). Designing constructivist learning environments. In C M . Reigeluth (Ed.), Instructional theories and models (2nd ed.). M a h w a h , N J : Erlbaum. Available: h t t p : / / w w w . e d . p s u . e d u / i n s y s / w h o / j o n a s s e n / c l e / c l e . h t m l (pp. 1-21). Keller , V . (1999). Adaptation and application of a transcript analysis tool to analyze a computer-mediated communication (CMC) distance education course transcript. Unpubl ished master's thesis, Athabasca Universi ty . Maier , P., Barnett, L . , Warren, A . , & Brunner, D . (1996). Using technology in teaching and learning. L o n d o n : Kogan Page. N i x o n , T., & Salmon, G . (1996). Computer-mediated learning and its potential. In R. M i l l s & A . Tait (Eds.), Supporting the learner in open and distance learning (pp. 88-100). London: Pi tman. Z h u , E. (1996). M e a n i n g negotiation, knowledge construction, and mentoring in a distance learning course. In Proceedings of Selected Research and Development Presentations at the 1996 National Convention of the Association for Educational Communications and Technology (18th National Convention, Indianapolis). 

The Development and Testing of a Tool for Analysis of Computer-Mediated Conferencing Transcripts

Patrick J. Fahy

Gail Crawford

Mohamed Ally

Peter Cookson

Verna Keller

Frank Prosser

CiteSeerX

The development and testing of a tool for analysis of computer mediated conferencing transcripts

https://journalhosting.ucalgary.ca/index.php/ajer/article/download/54796/41853

The Development and Testing of a Tool for Analysis of Computer-Mediated Conferencing Transcripts

Abstract

Similar works

Full text

Available Versions

University of Calgary Journal Hosting

CiteSeerX