This work evaluates the application of different learning technologies and their suitability to support blended learning approaches in Higher Education. Chickering and Gamsons's Seven Principles for Good Practice in Undergraduate Education (Chickering & Gamson, 1987) were used as an underlying pedagogical framework to evaluate the "perceived affordances‟ (Norman, 1999) of learning technologies.Chickering and Gamson‟s principles were selected as a framework due to their "face-validity‟, the accessibility of their language and since they have been derived from numerous years of reflective and effective teaching.Along with the principles we describe and recommend an innovative methodology for evaluation. This methodology can be used in a context of similar evaluation exercises.Final Accepted Versio

Cubric, Marija

Russell, Mark

English

This work evaluates the application of different learning technologies and their suitability to support blended learning approaches in Higher Education. Chickering and Gamsons's Seven Principles for Good Practice in Undergraduate Education (Chickering & Gamson, 1987) were used as an underlying pedagogical framework to evaluate the "perceived affordances‟ (Norman, 1999) of learning technologies.Chickering and Gamson‟s principles were selected as a framework due to their "face-validity‟, the accessibility of their language and since they have been derived from numerous years of reflective and effective teaching.Along with the principles we describe and recommend an innovative methodology for evaluation. This methodology can be used in a context of similar evaluation exercises

University of Hertfordshire Research Archive

Mapping the technology landscape: Linking pedagogy to the affordances of different technologies           Mark Russell      Marija Cubric Learning and Teaching Institute           Business School    University of Hertfordshire                            University of Hertfordshire       United Kingdom                                                United Kingdom   Introduction  This work evaluates the application of different learning technologies and their suitability to support blended learning approaches in Higher Education. Chickering and Gamsons‟s Seven Principles for Good Practice in Undergraduate Education (Chickering & Gamson, 1987) were used as an underlying pedagogical framework to evaluate the „perceived affordances‟ (Norman, 1999) of learning technologies.   The Seven Principles state, good practice... 1. encourages contact between students and faculty, 2. develops reciprocity and cooperation among students, 3. encourages active learning, 4. gives prompt feedback, 5. emphasizes time on task, 6. communicates high expectations, and 7. respects diverse talents and ways of learning.  Chickering and Gamson‟s principles were selected as a framework due to their „face-validity‟, the accessibility of their language and since they have been derived from numerous years of reflective and effective teaching. Interested readers can gain a background to the development of the principles in Chickering & Gamson (1999) and Gamson (1991). Despite the issues with the variability of principles across “disciplines, methods, learning styles, and institutions” (Sorcinelli, 1991) the principles remain a popular tool for guiding course design (Chizmar & Walbert, 1999) and for evaluating quality of teaching, learning and online courses (Graham, et al., 2001).    Along with the principles we describe and recommend an innovative methodology for evaluation. This methodology can be used in a context of similar evaluation exercises.  Methodology   The evaluation, of the technology and its ability to support blended learning, was performed using the “planning poker game” (Cohn, 2005), a popular technique borrowed from the field of software development and project management, where it is used for estimating the implementation effort of individual software tasks.   We exploited the game and its ability to help reach group consensus on the suitability of various popular technologies in supporting Chickering and Gamsons‟s principles of „good (education) practice‟.   In our case, the participants in the game were six lecturers from the University of Hertfordshire. All participants were familiar with the technologies being investigated and the underlying pedagogical framework.  A total of twelve technologies were evaluated including: wikis, blogs, discussion forum, electronic voting systems (EVS), wikis, tablet PCs, podcast, video clips, virtual classroom, computer assisted assessment, e-mail and audio-visual presentations.  In each round of the game a specific technology was evaluated with respect to one of the principles, leading to a total of eighty-four rounds of the game. The evaluation was completed in a course of five, one and two hour, sessions and the results were aggregated using the format presented in Table 1.   During a session, each participant was given a deck of identical cards numbered 1-5, and an ordered list of technology-principle (T,P) pairs to be evaluated, one in each round.  In each round, all participants worked individually in fixed time slots (up to 3 minutes) to describe a scenario in which the technology (T) might be used to support the given principle (P), and to rate the suitability of „pairing‟ (T,P) using the scale 1-5, where 5 denoted the highest suitability.  The cards were used to show the ratings each participant had selected.  After the expiration of a timer (alarm clock!), all participants had to lay facedown their selected cards, and to simultaneously turn them over to reveal the card value. In case of a disagreement between the participants, the highest and the lowest bidders were asked to justify their ratings. The primary descriptors used for justification were: strengths, weaknesses, opportunities and threats, of using the T in the context of the given P.  Using the justifications (when disagreement occurred), this procedure was repeated until either a consensus was reached or three rounds were completed without agreeing. In the latter case, the average of all ratings was recorded.  After each session, the notes including estimates, and specific learning and teaching scenarios recorded by the participants, were collected and presented using the same format (Table 2), one table per technology under the investigation. For example, the third column in the table for „EVS technology (Table 2) contains a description of a specific learning and teaching scenario that supports the third pedagogical principle („encourages active learning‟) and the corresponding strengths, weaknesses, opportunities and threats identified in the context of the scenario.   Attributes of the technology that were common to all scenarios were separated in column labelled „generic evaluation‟.  The justification descriptors had the following agreed meaning during the course of the evaluation:  Strengths - what are the intended and inherent benefits of using the specific technology (as opposed to anything else) in the context of a particular scenario?  Weaknesses - what are the drawbacks of using the specific technology (compared to other means) in the context of a particular scenario?  Opportunities – what are the non-intended or derived (“free”) benefits of using the specific technology (as opposed to anything else) in the context of a particular scenario?   Threats – what are the risks associated with using the specific technology in the context of a particular scenario (where it was assumed that the risk can reduce the positive impact or even reverse it to a negative or damaging)?   Evaluation  The results of our evaluation show which technologies are better suited to the different pedagogical objectives (Table 1) and also provide a description of different blended learning approaches (scenarios), which can be supported by specific technology in aiding a specific pedagogical need.   While there were no surprises in ratings of technologies (Table 1) true benefits have arisen through the formulation and analysis of the data i.e. it is not the technology per se but rather the use to which the technology is put that is important. The uses, or scenarios of the technology, are articulated to demonstrate the importance of having a pedagogical purpose in which to deploy the technology and to let „the pedagogy provide the lead‟ (Kirkwood& Price, 2005).    Table 1 Pedagogy vs. Technology Value Matrix  Principle -> Technology 1 2 3 4 5 6 7 Discussion forum 3 3 2 3 4 3 2 EVS  2 3 5 5 3 3 2 Wikis  1 4 3 3 3 4 2 Tablet PCs  2 1 3 2 1 1 1 Podcast  3 2 2 4 3 2 3 Video clips  2 3 2 3 2 3 3 Virtual Classroom 3 2 4 3 1 2 2 CAA  1 2 4 4 5 2 3 Email  4 1 1 2 1 1 3 Blogs  2 2 4 3 2 3 2 A/V presentations 2 1 3 1 1 2 4  A complete evaluation document containing analysis and scenarios for all twelve technologies under the considerations can be found at University of Hertfordshire LTI Resources web-site (http://bit.ly/aBQP7K ).  The results demonstrate that the methodology has lead to accurate evaluation of the technologies that is in line with the current body of knowledge in e-learning. The accuracy is primarily due to the competencies of participants (Jørgensen & Shepperd , 2007) but also, due to the new knowledge and learning occurring while justifying decisions (Jørgensen & Moløkken-Østvold , 2002), as not all participants were equally familiar with all technologies under the investigation.  Some of the already documented limitations of the methodology include: ratings become less optimistic after discussion i.e. there is a tendency to „over-rate‟; group decision could be vulnerable to peer-pressure (Moløkken-Østvold & Haugen, 2007).  In addition to that, choosing a very narrow rating scale (1-5), have led to high concentration of „average‟ ratings (Table 1). To deal with the issue a non-linear scale such as exponential or Fibonacci sequence could be used in future (Cohn, 2005).   The most important tacit outcome of this evaluation, as recognised by all participants, was that the methodology employed was crucial in instigating an in-depth conversation about the quality of blended learning and the resulting student experience.   Table 2 Analysis of learning and teaching scenarios supported by EVS Scenarios Generic evaluation 1 Using questions to identify what the class would like the tutor to focus on 2 EVS used to trigger discussion/or for group discussion  3 In-class structured formative testing  Strengths Instant response; In-class activity; Transparency of results-students can compare to others; Responses can be both anonymous or not; Fun Could stimulate discussion, non threatening Anonymous responses;   Results can be used to guide students on their own performance; Supports notions of prompt feedback and adaptive teaching; Breaks up the lecture and brings variety to the session; Weakness Cost prohibitive (loss of handsets); Requires staff training/confidence in use; Could promote guessing and „surface learning‟  Could discourage the asking of questions and dialogue amongst group members; Could be overly simplistic; Might not be sufficient to trigger discussion;  Need to bring students back to “order” after the activity; Authoring questions can be burdensome for staff;  Opportunities Stimulates discussion/class interactions Can be used to monitor attendance / hence encourages attendance at sessions; Students can be asked to write questions;  Could use after a series of questions to revisit something not understood; Could set up with complex questions Focus on in-class activities – creativity; Might stimulate extra studying by students – i.e.  in response to seeing their own incorrect responses Staff receive information feedback on how well their students understood the material;  Threats When it doesn‟t work you can look foolish and lose students‟ interest/authority; Possible overload of experience if all lecturers engage in the same format.  Bit school like – another barrier in the classroom? Not all students might engage; Students forget to bring handsets; Length of quizzes can be too long ; Requires teaching skill to break up the lectures;  Scenarios  4  EVS used in-class for formative feedback 5  EVS  quizzes used to test at start of lecture 6 Students mark or feedback a piece of work/or create topic questions  7 Exploring new areas where knowledge may not be well developed Strengths Quizzes are fun/interesting; Good for large groups  Higher in-class engagement;  Encourages students to prepare for lecture; Encourages student to prepare for specific topics;  New type of formative assessment;  Support negotiating skills, stimulates & encourage debate and discussion; Increase understanding of the marking criteria; Everyone can contribute anonymously, non threatening; Facilitate understanding the depth of subject & different levels of knowledge;  Weakness Competitive, not cooperative; In-class only technology; Group feedback only (not private); Need to bring students back to “order”; Mainly teacher led;  Value is finding the good examples  Giving a quick response is not good for students who are reflectors or with slower language comprehension like overseas students Opportunities Use in a group (e.g. pairs) ; Record and use for summative assessment; „On-spot‟ questions – checking understanding (feedback to staff too)  Forces teachers to think about student-centeredness; Lectures at least start by being student centred; Ask students to prepare questions.  Extend question base; Clarify areas of confusion Could be useful for getting a quick check on comprehension or answers to problems before they are worked through in class Threats Could encourage surface learning (students take it too seriously); Could disrupt lecture Relevance for high stakes – handsets missing/not working/forgotten   Students might perceive it as no value for them (need to be justified and explained beforehand) Does EVS activity suggest, “life is multiple choice”? Conclusions    Our 'technology / pedagogy map' flows from a consensus-building activity on the abilities of various technologies to respond to differing pedagogical drivers.  Using Chickering and Gamson's „Seven Principles for Good Practice in Undergraduate Education‟, we identified pedagogical scenarios for each of the technologies. These scenarios were analyzed and rated on their ability to respond to each of Chickering and Gamson's principles.   The results of this work, articulated through a 'technology/pedagogy map', are being used as part of the University of Hertfordshire's Curriculum Design Toolkit (Russell, 2010). It is anticipated that the technology map, and our Curriculum Design Toolkit will be of help in guiding other teachers and institutions that aspire to provide quality blended learning education in the future as well as in informing other providers of blended learning materials and tools.    Acknowledgment  We would like to thank our collegues Amanda Relph, Helen Barefoot, Sarah Flynn, Phillip Porter, who participated in the „planning poker‟ game and provided valuable informaiton for this paper. We sould also like to thank University of Hertfordshire LTI/Business School, for providing funds for presenting this work.  References   Norman D. (1999) Affordances and Design, Available  online at: http://www.jnd.org/dn.mss/affordances_and.html   Chickering, A. W., and Gamson, Z. F. (1987) “Seven Principles for Good Practice in Under- graduate Education.” AAHE Bulletin, 1987, 39(7), 3–7  Chickering, A.W., Gamson Z.F. (1999)  Development and Adaptations of the Seven Principles for Good Practice in Undergraduate Education, New Directions for Teaching and Learning, Volume 1999, Issue 80, pages 75–81, Winter 1999  Cohn, M. (2005) Agile Estimating and planning, Prentice Hall  Gamson Z.F. (1991) A brief history of the seven principles for good practice in undergraduate education, New Directions for Teaching and Learning, Volume 1991, Issue 47, pages 5–12, Autumn (Fall) 1991  Chizmar J.F and Walbert M.S. (1999) Web-Based Learning Environments Guided by Principles of Good Teaching Practice, The Journal of Economic Education, Vol. 30, No. 3,p 248-259  Graham, C., Cagiltay, K., Lim, B., Craner, J., & Duffy T. M. (2001). Seven principles of effective teaching: a practical lens for evaluating online courses. Technology Source. Available online at: http://ts.mivu.org/default.asp?show=article&id=839.  Jørgensen,  M. and  Shepperd  M. (2007) , A Systematic Review of Software Development Cost Estimation Studies, IEEE Transactions on Software Engineering 33(1):33-53  Kirkwood A., and Price L. (2005) Learners and learning in the twenty-first century: what do we know about students' attitudes towards and experiences of information and communication technologies that will help us design courses? Studies in Higher Education, Volume 30, Issue 3 June 2005 , pages 257 – 274  Jørgensen, M. and Moløkken-Østvold  K.J. (2002)  Combination of software development effort prediction intervals: Why, when and how? In Fourteenth IEEE Conference on Software Engineering and Knowledge Engineering (SEKE'02). -, Ischia, Italy, pages 425-428  Moløkken-Østvold  K.J & Haugen N.C. (2007) Combining estimates with planning poker, Proceedings of the 2007 Australian Software Engineering Conference  Russell M. (2010) Curriculum Design Toolkit. Available at: http://bit.ly/cOf4FF   Sorcinelli, M. D. (1991) Research Findings on the Seven Principles. New Directions for Teaching and Learning, No. 47, p13-25 Fall 1991  Biographies    Marija Cubric is a Principal Lecturer at University of Hertfordshire Business School, where she teaches information systems and project management related subjects. She is also a member of the Learning and Teaching Institute that promotes the use of innovative and effective learning and teaching practices across the University. Before joining academia in September 2004, she worked on system and software development for telecommunication industry in UK and Canada. She holds PhD degree in Computer Science (1994) from Concordia University in Montreal and MA in Learning and Teaching in Higher Education from University of Hertfordshire (2007).  In October 2009, Marija was awarded the first University of Hertfordshire Readership in the area of e-learning. Her research interest include collaborative learning with wikis, computer assisted assessment, learning ontologies and agile practices in project management and business development. She is a Fellow and Chartered IT Professional of British Computer Society, and Fellow of the Higher Education Academy.     Mark Russell gained his first degree in Building Services Engineering and is recognised as a Chartered Engineer by the Engineering Council. His PhD is in the area of educative assessment relating and in the use of technology to provide educationally effective and resource efficient assessment. Mark holds a Post-Graduate Certificate in Teaching and Learning in Higher Education and is a Fellow of the Higher Education Academy. This duality of profession, Engineer and Teacher, has enabled Mark to present his work to his peers in the arenas of both engineering and education. Mark‟s teaching and learning interests are varied, and include assessment, exploring the effective use of technologies to support in-class activities, developing collaborative learning opportunities and developing innovative tools for electronic-assessment. Mark‟s current interests lie in the area of Just-In-Time teaching, using the students' own understandings to help guide the lecture experience and curriculum design. In addition to winning the Times Higher Education Supplement / Generic Centre of the Learning and Teaching Support Network 'E-Tutor of the year competition 2003', Mark is also a National Teaching Fellowship holder (2005). Mark is currently seconded from the School of Science and Engineering Technology to work as the Blended Learning Unit‟s Deputy Director. The Blended Learning Unit is a UK HEFCE funded Centre for Excellence in Teaching and Learning. 

Mapping the technology landscape : linking pedagogy to the affordances of different technologies

Abstract

Similar works

Full text

Available Versions

University of Hertfordshire Research Archive