This paper is a research in progress to assess changes in perceptions of the academic value of creating web-based learning materials. A survey was conducted in 2002 that assessed perceptions of faculty and administrators concerning the value of creating web-based learning materials, especially as it pertained to tenure decisions. A second survey was completed in 2008 (approximately one “tenure generation” later) to see if perceptions concerning academic value of web-based had changed. The use of web-based materials has increased in the years since the first survey as well as the arrival of new ways to create web-based materials. While some may feel that the passing years and increased use of information technology in the classroom should have changed perceptions, initial results between the two surveys were generally similar. This paper is a presentation of the basic survey results and a more detailed analysis will be presented at the SAIS Conference. Survey collection ended in November and a detailed analysis of 2008 data has not yet been completed. Specifically, we wish to pursue a deeper analysis based on the respondents’ tenure status and gender

He, Ling

Schell, George

English

AIS Electronic Library (AISeL)

Association for Information SystemsAIS Electronic Library (AISeL)SAIS 2009 Proceedings Southern (SAIS)3-1-2009Changes in Perceptions of Web-Based learningMaterialsGeorge SchellHEL@UNCW.EDULing HeFollow this and additional works at: http://aisel.aisnet.org/sais2009This material is brought to you by the Southern (SAIS) at AIS Electronic Library (AISeL). It has been accepted for inclusion in SAIS 2009 Proceedingsby an authorized administrator of AIS Electronic Library (AISeL). For more information, please contact elibrary@aisnet.org.Recommended CitationSchell, George and He, Ling, "Changes in Perceptions of Web-Based learning Materials" (2009). SAIS 2009 Proceedings. 16.http://aisel.aisnet.org/sais2009/16He and Schell  Changes in Perceptions  CHANGES IN PERCEPTIONS OF WEB-BASED LEARNING MATERIALS Ling He University of North Carolina Wilmington HEL@UNCW.EDU George Schell University of North Carolina Wilmington SCHELLG@UNCW.EDU   ABSTRACT This paper is a research in progress to assess changes in perceptions of the academic value of creating web-based learning materials. A survey was conducted in 2002 that assessed perceptions of faculty and administrators concerning the value of creating web-based learning materials, especially as it pertained to tenure decisions.  A second survey was completed in 2008 (approximately one “tenure generation” later) to see if perceptions concerning academic value of web-based had changed. The use of web-based materials has increased in the years since the first survey as well as the arrival of new ways to create web-based materials. While some may feel that the passing years and increased use of information technology in the classroom should have changed perceptions, initial results between the two surveys were generally similar.  This paper is a presentation of the basic survey results and a more detailed analysis will be presented at the SAIS Conference.  Survey collection ended in November and a detailed analysis of 2008 data has not yet been completed.  Specifically, we wish to pursue a deeper analysis based on the respondents’ tenure status and gender. Keywords Online learning, web-based learning, tenure, learning effectiveness. INTRODUCTION The 2002 survey found that universities marginalize the value of web-based learning materials when tenure decisions are made (Schell, 2004).  During promotion and tenure decisions the value of research was important as was the value of teaching, but the creation of web-based learning materials by faculty was not considered important to the promotion and tenure decision.  It is common for the promotion and tenure decision process to be a sequence of gates or hurdles that a faculty member must pass in order to receive promotion/tenure.  Each gate must be successfully passed or the candidate is not considered further.  Although there are typically appeal processes at each step and although school processes differ, this model can serve as a basis for comparing the relative chances of obtaining promotion and/or tenure if a faculty member chooses web-based learning materials as a substantial component of his/her tenure package. Interest in what activities constitute scholarship worthy to count toward promotion and tenure was certainly stimulated by Boyer (Boyer, 1990).  He argued for a broader view of academic scholarship, one that shifted away from the current model which he felt was overly focused on research and publication.  With this broader view a resurgence in the academic value of teaching (the original value focus in higher education) and methods for teaching that better engage students and enlist their interaction in the education experience has been a subject of debate. From the preliminary analysis of the 2008 survey responses we want to see if differences occurred between the 2002 and 2008 data.  Specifically, is there a change in perceptions of the importance of research as a requisite condition to achieve promotion and/or tenure, a change in perceptions of teaching importance, and perceptions about the effectiveness of web-based materials versus traditional teaching methods?  Most importantly, we wish to see if development of web-based learning materials is more highly regarded as an academic endeavor in 2008 compared to 2002. SURVEY Thirty five questions are in the survey and they cover faculty demographics, school demographics, perceptions about web-based materials use and perceived effectiveness, and finally the respondent’s perceptions of the academic value of creating web-based learning materials by those persons and committees that will make decisions during the promotion and tenure process.  An e-mail was sent to 18,134 recipients explaining the survey purpose and asking the recipients to participate.  There were 1,075 usable responses for a response rate of almost 6%. Proceedings of the Southern Association for Information Systems Conference, Charleston, SC,  March 12th-14th, 2009 76He and Schell  Changes in Perceptions  The e-mail was sent to recipients that have publicly associated with web-based learning materials through a discussion forum, journal or conference presentation or publication, membership in a user group focused on web-based learning materials, or some similar connection.  In each instance, the person had furnished his/her e-mail address.  Of the respondents, 70% had taught a web-based course and 96% had used web-based learning materials in courses they taught.  This is important because this research seeks to measure perceptions of those faculty actively pursuing web-based materials and their perceptions on its effect of attaining promotion and/or tenure. A key component of the survey is the set of questions about the respondent’s perceptions of how others view the academic value of developing web-based learning materials.  Respondents rate their department colleagues, their department chair, the school committee for tenure and promotion, the dean, and the university administration.  Each of these persons/groups has input to the respondent’s tenure and promotion decision. SURVEY RESULTS The main purpose of the survey was to see if perceptions of academic value to web-based material development has changed during the last six years.  The six year time frame was chosen since it approximates the time between an initial hiring decision and a decision to award tenure.  If tenure is not awarded the faculty member would look for another position during the seventh year. The results had some consistency with 2002 data but did contain a surprise. The process of obtaining tenure and/or promotion can be modeled as a sequence of decisions that must be passed.  Each decision must be affirmative for the process to continue.  The model used here asserts that a tenure and/or promotion decision starts with department colleagues.  If a positive outcome occurs from the department colleagues then the department chair makes his/her decision.  Subsequent decisions are made by the school committee for promotion and tenure, the school’s dean, and the university administration.  Assuming all decisions are affirmative, the faculty member is promoted and/or tenured. An ANOVA was performed on each question concerning the perceptions of each group towards academic value of web-based materials.  For example, survey respondents were asked “What are your department colleagues’ views of academic worth for developing web-based learning materials?”  The respondent could choose 0 (no value in the promotion and tenure decision) to 10 (critical to the promotion and tenure decision).  There was no statistically significant difference between 2002 and 2008 responses for any of the questions on academic value – department colleagues’ views, department chairperson’s views, school committee on promotion and tenure, school dean, nor university administration. However, when considering the sequential affects of the promotion and tenure process, there was a considerable change as shown in Table 1.  The table is read as a sequence where the “Department Colleagues” row represents the number of candidates passed or rejected at that step of the promotion and tenure process.  The next step in the process, the department chair, is made only on those candidates that receive an affirmative vote from the department colleagues.  The process continues through the university administration decision.  The table contrasts the number of successful candidates, based upon the respondents’ perceptions, to traverse the promotion and tenure process. Entries in the table reflect a respondent’s entry of 5 or higher for the question.  For example, from the 2008 survey 58.8% of respondents marked that their department colleagues feel creating web-based learning materials was neutral (a score of 5) up to critical (a score of 10) in the decision for tenure and promotion.  58.8% of 1,000 yields 588.  Of those 588 passed by the colleagues, 389 candidates would be passed by the department chairperson while 199 would be rejected.  The process continues until 101 candidates would be passed through the process in 2008 compared to only 28 from the 2002 survey data. Decision Maker 2002  2008 Passed  Rejected  Passed  Rejected Department Colleagues  437  563  588  412 Department Chair  258  179  389  199 School Committee  112  146  222  167 School Dean  56  56  153  69 University Administration  28  28  101  52 Table 1. Candidates Out Of 1,000 That Would Be Passed Or Rejected At Each Stage Of The Promotion/Tenure Process This brings up an interesting issue.  Although more than three times as many candidates would persevere through the process in 2008 compared to 2002, the reality is that only 10% of the original candidates would have favorable decisions at the end.  Few universities would be able recruit new faculty members if they could only offer a 10% chance that a newly employed Proceedings of the Southern Association for Information Systems Conference, Charleston, SC,  March 12th-14th, 2009 77He and Schell  Changes in Perceptions  faculty member might achieve promotion and/or tenure.  Faculty that choose the development of web-based learning materials as the basis for their promotion and tenure do so at their peril. Several other differences appeared between 2002 and 2008 survey results.  The respondents ranked the importance of research in the promotion and tenure decision with a slightly higher mean in 2008 compared to 2002.  However, the F value produced a statistical significance of .088, more than the .01 or .05 values typically used as a basis for accepting a hypothesis that change has occurred.  See Figure 1 below.  Figure 1. Comparison Of Responses Concerning The Importance Of Research  The importance of teaching shown in Figure 2 indicated a statistically significant increase (.044) from 2002 to 2008.  Perhaps the debate over Boyer’s model is beginning to influence promotion and tenure decisions.  Or it could be that after six years more faculty are entering universities with the belief that some delivery of web-based materials are not really technology-based.  As Jason Frand described it “… if you can remember using your first one ever, it’s technology.” (Frand, 2000, page 16) By that description, many faculty would describe cell phones as ‘technology’ while their students would not.  Similarly, using Facebook to communicate with students is probably technology for some faculty members but not considered as such by others.   Figure 2. Comparison Of Responses Concerning The Importance Of Teaching  Proceedings of the Southern Association for Information Systems Conference, Charleston, SC,  March 12th-14th, 2009 78He and Schell  Changes in Perceptions  Figure 3 shows a disturbing change in the perceived effectiveness of web-based learning materials versus traditional course materials.  The 2008 responses show a decided drop in percieved effectiveness compared to 2002. The statistical significance level was .000 across the two survey years. Research is clear that web-based materials and traditional materials are equally effective (Bargeron, Grudin, Gupta, Sanocki, Li and Leetierman, 2002; Hiltz and Turoff, 2002).  Not only has this been shown in earlier research but it has been confirmed recently as well (Hiltz and Goldman, 2005).  The reason for the change in perceptions may not be captured from the data since the survey did not seek detailed responses on what makes teaching materials effective.  Figure 3. Comparison Of Responses Concerning Effectiveness Of Web-based Versus Traditional Teaching Materials  CONCLUSION The analysis of the 2008 survey data is in its early stages since collection ended in November, 2008.  However, initial results are interesting.  First, the respondents  perceived the attitudes of promotion and tenure decision makers in 2008 very similar to 2002 results.  No statistically significant changes, on a decision maker by decision maker basis, were found at any given point in the promotion and tenure decision process.  Yet the combined effects through the sequence showed that 2008 respondents felt there was much greater probability of succeeding through the promotion and tenure process than 2002 respondents – 10.1% as compared to 2.8%. Unfortunately, even the 2008 responses showed a perception that few faculty who make the creation of web-based learning materials the basis for their promotion and tenure would succeed.  This might be explained by the combination of the findings on the importance of research and the importance of teaching.  There is a change in the views toward research, an increase that is not quite statistically significant.  But the change in the views toward the impact of teaching importance on the promotion and tenure decision shows it has increased and by a statistically significant margin.  The creation of web-based learning materials is more closely associated with teaching than with research so the perceived increase in the number of faculty to achieve tenure on this basis is reasonable yet tempered by the realization that research is very important to a successful promotion and tenure decision. The large number of survey respondents and the focus on respondents who actually generate and/or use web-based materials is a key strength of the data.  Further analysis can be performed on school demographics such as the attitudes at schools offering doctorates versus schools that do not.  There could also be differences between public and private schools or between different schools within a university.  It would be interesting to see if professional schools (business, education, et al) view the value of creating web-based learning materials differently from their arts and sciences colleagues. Respondent demographics can also be explored.  Initial reviews of the data seem to imply that gender differences between 2002 and 2008 respondents has widened.  Gender issues when combined with other factors such as having a doctorate, tenure Proceedings of the Southern Association for Information Systems Conference, Charleston, SC,  March 12th-14th, 2009 79He and Schell  Changes in Perceptions  status, or length of time teaching may yield interesting results. This paper is still research in progress and we will work on further data analysis before the conference. The most puzzling finding of the 2008 data was the decrease in the perceived effectiveness of web-based learning materials compared to traditional materials.  In 2002 the frequency distribution was almost the exact symmetric distribution that research predicted. Quantitative analysis of material learned shows very little if any difference between the effectiveness of web-based materials compared to traditional materials.  Those findings continue even as new venues for web-based materials are used. The perceived decrease in effectiveness by the very faculty using and creating web-based learning materials is disturbing.  This is an area that requires further study of the data.  An expanded literature review will be performed to see if a theoretical explanation can be found. REFERENCES 1. Bargeron, D., Grudin, J., Gupta, A., Sanocki, E., Li, F. and Leetierman, S. (2002) Asynchronous collaboration around multimedia applied to on-demand education, Journal of MIS, 18,4, 117-145. 2. Boyer, E. L. (1990) Scholarship reconsidered: priorities of the professoriate, Princeton University Press, Lawrenceville, New Jersey. 3. Frand, J. L. (2000) The information age mindset: changes in students and implications for higher education, Educause Review, September/October, 15-24. 4. Hiltz, S. and Turoff, M. (2002) What makes learning networks effective, Communications of the ACM, 45, 4, 56-59. 5. Hiltz, S. and Goldman, R. (Editors)  (2005) Learning together online: research on asynchronous learning networks, Lawrence Erlbaum Publishers, Mahwah, NJ. 6. Schell, G. (2004) Universities marginalize online courses, Communications of the ACM, 47, 7, 53-56. Proceedings of the Southern Association for Information Systems Conference, Charleston, SC,  March 12th-14th, 2009 80

Changes in Perceptions of Web-Based learning Materials

https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1015&amp;context=sais2009

Changes in Perceptions of Web-Based learning Materials

Abstract

Similar works

Full text

Available Versions

AIS Electronic Library (AISeL)