In-service teacher enhancement workshops are held as part of the NASA teacher preparation and enhancement programs. For these workshops, teachers are brought to NASA field centers during summer months for training in various aerospace related areas. The National Research Council, Committee on NASA Education Program Outcomes (NEPO) has recommended comprehensive data collection to show program efficacy. Teacher workshops are a part of the nearly 300 NASA educational programs for students and teachers from pre-school through post-graduate, and NEPO has made specific recommendations for the evaluation of the teacher workshops. In accordance with one recommendation, a questionnaire was administered during four of the summer 1995 teacher workshops to measure participant concerns about using workshop material (space science) in their classes. Participant responses to the questions reflect changes in concerns relative to using workshop materials in teaching. (Work performed under NASA contract: NAS1O-1218O

Dreschel, Thomas

Dutczak, Steven

Fronk, Robert

Hodges, Jane

English

Embry-Riddle Aeronautical University

The Space Congress® Proceedings 1996 (33rd) America's Space Program -What's Ahead? Apr 24th, 2:00 PM - 5:00 PM Paper Session II-C - Measuring the Concerns of Teachers: A Possible Means for Evaluating the Efficacy of Kennedy Space Center Teacher Enhancement Workshops Thomas Dreschel The Dynamac Corporation Jane Hodges NASA, Kennedy Space Center Steven Dutczak NASA, Kennedy Space Center Robert Fronk Florida Institute of Technology Follow this and additional works at: https://commons.erau.edu/space-congress-proceedings Scholarly Commons Citation Dreschel, Thomas; Hodges, Jane; Dutczak, Steven; and Fronk, Robert, "Paper Session II-C - Measuring the Concerns of Teachers: A Possible Means for Evaluating the Efficacy of Kennedy Space Center Teacher Enhancement Workshops" (1996). The Space Congress® Proceedings. 22. https://commons.erau.edu/space-congress-proceedings/proceedings-1996-33rd/april-24-1996/22 This Event is brought to you for free and open access by the Conferences at Scholarly Commons. It has been accepted for inclusion in The Space Congress® Proceedings by an authorized administrator of Scholarly Commons. For more information, please contact commons@erau.edu. Measuring the Concerns of Teachers: A Possible Means for Evaluating theEfficacy of Kennedy Space Center Teacher Enhancement Workshops.Thomas Dreschel*, Jane Hodges**, Steven Dutczak** and Robert Fronk****The Dynamac Corporation, Mail Code: DYN-1,**NASA, Mail Code: PA-ESB, Kennedy Space Center, FL 32899 and***Florida Institute of Technology, Melbourne, FL 32901EXECUTIVE SUMMARYIn-service teacher enhancement workshops are held as part of theNASA teacher preparation and enhancement programs. For theseworkshops, teachers are brought to NASA field centers during summermonths for training in various aerospace related areas. The NationalResearch Council, Committee on NASA Education Program Outcomes(NEPO) has recommended comprehensive data collection to show programefficacy. Teacher workshops are a part of the nearly 300 NASA educationalprograms for students and teachers from pre-school through post-graduate,and NEPO has made specific recommendations for the evaluation of theteacher workshops. In accordance with one recommendation, aquestionnaire was administered during four of the summer 1995 teacherworkshops to measure participant concerns about using workshop material(space science) in their classes. Participant responses to the questions reflectchanges in concerns relative to using workshop materials in teaching.(Work performed under NASA contract: NAS1O-1218O)INTRODUCTIONNASA Education ProgramsThere is a persistent national concern for education in science andengineering relative to preparing and motivating young men and women forcareers in these areas (American Association for the Advancement of Science[AAAS], 1990). The National Aeronautics and Space Administration (NASA)administers educational programs encompassing all age groups frompreschool through graduate school and post-doctoral research (NASA, 1993).The National Research Council (NRC) Committee on NASA EducationProgram Outcomes (NEPO) reports that almost 300 of these programs exist(NRC, 1994).Inservice teacher workshops are held as part of Teacher Preparationand Enhancement Programs. Teachers are brought to NASA field centersduring the summer months for training in various aerospace related areas.Each summer, since 1984, teacher workshops have been hosted by the NASAPublic Affairs Office, Education Services Branch (PA-ESB), at Kennedy SpaceCenter (KSC). As with other NASA programs, the effectiveness of educationalprograms is of concern due to the time and funds involved.6-1The National Research Council (NRC) Committee on NASAEducation Program Outcomes (NEPO) was formed to evaluate the many anddiverse NASA educational programs (NRC, 1994). The NEPO was tasked toassist NASA in defining goals for their education programs. Theyrecommended comprehensive data collection related to: 1) the teacher’sscientific interest, attitudes, and awareness; 2) their sense of self-efficacy andempowerment and their associated perception of constraints in the workenvironment and; 3) their pedagogical beliefs and practice (NRC, 1994).Teacher Concerns about ChangeTeacher concerns have become of particular interest because of theirimpact on the classroom. The Concerns Based Adoption Model (CBAM) wasdeveloped to describe the effect of educational change and the use ofinnovations in teaching. Seven stages of concern make up this model (Hall,George and Rutherford 1977). These stages of concern are listed with theNEPO goal or outcome indicators that they relate to in Table 1. Hall, George,and Rutherford (1974) developed the Stages of Concern Questionnaire (SoCQ)which is intended to provide a framework and diagnostic tools for thedevelopment and enhancement of inservice training. This has been found tobe valuable in evaluating curriculum and staff development activities(O’Brien, 1992). The stage of concern varies relative to exposure (O’Brien,1987), peaking at the level of concern that is exhibited most strongly. This canbe used to predict use of the innovation, which in this case is the use of spacescience in teaching. Prior research indicates that this can be done with a betterthan 900/0 accuracy (Rutherford and George, 1978).Lombard, Konicek and Schultz (1985) used the SoCQ with secondaryscience teachers. They observed a shift during the workshop from concernsabout awareness to concerns about collaboration. In a study with teachers ofvarious grade levels, O’Brien (1987) found that the workshop was successfulin advancing the level of teachers’ concerns about chemical demonstrations.Showers, Joyce, and Bennett (1987) evaluated over 200 research studies andfound that teacher characteristics such as self-esteem, enthusiasm, andflexibility contribute significantly to teacher effectiveness.The studies discussed above emphasize the importance of teacherbehaviors when presented with educational innovation. Facilitating changein these behaviors is important in successful inservice training. This entailshaving an impact on concerns pertaining to educational innovations.RESEARCH QUESTIONSTwo research questions were investigated in this study:1) Is there a difference in the level and degree of concern for using SpaceScience in teaching before and after participation in the workshops?2) Are differences dependent on workshop length?6-2Table 1. Stages of Concern and NASA Education Programs Outcomes (NEPO)Committee-Indicators and GoalsStage of Concern0) Awareness1) Informational2) Personal3) Management4) Consequence5) Collaboration6) RefocusingNEPO IndicatorsTeacher awarenessparticipation incontinuing ed. activities.Change in teachermath and scienceknowledge.Changes in teacherattitudes andpractice.Changes in teacherattitudes andpractice.(Lessonmodification orenhancement)Increasing studentinterest andachievement in mathand science asperceived by theteacher.“Multiplier” effecton other teachers.Changes in teacherattitude and practice(Lesson Plan redesign).NEPO GoalsDissemination ofinformation.Increasing teachers’content knowledgein math and science.Increasing teachers’pedagogicalknowledge inmath and science.Increasing teachers’capability to designand implement morestimulating and engaginglessons andexperiences.Increase studentinterest and achievementin math and science.Extend benefitsbeyond participantsto other teachers.Increasing teachers’capability to designand implement morestimulating andengaging lessons andexperiences.6-3METHODS AND MATERIALSThe target population is southeastern United States teachers andaccessible population was teachers involved in NASA-Kennedy Space Centerteacher workshops. Teachers participated in summer 1995 NASA-KennedySpace Center workshops of varying lengths. The workshop samples wereteachers completing the instruments while attending: 1) The SummerTeacher Enhancement Program (Step) with 25 kindergarten through highschool teachers for four weeks; 2) The Brevard Summer Science andMathematics Institute (SS&M), consisting of 17 Brevard County teachers witha length of three weeks; 3) The NASA Educators Workshop for ElementarySchool Teachers (Newest) consisting of 16 elementary school teachers fromacross the U. S. (two weeks) and; 4) a University of South Florida (USF)teachers workshop with 19 teachers for one week.The Stages of Concern questionnaire or SoCQ (Hall et al., 1974) is aLikert-type instrument with 35 statements indicating the respondents feelingstoward an educational innovation. The respondents indicate their agreementwith each statement by designating their feelings according to the scale below:Irrelevant /Not true of me /Somewhat true of me /Very true of me now.Five randomly-distributed questions of the SoCQ pertain to each of theseven Stages of Concern (Table 1). The total of these five questions is a scoreused to evaluate teacher attitude toward using an educational innovation. Inthis case, the innovation is the use of space science in their classroomteaching.Differences between the pretest and posttest from the 1995 workshopswere used to identify changes that occurred during the workshop. Changes inthe level of concern over using workshop materials indicate areas whereprogram content and instructional approaches facilitate changes in teacherconcerns in these categories. The scores for each stage were calculated andpaired t-tests (SPSS, 1993) run to find significant differences between pre-workshop and post-workshop responses. The scores were also averaged foreach workshop and Stages of Concern plots were generated for analysisaccording to the recommendations of Hall, George and Rutherford (1977).RESULTS AND DISCUSSIONThe Stages of Concern mean values for each workshop, pre- and post-,were converted to percentile values relative to a reference population asrecommended by Hall, George and Rutherford (1977) prior to the creation ofthe profiles presented in Figure 1. These profiles are characteristic of non-users, with relatively high stages 0, 1, 2 and 3 (Hall, George and Rutherford,1977). However, they graphically demonstrate changes in concerns as did thet-test. Particularly, data from the Step workshop showed a marked change in6-4Stages 0, 1, and 2, indicating significantly reduced concerns toward awareness,understanding, and personal confidence toward using space science in theirclassrooms.Figure 1. Stages of Concern Profiles for the Four 1995 Teacher Workshops.Inferential statistics were performed on the raw scores taken from theSoCQ, totaled for each Stage of Concern. A Kolmogorov-Smirnov (Lilliefors)test (SPSS, 1993) for normal distribution was performed on the total samplefor each Stage of Concern and indicated a normal sample distribution for eachStage. Because of the pre-post sampling in this study, paired t-tests (SPSS,1993) were performed to test for significance. An alpha level of 0.05 waschosen as a small to moderate effect was expected. The results of the paired t-test for each Stage of Concern by workshop are presented in Table 2.6-5Significant changes were indicated for Stage 0 (awareness) from the oneweek (USF) and the four-week (Step) workshops: Changes for Stage 1(informational) and Stage 2 (personal) were indicated for the Step-workshop.There were no significant changes in management concerns (Stage 3)observed. The three-week workshop (SS&M) yielded a change indicated forStage 4 (consequence) concerns and the two-week, elementary school(Newest) workshop, a change in Stage 5 (collaboration) concerns. The one-week (USF) workshop participants also showed a change in concernspertaining to refocusing (Stage 6). When the data were pooled and the groupviewed as a whole, changes in responses relative to Stage 0, Stage 4, Stage 5,and Stage 6 were observed. Stage 0 and Stage 1 relate to awareness andunderstanding of the materials from the workshop (printed and presentedmaterials on space science).The results of this study indicate that there was an effect of the lengthof the workshop on the understanding of and confidence in using theworkshop materials. When the samples were pooled and participation in theworkshops was examined (total), the participant’s level of concern decreasedrelative to awareness of the material and increased relative to theconsequences of utilizing the workshop materials, forming collaborativerelationships relative to the materials, and modifying (refocusing) theircurricula relative to the workshop materials. This shift from concerns overawareness and understanding to concerns about collaboration fromworkshops has also been observed by Lombard , Konicek and Schultz (1985).O’Brien (1987) observed a similar shift in concerns during a chemistrydemonstrations workshop that he studied.6-6CONCLUSIONS AND RECOMMENDATIONSThe Concerns-Based Adoption Model (CBAM) was used in this studyto answer the two research questions. Differences were demonstrated in thelevels of the seven Stages of Concern before and after participation.Differences were also demonstrated between workshop lengths as indicated bydifferent changes in the levels of the Stages of Concern in each workshop.The change in responses indicate that a four-week workshop facilitatedgreater change in the levels of the Stages of Concern than one-, two-, andthree-week workshops, particularly in the areas that would indicateworkshop success (awareness, informational, and personal). The shifts in theStages of Concern observed from the pooled data set are consistent withobservations made by other researchers.The current study demonstrates the utility of the CBAM in evaluatingthe NASA teacher workshops held at Kennedy Space Center. It isrecommended that this tool continue to be utilized to evaluate theseprograms. The increase in concerns over consequence, collaboration andrefocusing indicate that greater emphasis should be placed on facilitatingteacher networking and on demonstrating how the materials might beincorporated into lesson plans. Activities such as lesson plan developmentnear the end of the workshop, with collaboration between participants shouldbe given greater emphasis. Follow-up inservice training or participation in asecond-year workshop may help to address these concerns.LIMITATIONSThis study was pre-experimental in design. The lack of randomselection and random assignment limits the ecological validity (application toother teacher populations). The teachers participating in these workshopshave already shown a high level of involvement in their work by their desireto spend vacation time in this form of training. The conclusions from thisstudy must be limited to teachers with this characteristic. Although thepooled data were normally distributed for each of the Stages of Concern andthe material covered was similar, the workshops were administered bydifferent persons and covered different time frames. Thus, the amount ofmaterial and the emphasis on the various workshop materials differed. Thisfact may limit the degree to which comparisons are made between thedifferent workshops.ACKNOWLEDGMENTSThe authors would like to recognize Dr. Paul Schmalzer for hiscontribution to the data analysis and the staff of the NASA Biomedical Officeand the Dynamac Corporation at the Kennedy Space Center for critical reviewof the manuscript.6-7LITERATURE CITEDAmerican Association for the Advancement of Science. (1990). Science for allAmericans, Project 2061. New York, NY: Oxford University Press.Hall, G. E., George, A. A. & Rutherford, W. L. (1977). Measuring stages ofconcern about the innovation: A manual for the use of the SoCquestionnaire. Research and Development Center for TeacherEducation, The University of Texas at Austin.Lombard, A. S., Konicek, R. D., & Schultz, K. (1985). Description andevaluation of an inservice model for implementation of a learningcycle approach to the secondary science classroom. Science Education,69(4): 491-500.NASA. (1993). NASA’s education program, NASA report #EP-297.Washington, D. C.: National Aeronautics and Space Administration,Office of Human Resources and Education, Education Division.National Research Council (NRC). (1994). NASA’s Education Programs:Defining Goals, Assessing Outcomes. National Research Council,Committee on NASA Education Program Outcomes, Studies andSurveys Unit. National Academy Press, Washington, D. C.O’Brien, T. P. (1987). A concerns-based field study of a chemicaldemonstration inservice program. Dissertation, University ofMaryland (0117), 225 pp.O’Brien, T. (1992). Science inservice workshops that work for elementaryteachers. School Science and Mathematics, 92(8), 422-426.Rutherford, W. L., & George, A. A. (1978). Affective and behavioral changein individuals involved in innovation implementation. (Report No.3046 Austin: The University of Texas, Research and DevelopmentCenter for Teacher Education. (ERIC Document Reproduction ServiceNo. ED 158 408).SPSS Inc. (1993). Statistical Package for the Social Sciences. SPSS forWindows, Base System User’s Guide, Release 6.0. Chicago, Illinois.Showers, B., Joyce, B., & Bennett, B. (1987). Synthesis of research on staffdevelopment: A framework for future study and a state-of-the-artanalysis. Educational Leadership, 45(3), 77-87.6-8

Paper Session II-C - Measuring the Concerns of Teachers: A Possible Means for Evaluating the Efficacy of Kennedy Space Center Teacher Enhancement Workshops

https://commons.erau.edu/cgi/viewcontent.cgi?article=1562&amp;context=space-congress-proceedings

Paper Session II-C - Measuring the Concerns of Teachers: A Possible Means for Evaluating the Efficacy of Kennedy Space Center Teacher Enhancement Workshops

Abstract

Similar works

Full text

Available Versions

Embry-Riddle Aeronautical University