Houston, Samuel

Kayne, Jon

Shannon, Lee

English

Reynolds, William M.

UND Scholarly Commons (University of North Dakota)

Teaching and Learning: The Journal of Natural Inquiry & Reflective Practice Volume 5 Issue 1 Article 5 4-1980 Normal Curve Equivalents in Bilingual Program Evaluation Samuel Houston Jon Kayne Lee Shannon Follow this and additional works at: https://commons.und.edu/tl-nirp-journal  Part of the Scholarship of Teaching and Learning Commons Recommended Citation Houston, Samuel; Kayne, Jon; and Shannon, Lee (1980) "Normal Curve Equivalents in Bilingual Program Evaluation," Teaching and Learning: The Journal of Natural Inquiry & Reflective Practice: Vol. 5 : Iss. 1 , Article 5. Available at: https://commons.und.edu/tl-nirp-journal/vol5/iss1/5 This Article is brought to you for free and open access by UND Scholarly Commons. It has been accepted for inclusion in Teaching and Learning: The Journal of Natural Inquiry & Reflective Practice by an authorized editor of UND Scholarly Commons. For more information, please contact und.commons@library.und.edu. Normal Curve Equivalents in Bilingual Program Evaluation Samuel Houston University of Northern Colorado Jon Kayne Hillsdale College Lee Shannon Regis College Introduction The formative evaluation of bilingual programs poses special problems for the outside evaluator . What are the criterion variables to be analyzed--project objectives, measures in the cognitive domain, measures in the affective area? Are the subjects assigned on a random or matched basis to the experi-mental and control groups? What about the use of grade--equivalent scores, percentile ranks and T-scores in the data analysis? Why Normal Curve Equivalents (NCEs)? One of the most popular and widely used measures of achievement are grade-equivalent scores , Not only do they tend to be grossly inadequate because of the manner in which they are constructed, they are scaled in such a way that the usual methods of statistical averaging are not legitimate , Percentile ranks are not unlike grade-equivalent scores in that statistical averaging should also be avoided because they too do not form an equal-interval scale . While T-scores (standard scores with a mean of SO and a standard deviation of 10) possess the required features for the usual statistical operations, it is felt they lack meaningfulness for the usual consumers of educational research. 39 Normal Curve Equivalents (NCEs) are normalized scores with a mean of SO and a standard deviation of 21.06; these values were selected because they pro-duce an exact match between NCEs of one and 99 per-centile ranks of one and 99 . Thus, percentile ranks and NCEs have the same range and the same midpoin t but NCEs have an equal-interval scale. A gain of ten NCEs corresponds exactly to the same amount of student performance achievement at the extreme low end of the achievement distribution as it does for performers in the middle range. An important feature of the use of NCEs is that the experimental group serves as its own control group. Its later success in NCE units is measured against the baseline data gathered initially which also are expressed in NCE units. In the evaluation of a research project in which NCEs are utilized, all NCE gains greater than zero are considered useful o Whenever a particular program shows that an NCE is greater than zero, it is assumed that the participating s t udents profited from the project. While it is not possible to designat e any specific NCE gain as a minimal standard for project success, nevertheless a.S-NCE gain produced in an experimental group of 100 pupils might be considered as good as a 10-NCE gain produced in a group of SO pupils, assuming the same dollars were spent (Tall -madge and Wood, 1976) in both studies . Grade Two Bilingual Example An example of the NCE approach is presented for Grade 2 Bilingual Project students selected from Weld County School District RE-SJ in Colorado . The data are selected from a 1979 evaluation report of the Bilingual Project at the primary level which also included results from the Kindergarten, Grade 1 and Grade 3. The Bilingual Project objectives and the extent to which they were satisfied are presented in Table 1. The six Bilingual Project objectives inc l uded 40 ..,. f--' TABLE 1 GRADE TWO BILINGUAL DATA (N=44) TEST 1 TEST 2 TEST 3 TEST 4 TEST 5 TEST 6 Oral Identi- Part I Part II PART II I Total Language fication Reading Reading Reading Points Highest Pre 80 20 25 10 10 145 Possible Points Post 80 20 25 10 10 145 Mean (X) Pre 10.2 16.1 19.2 503 5.9 55.4 Standard Deviation (SD) Pre 9.2 3.2 4.3 3.1 3.1 16.5 Mean (X) Post 36.7 18.3 22.2 8.2 8.5 95.3 Standard Deviation (SD) Post 25.5 2.3 3.5 2.2 1. 8 32.0 t-test on Gains Mean Gain (D) 29.4 2.1 2.8 2.8 2o9 40.5 t-statistic 8.5* 5.7* 4.7* 6.4* 5.4* 10.8* Summary Statement MET MET MET MET MET MET about Objective *Results oft-test for Dependent Samples Significant at 0.01 Level. I-'• Q rt HQ ;:i ;:i I H >-j p.. rt PJ I-'• (1) (1) ::,:, f--' ()PJtll(l) PJ()rtPJt:""' rt ::r p.. PJ (1) H) I-'· ;:i 00;:i(IQ OH)>-i(IQC f--' .. PJ (1) rt p.. (IQ PJ ::r (1) "O (1) >-j(l)'rjPJ .. f--' (1) >-j '<tn;:irtH I-'· p.. p.. rtX(l)H(l) ::,-- ;:i H ;:i PJOrtHrt rt o" I-'• w . tn ::,:, H) PJ (1) PJ (1) I-'· f--'0:lPJO f--' rt 'tj p.. PJ I-'• f--' I-'• rt "d < (1) ;:i I-' · >-iCl>tn(IQO 0 tn ;:i C'Q O ~PJ'->-j PJ ;:i PJ tn p.. "O :l ::,:, PJ (1) PJ ...,J >-j Qtn"dOrt o" .::: "d rt w . f--' f--' PJ H (1) rt I-'· f--' () tn (1) ::,:, rt p.. "O (1) I-'· H) 0 PJ < >-j H) I-' • p.. Cl> 0 0 ;:i I-'· tn:l>-jrt;:i tn (IQ ~ ...,J z ... (1) PJ >-jo"II "O (1) f--' )> PJ (1) ..,. >-j :l ..,. rt (1) f--' rt .,. N TARLF. 2 GRADE TWO SELF OBSERVATION SCALES DATA (N=44) Mean Normal Curve Equivalent (NCE) Percentile Rank of NCE Score NCE Mean in 1978 (Grade 1) Percentile Rank of NCE Mean in 1978 Mean NCE Gain Percentile Rank Change on NCE Means Summary Statement about SOS Objective* SCALE I SCALE II SCALE III SCALE IV Self Social School Self Acceptance Maturity Affiliation Security 50.4 52 05 50.2 50.8 50.8 54.8 50.4 51. 6 45.5 40.5 38.1 47.9 41.5 32.6 28.6 46.0 +4 09 +12 o0 +12.1 +2.9 +9.3 +22.2 +21. 8 +5.6 MET MET MET MET *An SOS Objective is Met if there is a Positive Mean NCE Gain from the Prior Year. Pl 'd () S ;;:J :E: f-'• ;:l C/l 0 () 0 0 (1) Pl Pl ;:l O () H) rtVl;:lPJrt'<Vl>-iPl (1) f-'• () VI N rt S I-' rt >-irt(l)S::(1)0>-i(l)(l)::," f-'· f-'· 'd >-i ;:, ::,- c p. VI ro ""'I VI < rt (1) S f-'• S .. ::," rt(l)• Vl(l))-'(1) ,......_Vl(l) f-'• '< p.. ;:, (1) C/l () ns H)(l)>-irtSOOH) Pl tT1 0 >-i (1) 'd C/l >-i f-'· o ;:l Pl c .. ;:l f-'· f-'• '--' ro >-i H) ;:, () >-i ;:, >-i • VI VI (1) ::," I-' 'd f-'· rt rt>-i P.I.O(l)rt() H) ::," o f-'· --.1 >-i ::," Pl ""'I o c (1) :E: H)H)+'-0 (1) 1-'::,">-i VI ::," H)'--'(1) )-'(1) (1) VI f-'• rt (1) • f-'· Pl '< C) nn::,">-i <H"l C/l>-iO pi::,"(1)(1) roH)<OPlH) I-' ;:l...,J (l)p)C/)P,.. ro::,-H)rt::,-rtn ...... roz f-'· o (1) ::,- rt f-'· f-'· n H) (1Q C Pl (1) f-'· p. ;:, N tTl PJ::,">-iVl'dS<PlVI VI n 'df-iVl(l)rtrtO rt VI VI (1) f-'• (1) (1) >-i;:, Pl onnnSl-'P.P..c 'd >-iOPlrtPJ<(l) .. Srt'd >-i I-' VI >-i (1) s (1) ::," (1) 0(1)(1) '<VIPJS;:l(l)pi >-i VI VI O f-'· c rt >-i H) I-' ,......_ ;:, I-' C/l (1) ;:, 0- f-'• (1) C/l rt f-'· (1) p.. Pl (1) VI () < rt H) f-'• VI I-' S f-'• ::," (1) (1) 0 I H) f-'• (1) ;:, ...... f-'· I-' ;:, >-i p. Pl ;:, oq Pl I-' ;:l f-'• 0 0-P.O (1) S S ;:l cr'rt S(l)>-iH)f-i(l)(l)PJVI::," 01-'(1) pi;:lrt(l)(l) VI (1);:, rt Pl VI VI f-'• >-i rt P. ~ ::,- ;:, c f-'• o < Pl Vl(l)P,..>-jO;:lPJ;:l () f-'• f-'· ;:, Pl rt Pl ::," ;:, VI C/l ;:, Pl I-' f-'· ...... Pl (1)0 OQl-'1-'0'< >-iPll-'C/l '<;:lVI I H) rt f-'• ::," VI (1) Table 2 presents SOS data comparing the Grade 2 class in 1979 . This approach seems more feasible in an educational setting in which it is virtually impossible to conduct a true experimental study. The research strategy utilized is a quasi-experimental design in which the class serves as both the control and experimental group . An SOS objective is met if there is a positive mean gain in the NCE score from the prior year . An examination of Table 2 shows that all four SOS objectives (Self Acceptance, Social Maturity, School Affiliation and Self Security) were met, since there was a positive mean NCE gain for each objective. It is especially encouraging to note the magnitude of the gain in Scale II (Social Maturity) and Scale III (School Affiliation). While the NCE mean in 1978 was below the mean on all four scales for the students as first graders, their performance as second graders was above the mean of 50 on all four scales . In Table 3 are presented the Grade 2 Metro-politan Achievement Test for the Bilingual students (N = 44) both as second and first graders . Com-parative data are available in two areas - Reading and Mathematics. If the second grade Normal Curve Equivalent mean is significantly greater than their first grade mean , then that particular achievement test objective is said to be satisfied. It can be seen from an analysis of Table 3 that the Bilingual students showed a significant gain in Mathematics (objective met) but not in Reading (failed to meet objective) . 43 ~ ~ TABLE 3 GRADE TWO METROPOLITAN ACHIEVEMENT TEST COMPARISON (N=44) Bilingual Student Mean NCE Score (1979) Percentile Rank of Mean NCE 1979 Score Bilingual Student NCE Mean in 1978 (Grade 1) Percentile Rank of NCE Mean in 1978 (Grade 1) Mean NCE Gain Percentile Rank Change on NCE Mean Summary Statement about Metropolitan Achievement Test* TEST I Reading 51. 5 52.9 54.5 58.5 -3.0 -5.6 NOT MET TEST II Mathematics 49.9 49.8 45.0 40.6 +4.9 +9.2 MET *A Metropolitan Achievement Test Objective is met if there is a positive mean NCE gain from the prior year. Summary Comments and Observations Based on an analysis of the data, the following comments and suggestions are made: 1. The Bilingual Project staff should be en-couraged as to the extent to which their program objectives were met for Grade 2. All six of the objectives were satisfied. 2. An analysis of Self Observation Scales at the primary level for Grade 2 is very en-couraging as all four of the SOS objectives were met. 3. The Metropolitan Achievement Test data for Grade 2 resulted in positive gains in Mathematics but negative gains in Reading. 4. The use of a quasi-experimental design with percentile rank scores changed to Normal Curve Equivalents (NCEs) is recommended for subsequent educational research studies in which the use of randomly assigned control and experimental groups is not possible. 5. In summary, the Bilingual Project staff should be very encouraged with the results for Grade 2 as all six program objectives were met, all f~ur SOS objectives were satisfied and positive gains were detected in one out of the two Metropolitan Achieve-ment tests. 45 REFERENCES Stenner, A.M o and Katzenmeyer, W.G. Self Observation Scales (SOS): Primary Level, National Testing Service, Inc., Durham, North Carolina, 1974 . Tallmadge, G.K o and Wood, C.T o User's Guide: ESEA Title I Evaluation and Reporting System, RMC Research Corporation, Mountain View, California, 1976. 46 

Is Childhood Disappearing?

https://commons.und.edu/cgi/viewcontent.cgi?article=1085&context=tl-nirp-journal

Normal Curve Equivalents in Bilingual Program Evaluation

Abstract

Similar works

Full text

Available Versions

UND Scholarly Commons (University of North Dakota)