Drane, J. W.

Spradlin, B. C.

English

Duquesne University: Digital Commons

Duquesne Law Review Volume 17 Number 3 Article 10 1978 Additional Comments on the Application of Statistical Analysis to Differential Pass-Fail Rates in Employment Testing B. C. Spradlin J. W. Drane Follow this and additional works at: https://dsc.duq.edu/dlr  Part of the Law Commons Recommended Citation B. C. Spradlin & J. W. Drane, Additional Comments on the Application of Statistical Analysis to Differential Pass-Fail Rates in Employment Testing, 17 Duq. L. Rev. 777 (1978). Available at: https://dsc.duq.edu/dlr/vol17/iss3/10 This Article is brought to you for free and open access by Duquesne Scholarship Collection. It has been accepted for inclusion in Duquesne Law Review by an authorized editor of Duquesne Scholarship Collection. Additional Comments on the Application ofStatistical Analysis to Differential Pass-Fail Rates inEmployment TestingB. C. Spradlin*andJ. W. Drane**An article' in a recent issue of The Harvard Law Review compareda statistical technique for assessing the significance of differences inpass rates of applicant groups on employment tests with the "80%rule" adopted as part of the Federal Executive Agency Guidelines.The author demonstrated that the two methods may lead to con-flicting results when applied to the test pass rates of applicantgroups to determine if the tests have a disproportionate impact onone group.The advantages of the statistical technique selected-a test of thedifference between independent proportions2-when compared tothe procedure adopted by Federal Guidelines were appropriatelynoted. The principal advantages were that the statistical test takesinto account both sample size and the magnitude of the differencein the sample proportions. These are important considerations, andProfessor Shoben presents a well-written advocacy for using statisti-cal analysis for a more reliable inference concerning whether anemployment test has an adverse impact on a protected class em-ployee under Title V113 statutes.However, the opinion expressed in this note is that the commentleaves two points that need clarification. The first is that statisticaltests are, by and large, general in applicability and are not uniqueto any one setting. The second is that conflicting inferences about* Ph. D; Vice President, Criterion Analysis, Inc., Dallas, Texas.** P.E., Ph. D; Associate Professor, Department of Statistics, Southern Methodist Univer-sity, Dallas, Texas.1. Shoben, Differential Pass-Fail Rates in Employment Testing: Statistical Proof UnderTitle VII, 91 HARv. L. REv. 793 (1978).2. Eberhardt & Fligner, A Comparison of Two Tests for Equality of Two Proportions, 31Am. STATISTICIAN 151 (1977); P. HoEL, INTRODUCTION TO MATHEMATICAL STATISTICS § 9.3 (1971);W. MENDENHALL, INTRODUCTION To PRoBABILITY AND STATISTICS 202 (1975).3. 42 U.S.C. §§ 2000e(1)-(17) (1970 & Supp. V 1975).777Duquesne Law Review Vol. 17: 3-4adverse impact on a given employee group may result when apply-ing different methods of statistical analysis, in which case no infer-ence should be made.I. GENERAL APPLICATION OF SUPREME COURT CRITERION FORSTATISTICAL SIGNIFICANCEThe first point concerns the applicability of the statistical tech-nique employed by the Supreme Court in the cases of Castaneda v.Partida4 and Hazelwood School District v. United States5 to casesconcerning the significance of differences in pass-fail rates in em-ployment tests. It is suggested here that the opinion expressed bythe Court in Castaneda and Hazelwood implies a much broaderconcept of statistical analysis than the particular technique used inthose cases. Although the sample statistics calculated in the exam-ples of Professor Shoben's comment' are differences in proportionsrather than proportions, the wording of the statistical inferencescited by the Supreme Court in Castaneda and Hazelwood applydirectly to those examples.7 A sample difference statistic as advo-cated by Professor Shoben is different from the sample statisticscited by the Court in the cases mentioned above in form only. Theprocess of forming statistical inferences used by the Court may be4. Castaneda v. Partida, 430 U.S. 482 (1977). The Court states: "As a general rule for suchlarge samples [n=870 in this case], if the difference between the expected value and theobserved number is greater than two or three standard deviations, then the hypothesis thatthe jury drawing was random would be suspect to a social scientist." Id. at 496 n.17.5. Hazelwood School District v. United States, 433 U.S. 299 (1977). The Court states:"The Court in Casteneda noted that 'as a general rule for such large samples, if the differencebetween the expected value and the observed number is greater than two or three standarddeviations' then the hypothesis that teachers were hired without regard to race would besuspect." Id. at 309 n.14.6. Shoben, note 1 supra.7. Id. at 807. In id. at 807 n.53, under the null hypothesis, the expected value of the sampledifference statistic Pw (sam)-Pb (sam) is 0. For this example, the calculated z is the numberof stahdard deviations that the sample statistic Pw (sam)-Pb (sam) lies from its expectedvalue under the null hypothesis. Therefore, in the wording of the Court in Castaneda andHazelwood, "Since the difference between the expected value (0) and the observed value (.1)is less than two or three standard deviations the hypothesis of no difference in impact of theemployee tests could not be rejected." See notes 4 & 5 supra. In Shoben, supra note 1, at 807n.55, according to the Supreme Court criterion for significance, since the difference betweenthe expected value of the statistic and the observed or sample value of the statistic is 3.13standard deviations, the hypothesis of no difference in the impact of the employment testson different employee groups is suspect.1978-79 Statistical Analysisapplied generally and so may the criteria for rejection of the nullhypothesis. In fact, reduction in sample size does not restrict thegenerality of the process of developing inferences. In cases when thesample size does not comply with the Court's definition of "large","a translation can be made so that the number of standard deviationsbetween the observed statistic and its expected value can be statedin terms that reflect a large sample equivalent.This translation is called a normal equivalent deviate. 0 The pro-cedure allows the results from any small sample statistical test ofsignificance (for example a t test)" to be stated in terms of a largesample standard normal deviate (a z statistic). For example, sup-pose it is of interest to determine whether the differential in averagesalaries for men and women is statistically significant for a givensmall group of employees. Further, suppose there are three male (n=3) and four female (n =4) employees in the group and a t statisticreflecting sample salary differences is calculated 3 to be equal to8. See notes 4 & 5 supra.9. See 430 U.S. at 496 n.17.10. D. FINNEY, STATISTICAL METHOD IN BIOLOGICAL ASSAY 443, 452 (2d ed. 1964).11. See authorities cited in note 2 supra.12. See Shoben, supra note 1.13. For this calculation, the hypothesis to be tested is Ho:M, =M, (i.e., there is nodifference in the means of male and female salaries) or stated alternatively H,:M-M, =0.WhereM, =universe mean salary for men, andM =universe mean salary for women. The tstatistic is used to test the hypothesis. To calculate t, we need X, =sample mean salary formen and R, =sample mean salary for women and the standard error of the difference X, -X,. The degrees of freedom are given by n,+n, -2=5, since n, =3 and n,.=4. The t statistichas the form:(n -1)S +(n -1)S1 1 2 2 ( 1 - )t (X -X)- n +n - 2  n n1 2 1 2where S' and S,' are the sample variances for men and women respectively. The denominatoris the standard error of X,-X 2 •If we assume that a t calculated in this manner is equal to 2.015, we then find from TableIV of R. FISHER, STATISTICAL MEMODS FOR REsEAacH WoRmRS that the level of significance is= .1. In other words, we would expect to see a difference in men and women's salaries as largeor larger than the one we observe about .1 of the time in repeated sampling where n,=3, andn, =4.14. The translation from t=2.015 to a standard normal deviate z=1.64 can be shown bythe following diagram:Duquesne Law Review2,015. This small t value can be translated" directly to a standardnormal deviate (z statistic) equal to 1.64. The resulting value canthen be analyzed in the context of the Court's definition of signifi-cance for large samples. In this example, the hypothesis of no differ-ence in male and female salaries could not be rejected according tothe Court's criterion. A similar procedure can be followed allowingthe results of any statistical test of significance to be placed withinthe context of the Court's definition of significance in Hazelwoodand Castaneda.If. CONFLICTING RESuLTS FROM APPLICATION OF DIFFERENTSTATISTICAL TESTSThe second point for clarification concerns the fact that conflict-ing inferences about adverse impact on a given employee group mayoccur when different methods of statistical analysis are used. Thiscan happen using the statistical approach on the same data. Thispoint can be illustrated with the following hypothetical set of data.z=1.64 t=2.015 z=1.64In other words Za/2= 1.64 is the standard normal deviate that implies the same level ofsignificance as the t value of 2.015 from the given small sample.15. Shoben, supra note 1, at 804.Vol. 17: 3-41978-79 Statistical Analysis 781Suppose five hundred (500) white employees take an employmenttest and nineteen (19) are successful. One hundred fifty (150) blackemployees take the same test and one (1) is successful. These resultscan be displayed in a two-way table as shown below.Pass Fail TotalWhites 19 481 500 (Number of Whites N )WBlacks 1 149 150 (Number of Blacks N )B20 630 650The null hypothesis to be tested is that there is no difference in thepass rates for black and white employees (i.e., H :P (pop)-P(pop) =0). The statistical test'" suggested in Professor Shoben'scomment applies to these data as follows:(a.) H :P (pop)-P (pop)=O0 W B(b). P (sample)=19/500 = .0380; P (sample) =1/150 = .00667W B(c.) Calculate the overall proportion of people in both groups who passed thetest. For all 650,19+1650 = .0308(d.) Compute the overall proportion of people in both groups who did not passthe test. For all 650 employees,1-.0308=.9692(e.) Multiply the proportions from steps c and d.Prod.= (.0308) (.9692) =.02985(f.) Apply the following formula to calculate the standard error:Prod. Prod.Standard Error = NW Rwhere N and N are the number of whites and the numberW Bof blacks respectively in the sample. Therefore,.02985 .02985 = .0161.Standard Error = 500 + 150Calculate the z statistic according to the following fomula:P (sample)-P (sample)- W B .038-00667 - 1.95standard error .016116. Id.Duquesne Law ReviewThus, if the level of significance chosen for this test were a= .05, thecritical value for z is 1.96, and the null hypothesis of no differencein pass rates could not be rejected.Another acceptable statistical procedure for testing the null hy-pothesis of no difference in pass rates between white and blackemployees is the Likelihood Ratio Chi-Square Test.'7 When thisprocedure is applied to the data in the contingency table shownabove a statistic, G, is calculated.2G - 2[19 log(19) + 481 log(481) + 1 log(l) + 149 log(149)- 500 log(500) - 150 log(150) - 20 log(20) 630 log(630)+ 650 log(650)]= 5.078wherein log(X) is the natural logarithm of X. GI is to be comparedto tabled values of chi-square. Assuming a significance level of .05the critical value of chi-square with one degree of freedom is X =3.841 s while the calculated value of G is 5.071. G2 exceeds 3.841.Therefore, the null hypothesis of no difference in pass rates between17. See Eberhardt & Fligner, supra note 2, at 151; S. Wilks, The Large-Sample Distribu-tion of The Likelihood Raio for Testing Composite Hypothesis, 9 ANNALS OF MATHEMATICALSTATISTICS 60-62 (1938); S. KULLBACK, INFORMATION THEORY AND STATISTICS (1959); M. BISHOP,S. FIENBERG & P. HOLLAND, DIscRErE MULTIVAIATE ANALYSIS (1976).The Likelihood Ration Chi-Square, G, for contingency or cross tabulated tables has thegeneral form,G2= nij log[nij/(ni.n.. i/n..)]with degrees of freedom equal (r-1)(c-1), wherein r = number of rows and c = number ofcolumns; and log (.) is the natural logarithm of whatever is in parentheses.For a two by two contingency table,n n n11 12 1.n n n21 22 2.n. n. n.where n i. represents a row total, nj represents a column total and n.. represents the over-all total, the generalized form sho%A in (a.) supra may be reduced to.G= 2 [n, log(n,,) + n, log(n,) + n,, log(n,,) + n22 log(n22)- n,. log(n,.) - n2. log(n2.) - n., log(n.,) - n., log (n.2)+ n log(n.)]with degrees.of freedom given by,(2-1)(2-1) = 1.18. F. ROHLF & R. SOKOL, STATISTICAL TABLES, Table R (1969).Vol. 17: 3-4Statistical Analysiswhite and black employees must be rejected. This result is in con-trast to the conclusion reached when the same hypothesis about thesame data was tested through the calculation of a z statistic. Fur-ther, it sometimes occurs that a hypothesis which cannot be rejectedby the Likelihood Ratio Chi-Square may be rejected by the z test.The contradiction which may occur in test results does not alwayswork in the same direction for these same two tests.The point made by Professor Shoben ' is that the application ofFederal Executive Agency Guidelines for assessing significance indifferences in pass rates of employee groups on employee tests islacking as a reliable approach in determining adverse impact. Thecomment demonstrated that a statistical procedure for testing hy-pothesis is a more reliable inferential process. Further demonstratedwas the fact that the statistical test (a z test) may produce resultswhich contradict those produced by the Federal Guidelines whenapplied to the same data.The point made in this note is that the same kind of contradictionmay occur between the results of two equally acceptable statisticaltests. The authors of this comment suggest that when this happens,no inference of adverse impact can be made.19. Shoben, note 1 supra.1978-79

Additional Comments on the Application of Statistical Analysis to Differential Pass-Fail Rates in Employment Testing

https://dsc.duq.edu/cgi/viewcontent.cgi?article=2137&context=dlr

Additional Comments on the Application of Statistical Analysis to Differential Pass-Fail Rates in Employment Testing

Abstract

Similar works

Full text

Available Versions

Duquesne University: Digital Commons