Author Institution: Departments of Chemistry, University of Iowa and Miami UniversityAn algorithm for evaluating student laboratory results is presented in which both proximity to the true result (accuracy) and the reproducibility of these results (preeision) can be evaluated. The usefulness of such a scoring device and its pedagogical advantages are presented. The results are described in terms of a computer program LAB GRADE which is available from the authors upon request

Gordon, Gilbert

Mottel, Edward A.

The Ohio Journal of Science

KnowledgeBank at OSU

AN ALGORITHM FOR THE COMPUTER EVALUATON OFQUANTITATIVE LABORATORY UNKNOWNS BASEDON ACCURACY AND PRECISION1' 2EDWARD A. MOTTEL AND GILBERT GORDON, Departments of Chemistry, University ofIowa, Iowa City, Iowa 52242 and Miami University, Oxford, Ohio 45056Abstract. An algorithm for evaluating student laboratory results is presented in whichboth proximity to the true result (accuracy) and the reproducibility of these results(preeision) can be evaluated. The usefulness of such a scoring device and its peda-gogical advantages are presented. The results are described in terms of a computerprogram LAB GRADE which is available from the authors upon request.OHIO J. SCI. 77(2): 63, 1977Grading of experimental results will al-ways be somewhat subjective in nature.In an attempt to make the evaluation ofquantitative laboratory results less sub-jective, a general algorithm and computerprogram has been developed which canbe used to evaluate and to assign a nu-merical grade for quantitative determi-nations. Unlike most evaluation pro-cedures, the score is based both on theproximity of the results to the acceptedvalue and the reproducibility of thoseresults.The results of student determinationsof laboratory unknowns, like all scientificmeasurements, should fulfill the conceptsof the scientific principle; that is, theyshould be close to the true or acceptedvalue (accuracy), and reproducible (pre-cision). In addition, it is also importantthat the person making the determina-tions, in this case a student, knows howto make any necessary numerical calcu-lations.In general, the student performs aspecific experiment (e.g. a chemical acid-base titration) and records the raw data.In most laboratory situations the studentcan make the required numerical calcula-tions, or alternatively, the raw data canManuscript received April 5, 1976 and inrevised form August 9, 1976 (#76-35).2This paper was presented in part at the 9thGreat Lakes ACS Regional Meeting, June 5,1975, St. Paul, and at the 1st Chemical Con-gress of the North American Continent, De-cember 2, 1975, Mexico City, Mexico.be entered into a data reduction program,and the values appropriate to the stu-dent's unknown experimental result canbe computer calculated. Many suchprograms have been reported in thechemical literature (Altenburg et al, 1968;Galyan and Ryan, 1972; Reiter andBudig, 1974). Comparison of the stu-dent calculated value to the computercalculated value has many pedagogicaladvantages and it helps to insure thatthe student has correctly made the re-quired calculations. In some cases, au-thors have reported programs in which astatistical analysis was applied to theraw data (Rosenstein and Smith, 1962;Smith et al, 1965; Wellman, 1970; Wise,.1972), but only a few authors have notedthe importance of precision to the sci-entific experiment (Wartell and Hurlbut,1972; Jones and Lytle, 1973; Klatt andSheafer, 1974, and none to our knowledgehave included this as an integral part ofthe grading scheme. Either the studentor computer calculated values can beused with the computer program re-ported here. This program called LAB-GRADE is written in FORTRAN IV andrequires 40K bytes in addition to approxi-mately 1000 bytes for every seven stu-dents. A class of 400 students requires96K bytes. A sample printout and pro-gram listing is available from the authors.The input for LAB GRADE consists ofcalculated values, rather than raw data,and thus LAB GRADE is not meant toreplace the computer calculation pro-6364 E. A. MOTTEL AND G. GORDON Vol. 77grams, but rather to be used in conjunc-tion with them. In view of the growingneed for carefully documented programs(Hoffman, 1975) which meet specificteaching needs, we have developed boththe basic algorithm and the LABGRADEprogram.STATISTICAL ANALYSISLABGRADE is an iterative programwhich calculates the arithmetic mean andthe standard deviation (a) for all studentgenerated results which are supplied asinput data. Any results which lie out-side of the 3o- range are excluded, since aresult which has only normal uncertain-ties will occur within 3o- of the arithmeticmean 99.7% of the time, and a newarithmetic mean and standard deviationare computed. This procedure is re-peated, until all remaining results fallwithin three standard deviations of thetrue value or the class mean. A graphof the reported concentrations of NaOH,which were obtained from a potassiumacid phthalate titration, plotted as afunction of the frequency with whichthose concentrations were reported (fig.1) shows that the student-reported val-ues for a large typical class have approxi-mately a normal distribution. Graphi-cally, in the output from the LAB-GRADE program, a normal curve withthe same standard deviation as that cal-culated from the student results is plottedcentering about the true value (if it isknown) or the class average if the ac-cepted value is not supplied as an inputvalue.Each determination the student re-ported is assigned a variable number ofpoints for accuracy and, if the studenthas made multiple determinations, a sep-arate number of points is assigned forprecision. The number of points as-signed to accuracy for each determina-tion corresponds to the height of the nor-mal curve at that given abscissa as isshown in figure 1. A linear scale isestablished with the maximum pointvalue occurring at the true or acceptedvalue (i.e. O.Ocr) and a zero point value isestablished at 3a. For example, the firstdetermination reported by Student A,CLASS AVERAGE (0.1981)Accuracy"TRUE" VALUE (0.1982) Points-HO8.4 points max 10•50 186 0188 0 190 0192 0.194 0196 0.198 0.200 0.202 0.204 0 206 0 208 0 210NaOH Concentration3CF0.0094—a—0.0031FIGURE 1. Assignment of Accuracy Points (Frequency of Values Reported as Function ofValues Reported). Solid line corresponds to a normal distribution centered onthe true value. Circles denote student results.No. 2 COMPUTER EVALUATION OF LAB UNKNOWNS 65corresponding to the result noted by thedotted line in figure 1 was a NaOH con-centration of 0.2000 M. Based on theclass results this determination is worth8.4 accuracy points out of a maximumscore of 10 points (chosen for this ex-ample) as can be seen in figure 1. Thesame student also reported two addi-tional determinations as shown in table 1.TABLE 1Reported results and evaluation of student AReportedConc.±SD0.2000 ±0.00100.2024 ±0.00140.2007 ±0.0003Mean 0.2010Skew Pts.Total Score(maximum ±50)AccuracyPoints*8.44.17.3PrecisionPoints**4.84.55.0TotalPoints13.28.612.35.039.1*10 points maximum per determination foraccuracy.**5 points maximum per determination forprecision.Precision points can only be assigned ifthe student has made multiple determi-nations, since the term precision impliesreproducibility of results. If the studenthas made multiple determinations on thesame system, the average value for thatstudent is calculated and the deviationof each result from the individual stu-dent's arithmetic mean is determined.The number of precision points assignedto a given result is obtained by comparingthe deviation of that result with thestandard deviation for the whole class.Graphically, this is equivalent to cen-tering the normal curve about the indi-vidual student's arithmetic mean, andobtaining the precision point value fromthe height of the curve corresponding toeach determination as illustrated in fig-ure 2. Again the maximum point valuewill occur when the deviation of an indi-vidual determination is zero, and no pointvalue will result if the deviation for thatdetermination is larger than 3a.An optional base or skewing score,which corresponds to the minimum num-ber of points credit the student receivesfor just doing the experiment may also beemployed. Thus, for three titrations theprevious student would obtain a score of39.1, if 5.0 points were assigned for sim-ply doing a minimum of three titrations.In other words, the skew score is used todifferentiate between the student thatcarried out the experiment and obtainedPrecisionPointsStudent A Average Value (0.2010)5-r0.186 0.188 0.190 0.192 0.194 0.196 0.198 0.200 0.202 0.204 0.206 0.208 0.210NaOH ConcentrationFIGURE 2. Assignment of Precision Points (Frequency of Values Reported as a Function ofValues Reported). Solid line corresponds to a normal distribution centered onStudent A's average value. Circles denote student results.66 E. A. MOTTEL AND G. GORDON Vol. 77poor results and the student who failedto report the required determinations.Prior to each computer run, the in-structor determines the maximum num-ber of accuracy points and the maximumnumber of precision points (assumingmore than one result is submitted by thestudent) for each determination reportedby the student. At this point, the in-structor also decides if the student shouldbe assigned any base points and the maxi-mum and/or minimum number of indi-vidual student results to be used incomputation of the student's total score.DIAGNOSTIC CAPABILITIESPerhaps one of the most helpful andoften overlooked advantages of a programsuch as LABGRADE is its diagnosticcapabilities. For example, if the classaverage differs significantly from thetrue or accepted value, then it is likelythere is: an error in the instructions, theprocedure or some systematic error; acontaminated reagent; or an incorrectlydetermined true value. In any of thesecases it would be important to be able toreconsider the details of the grading pro-cedure used in this experiment.A second diagnostic aid might cor-respond to the observation of a bimodaldistribution of values as is shown infigure 3. This could arise from a con-REPORTED VRLUESFIGURE 3. Expected Class Distributionand Example of Bimodal ClassDistribution.taminated reagent being used in one ormore of the laboratories. A possible solu-tion to this problem would be to gradeeach of the laboratories separately, usingthe class average for the true value inthose laboratories suspected of havingcontaminated reagents.Finally, it should be pointed out that,when the class is considered as a group,the class average comes remarkably closeto the accepted value once the sta-tistically insignificant determinations arediscarded. As can be seen in table 2,after seven iterations the class averagebecomes virtually identical to the truevalue. After 10 iterations, all remainingdeterminations fall within 3a- of the ac-cepted values and the class average dif-fers from the accepted value by only0.0001 molar units, or one part per twothousand (0.05%). Thus a large num-ber of class determinations, even from aclass that is just learning to titrate, canbe a sensitive method to check the truevalue which is often blindly acceptedwithout any additional evaluation. Fortitration experiments, our true valueswere taken as the average of three deter-minations made by one of the laboratoryteaching staff immediately preceding eachlaboratory. For other experiments suchas the determination of the equivalentweight of magnesium, the accepted valueswere obtained from handbooks or otherreference materials.TABLE 2Determinations used in successive iterations for atypical, set of student determinationsStandard True or Class No. Detn.Deviation Accepted Average StatisticallyValue Retained0.69730.03860.02160.01270.00880.00580.00400.00350.00330.00310.19820.19820.19820.19820.19820.19820.19820.19820.19820.19820.29550.20830.20400.20090.19920.19850.19830.19820.19810.1981249244236228219211207204202202In comparing the results from thesame experiment among different lab-oratories, we have observed that themagnitude of the standard deviation isconstant if no changes in procedure, re-actants or scoring formula are made. Forexample, a comparison of three differentlaboratories doing the same experimentgave standard deviations of 2.711 x 10~;!,FREQUENCY OFVRLUES REPORTEDNo. 2 COMPUTER EVALUATION OF LAB UNKNOWNS G72.013 x 1()-3 and 3.134 x 1()~3. Inall cases, the class average refined towithin 0.0001 molar units of the acceptedvalue, even when the accepted valuewas not input. For small, laboratories(less than 100 total determinations) it ispreferable to input the true value, be-cause the sample size does not seem tobe large enough. For large laboratories(>200 determinations) the sample sizeappears to be large enough and citherthe accepted value or class average prop-erly refine to indistinguishable values.As a result it should be possible to pro-vide instantaneous grading for schoolswith on-line computer facilities, in addi-tion to the possibility of comparingclasses from year to year. Class com-parison may aid those laboratories whichhave relatively low enrollments, andwhich, as a result, may suffer from poorstatistics. A detailed comparison of thestandard deviations and the proximityof the class average to the accepted valueshould also provide a mechanism tomeasure the effects of changing the word-ing of a procedure, the procedure itself,or some other part of the experimentsuch as the descriptions, generally im-proved procedures or better indicatorswhich frequently result in smaller stand-ard deviations and more accurate results.Acknowledgments. We would like to thankKathy Dittemore for her aid in the initial de-velopment of our series of computer programsand Kenneth Sando, Dwight C. Tardy and JohnR. Doyle for their helpful comments. Wewould also like to acknowledge The Universityof Iowa Graduate College for computer fundsto develop this program, and The University ofIowa Computer Center staff for their as-sistance.LITERATURE CITEDAltcnburg, J. F., L. A. King and C. Campbell.19(58. Computer grading of general chem-istry laboratory reports. J. Chem. PMuc. 45:615-010.Galyan, R. H. and V. A. Ryan. 1972. Com-puter aided undergraduate radio-chemistryinstruction utilizing time sharing terminals.|. Chcm. Educ. 49: 591.Hoffman, A. A. J. (cd.) 1975. Proceedings ofthe 1975 Conference on computers in theundergraduate curricula (CCUC/(5), FortWorth, Texas.Jones, D. E. and F. E. Lytie. 1973. Computeraided grading of quantitative unknowns. J.Chem. Educ. 50: 285-28(5.Klatt, L. N. and J. C. Sheafer. 1974. Chang-ing student attitudes about quantitative an-alysis laboratory. J. Chem. Educ. 51: 239-242.Reiter, R. C. and J. F. Budig. 1974. A time-sharing computer application to generalchemistry laboratory work. J. Chem. Educ.51:43.Rosenstein, R. D. and S. R. Smith. 19(52.Using computing machines to grade studentanalysis reports. J. Chem. Educ. 39: 620-621.Smith, S. R., R. Schor and P. C. Donohue.1965. Computer grading of small numbersof laboratory unknowns. 1. Chem. Educ.42" 224-225Wartell, M. A. and J. A. Hurlbut. 1972. _ AFortran IV program for grading quantitativeanalysis unknowns. J. Chem. Educ. 49: 508.Wellman, K. M. 1970. Computer grading ofintroductory organic chemistry laboratoryresults. J. Chem. Educ. 47: 142.Wise, G. 1972. Computer programs for un-dergraduate physical chemistry students. J.Chem. Educ. 49: 559-560.

An Algorithm for the Computer Evaluation of Quantitative Laboratory Unknowns Based on Accuracy and Precision

https://kb.osu.edu/bitstream/1811/22430/1/V077N2_063.pdf

An Algorithm for the Computer Evaluation of Quantitative Laboratory Unknowns Based on Accuracy and Precision

Abstract

Similar works

Full text

Available Versions

KnowledgeBank at OSU