In proficiency tests the participants' results are usually converted into scores. In some schemes the participants are required to report the sum of the concentrations of a number of analytes and this total also is converted into a score. In such instances the scoring procedure for the total should be mathematically consistent with that for the separate analytes. When these analytes are determined from a single test portion, however, the errors in the results from a participant are likely to be correlated because some stages of the analysis are common to all of the analytes. For a consistent outcome, the scoring method must take account of such correlation

Michael Thompson

Thompson

English

Thompson, Michael

Name not available

   BIROn - Birkbeck Institutional Research Online  Enabling open access to Birkbeck’s published research output    Scoring the sum of correlated results in analytical proficiency testing  Journal Article   http://eprints.bbk.ac.uk/3436    Version: Published (Refereed)    Citation:       © 2010 M. Thompson    Publisher version    ______________________________________________________________  All articles available through Birkbeck ePrints are protected by intellectual property law, including copyright law. Any use made of the contents should comply with the relevant law. ______________________________________________________________  Deposit Guide  Contact: lib-eprints@bbk.ac.uk Birkbeck ePrints Thompson, M. (2010) Scoring the sum of correlated results in analytical proficiency testing – Analytical Methods 7 (pp.976-977) TECHNICAL NOTE www.rsc.org/methods | Analytical MethodsDownloaded on 13 June 2011Published on 17 May 2010 on http://pubs.rsc.org | doi:10.1039/C0AY00108BView OnlineScoring the sum of correlated results in analytical proficiency testingMichael ThompsonReceived 15th February 2010, Accepted 24th April 2010First published as an Advance Article on the web 17th May 2010DOI: 10.1039/c0ay00108bIn proficiency tests the participants’ results are usually converted into scores. In some schemes theparticipants are required to report the sum of the concentrations of a number of analytes and this totalalso is converted into a score. In such instances the scoring procedure for the total should bemathematically consistent with that for the separate analytes. When these analytes are determined froma single test portion, however, the errors in the results from a participant are likely to be correlatedbecause some stages of the analysis are common to all of the analytes. For a consistent outcome, thescoring method must take account of such correlation.Scoring in proficiency testsProficiency testing comprises the regular distribution of typicaltest materials for blind analysis by the participants in the scheme,using their routine methods.1 Its primary purpose is to enableparticipant laboratories to judge the success of their analyticalsystems in terms of fitness for purpose. Participation in anappropriate scheme is now mandatory for accreditation.In a round of a proficiency test, most scheme providers converta participant’s analytical result x into a z-score given by z ¼ (x xA)/sp, where the assigned value xA is the provider’s best estimateof the true value. The assigned value is usually taken as theconsensus of all of the participants’ results, often a robust mean.The standard deviation for proficiency sp in a round—informallycalled the ‘target value’—is preferably a prescribed uncertaintythat characterises fitness for purpose in the appropriate analyt-ical sector.1 It is important to emphasise, for the purposes of thefollowing discussion, that such a target value is not designed tocharacterise the participants’ results but to specify in advance therequired performance—it is independent of the participants’results. This sp value is predetermined by the scheme providerand available to the participants at the time of analysis.Correlation in resultsWhen a number of analytes are determined from a single testportion, many stages of the analytical method are common to allof the analytes. Variation in procedure at these stages will affectthe results of all of the analytes similarly. In a proficiency testthat circumstance gives rise to a correlation among the errors inparticipant’s results for the analytes. This is not a problem per sebecause the i-th analyte is scored separately according to itsindividual target value sp(i).In some instances, however, legislation or custom requireslaboratories to report the total concentration of a group ofanalytes, ‘‘total aflatoxins’’ for example, as well as the individualconcentrations. For the purposes of proficiency testing the totalconcentration should then be scored as well. However, the targetvalue for the total concentration should be mathematicallySchool of Science, Birkbeck University of London, Malet Street, London,WC1E 7HX, UK. E-mail: m.thompson@bbk.ac.uk976 | Anal. Methods, 2010, 2, 976–977consistent with the target values used for the individual analytes.Were that not the case, a participant could conceivably receive‘satisfactory’ z-scores for all of the individual analytes but an‘unsatisfactory’ z-score for the total concentration. (‘Satisfac-tory’ and ‘unsatisfactory’ are used non-pejoratively as informallabels for the conditions |z| < 2 and |z| > 3 respectively.)A consistent target value for the total sp(T) must take accountof the covariances2 covp(i,j) between the analytes i and j, so thatspðTÞ ¼ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃXis2pðiÞ þ 2Xisjcovpði; jÞq: (1)Here, however, the covariances (like the individual sp(i)values)have a special prescriptive meaning so cannot be estimateddirectly from the participants’ results. They have to be evaluatedfrom the prescribed individual target values and the observedcorrelation coefficients r(i, j) between analytes i and j over allparticipants, via the identity covp(i, j) ¼ r(i, j)sp(i)sp(j). Then theconsistent target value for the total concentration isspðTÞ ¼ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃsTpRspq; (2)where sTp ¼ [sp(1),sp(2),/,sp(n)] is the row vector of the nprescribed target values and R is the observed n  n correlationmatrix.In the absence of a knowledge of R, a possible ‘cautious’approach would be to assume that the analytes are perfectlycorrelated, that is, with all r(i, j) ¼ 1. Eqn (1) and (2) then reduceto s0p(T) ¼Pisp(i). A contrasting expedient, here called ‘naive’,would be to ignore any covariance by assuming that all of thecorrelation coefficients are zero-valued. The target value for thetotal concentration is then given by s00pðTÞ ¼ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃPi s2pðiÞq. Thediscrepancy between these extremes can be substantial,approaching a factor ofﬃﬃﬃnpwhen the sp values are comparable inmagnitude. A poor choice between these extremes could there-fore have a potentially serious effect on z-scores for the totalconcentration.ExampleThe data were taken from a single round (identity not disclosed)of a FAPAS3 proficiency test. There were four analytes,This journal is ª The Royal Society of Chemistry 2010Fig. 1 z-Scores for total aflatoxins calculated by using different esti-mates of the target value. Each point shows the result from a singleparticipant. Two extreme outliers are not shown.B1 B2 G1 G2B1 1.00 0.67 0.38 0.30B2 0.67 1.00 0.45 0.76G1 0.38 0.45 1.00 0.18G2 0.30 0.76 0.18 1.00Downloaded on 13 June 2011Published on 17 May 2010 on http://pubs.rsc.org | doi:10.1039/C0AY00108BView Onlineaflatoxins B1, B2, G1 and G2, expressed as ppb mass fraction.There were 74 participant laboratories submitting a complete setof results. The four predetermined target values were respectivelysTp ¼ [1.03 0.56 0.72 0.37]. The data showed a moderate amountof correlation, with the correlation matrix R as followsThe three alternative target values for the total aflatoxins werecalculated as: consistent sp(T) ¼ 2.09; ‘cautious’ s0p(T) ¼ 2.68; ‘naive’ s00p(T) ¼ 1.42.The corresponding z-scores for total aflatoxins are shown inFig. 1. There are substantial differences among the threeThis journal is ª The Royal Society of Chemistry 2010interpretations. The ‘naive’ method for calculating sp(T) identi-fied 12 laboratories with |z| > 3 (i.e., requiring investigative and/or remedial action on the part of the participant) but the‘cautious’ method identified only four. The consistent methodnecessarily gave an outcome between these extremes. (Note: inFAPAS the value of sp(T) used falls close to ‘cautious’ and thusavoids scores of unduly large magnitude.)ConclusionsProficiency test providers should be aware of a potentiallyimportant inconsistency in the target value for the totalconcentration of a number of analytes if the participants’ resultsare correlated and the covariances not accounted for. The sameproblem might affect individual participants wanting toconstruct an ‘overall’ score for a number of analytes.In these contexts it is important also to be aware that thecorrelations among a particular suite of analytes may varyconsiderably from round to round. This is largely determined bythe concentrations of the analytes: near detection limitssubstantial random contributions will almost eliminate anycorrelations that might predominate at higher concentrations.Because of this circumstance the use of a single ‘global’ estimateof R for successive rounds would probably be inappropriate.References1 M. Thompson, S. L. R. Ellison and R. Wood, The InternationalHarmonised Protocol for the Proficiency Testing of AnalyticalChemistry Laboratories, Pure Appl. Chem., 2006, 78, 145–196.2 AMC Technical Briefs, 2008, vol. 30, ISSN 1757 5958, http://www.rsc.org/images/brief%2030_tcm18-119170.pdf.3 FAPAS Secretariat, Central Science Laboratory, Food andEnvironment Research Agency, Sand Hutton, York YO41, 1LZ UK.Anal. Methods, 2010, 2, 976–977 | 977

Scoring the sum of correlated results in analytical proficiency testing

Birkbeck Institutional Research Online

Central Science Laboratory, Food and Environment Research Agency,

The International Harmonised Protocol for the Proﬁciency Testing of Analytical Chemistry Laboratories, Pure Appl. Chem.,

Crossref

Scoring the sum of correlated results in analytical proficiency testing

Abstract

Similar works

Full text

Available Versions

Name not available

Birkbeck Institutional Research Online

Crossref