Armstrong, Richard A.

Hilton, Anthony

English

Aston Publications Explorer

3Stat NoteIn the third of a series of articles about statistics for biologists, Anthony Hilton andRichard Armstrong test the difference between two groupsTesting the difference between two groupswww.sfam.org.uk30 December 2005www.sfam.org.uk 31December 2005Featurestwo treatments differ by100.16 units (Fig 1) and thetwo samples exhibit littleoverlap suggesting a realeffect of the supplement (Fig2). There is, however,variation in microbial biomassbetween replicate flasks withineach group. Hence, is thedifference between the meansactually attributable to theeffect of the supplement orcould it be accounted for byrandom variation between theflasks? To decide betweenthese two alternatives, thetreatment effect (U* - S*) iscompared with the degree ofvariation pooled from bothgroups by carrying out a ‘t’test. The statistic ‘t’ is theratio of the difference betweenthe two means to the SE ofthe difference between themeans:t = U* - S* / √s2 (1/n1 +1/n2) where ‘s2’ is the averagevariance of both groups andn1 and n2 are the number ofobservations within eachgroup (n1 = n2 = 25). Notethat this calculation is similarto that carried out in Statnote2 when an individual value ‘x’was converted so that itbecame a member of the SND.In the present example, the ‘t’distribution is used instead ofthe SND because ‘t’ describesthe variation of meanscalculated from small numbersof observations moreaccurately. Hence, when ‘t’ iscalculated, the differencebetween the means (U* - S*)becomes a member of the ‘t’distribution. The probability ofobtaining a value of ‘t’ of thismagnitude by chance fromtwo groups of samples wheneach comprises 25observations is obtained eitherfrom statistical software or byconsulting statistical tables(Fisher and Yates, 1963). Inthe present example, a valueof t = 16.59 was obtained andis taken to a table of the ‘t’distribution, entering the tablefor n1 + n2 – 2 DF (DF =48). When ‘t’ is equal to orFig. 1. Mean bacterial biomass (with 95% confidence intervals)using unsupplemented (UNS) and supplemented (+S) media (t-testfor comparing the means of both groups, t = 16.59, P < 0.001)difference between the meansof two samples, knowledge isrequired of how thedifferences between twosample means would vary.Hence, for each of our twosamples, the mean iscalculated and the differencebetween the mean of theunsupplemented (U*) groupsubtracted from the mean ofthe supplemented (S*) group.This difference represents thetreatment effect of theexperiment; i.e., the degree towhich the media supplementmay have increased bacterialbiomass. If this experimentwas repeated many times,several estimates of U* - S*would be obtained and wecould construct a frequencydistribution of the differencesbetween the means. However,if the distribution of themeans from the supplementedand unsupplemented groupsare normally distributed, thenthe distribution of thedifferences between pairs ofmeans taken from these twopopulations will also benormally distributed. As aresult, the standard normaldistribution (SND) can beused to test whether there is atrue difference between themeans. The group means of thenovel compound. Both vesselswere allowed to equilibrateand were subject to identicalenvironmental / incubationconditions. The vessels werethen inoculated with a cultureof Bacterium x at an equalculture density and thefermentation allowed toproceed until all the availablenutrients had been exhaustedand bacterial growth hadceased. The entire volume ofculture media in eachfermentation vessel was thenremoved and filtered torecover the bacterial biomass.This experiment was repeated25 times and the biomassmeasurements are given inStatnote 2.How are the calculationsdone?To determine whether anindividual measurement istypical of a populationrequires knowledge of howindividual measurements vary;i.e., the SD of the population(Statnote 2). Similarly, todetermine the degree of errorassociated with a sample meanrequires knowledge of howmeans vary; i.e., the SE of themean (Statnote 2).Extrapolating this principlefurther, to determine whetherthere is a significantN STATNOTE 1(Microbiologist,June 2005), wedescribed a test todetermine whether a sampleof measurements came from anormal distribution.If a variable is normallydistributed, it is referred to asa ‘parametric’ variable. InStatnote 2 (Microbiologist,September 2005) we showedthat if a variable is parametric,then a sample of observationscan be described by thesample mean ‘X*’ (‘centraltendency’) and standarddeviation (SD) ‘s’ (‘spread’).We also described twostatistical procedures based onthe normal distribution.Firstly, whether an individualmeasurement is typical oratypical of a larger populationof measurements andsecondly, we showed that themean of a small sample ofmeasurements also comesfrom a normal distribution,viz. that of the population ofsample means. The degree ofspread of this distribution canbe described by the standarderror (SE) of the mean. Thisinformation was used tocalculate a confidence interval(CI) for a sample mean, whichis a measure of the errorassociated with a sample meanas an estimate of the ‘true’population mean. In thisStatnote, these statistical ideasare extended to the problemof testing whether there is astatistically significantdifference between twosamples of measurements. The scenarioWe return to thehypothetical experimentdescribed in Statnote 2 toinvestigate the efficacy of anovel media supplement inpromoting the development ofcell biomass. To recapitulate,two 10-litre fermentationvessels were filled withidentical growth media withthe exception that the mediain one of the vessels wassupplemented with 10ml of aIwww.sfam.org.ukDecember 200532Fig. 2. Frequency distribution of the individual biomassmeasurements using unsupplemented (UNS) and supplemented(+S) media. Curves represent the normal distribution fitted toeach sample.greater than 2.01 (the value atP = 0.05 for 48 DF), thevalue is in a region of thedistribution which includes the5% most extreme values.Hence, t = 16.59 is anunlikely value to occur bychance and therefore, there isa real difference between thetwo means.One-tail and two-tailtestsIt is possible to proposetwo different null hypotheses.First, that the addition of thesupplement (S) would have noeffect on bacterial biomass.This hypothesis does notspecify whether a significantincrease or a decrease inbiomass would be necessaryto refute the null hypothesis.In this case, a two-tailed testwould be appropriate, i.e.,both tails of the ‘t’ distributionare used to test thehypothesis. Second, that thesupplement would onlyincrease biomass since it maybe known in advance that itcould not significantlydecrease biomass. If thehypothesis specifies whether apositive or a negative effect isnecessary to refute thehypothesis, a one-tail testwould be appropriate. Somestatistical tables indicate boththe one-tail and two-tailprobabilities corresponding toparticular columns. Moststatistical tables, with somenotable exceptions, onlyindicate the two-tailprobabilities. To find the one-tail probabilities in a two-tailtable, halve the probabilities;i.e., the 5% one-tailprobabilities are found in the10% two-tail column.Paired and unpaired ‘t’testsAn experiment involvingtwo treatments or groups canbe carried out in two differentways, viz., the unpaired andthe paired methods. Theexperiment described in ourscenario was carried out usingan unpaired design; i.e., themedia supplement wasallocated at random andwithout restriction to half ofthe 50 original flasks. In apaired design, however, the 50flasks are first divided into 25pairs with the intention ofprocessing a single pair (onesupplemented, the otherunsupplemented) on each of25 days and second, thesupplement is then allocatedto one flask of each pairindependently and at random.Hence, there is a restriction inthe allocation of thetreatments to the flasks and adifferent analysis is required.In a paired design, the ‘t’ testis made as follows:t = d*/ (sd/ √n) In this case, ‘d*’ is themean of the differencesbetween each of the 25 pairsof observations and ‘sd’ is thestandard deviation of thesedifferences. The same ‘t’ tableis used for determining thesignificance of ‘t’. In a paired‘t’ test, however, a differentrule is used for entering the ‘t’table; viz., ‘t’ has n-1 degreesof freedom, where ‘n’ is thenumber of pairs of subjects.Again, one-tail or two-tail testsmay be made as appropriate.Advantages of pairingIs a paired or an unpaireddesign the best method ofcarrying out the experiment?Each type of design hasadvantages and disadvantages.A paired design is oftenemployed to reduce the effectof the natural variation thatalways exists between flasksor replicates. How this may beachieved can be seen byexamination of the formula forthe unpaired ‘t’ test. A valueof ‘t’ is the difference betweenthe two treatment meansdivided by the SE of thisdifference. If variation amongflasks is large, say fromprocessing them at differenttimes of the day or ondifferent days, it will increasethe SE of the difference andlower the value of ‘t’ even ifthe difference between meansis large. Notice, however, thatin an unpaired design, the ‘t’table is entered for 48 DF.Pairing the flasks may reducethe SE because paired ‘t’ iscalculated from thedifferences between pairs ofobservations. In other words,the effect of the experimentaltreatment is being determinedwithin a matched pair offlasks. Pairing should only beconsidered, however, if thereis evidence that it actuallyreduces the variability, e.g.,pairing supplemented andunsupplemented flasks on thesame day when the day ofassay does significantly affectthe measurement. If there isno reduction in the SE bypairing, i.e., it does not matterwhich day the samples aremeasured, then there is adisadvantage of the paireddesign because the ‘t’ table isentered with only 24 DF (1less than the number ofgroups). Entering the ‘t’ tablewith a smaller number ofdegrees of freedom meansthat a larger value of ‘t’ will berequired to demonstrate asignificant difference betweenthe means.The ‘t’ test is a usefulmethod of comparing twogroups when the dataapproximate to a normaldistribution. In many cases inmicrobiology, however, datamay not conform to thisdistribution and differentmethods may be required totest differences between thegroups and these methods willbe reviewed in Statnote 4.References■ Fisher RA & Yates F (1963)Statistical Tables for Biological,Agricultural, and MedicalResearch. Sixth Ed, Longman,Bristol.■ Hilton A & Armstrong RA(2005) Are the data normal?Microbiologist June 2005, Vol 6No 2, 34-35■ Hilton A & Armstrong RA(2005) Describing the normal.Microbiologist September 2005Vol 6 No 3, 30-32Dr Anthony* Hilton and Dr Richard Armstrong***Pharmaceutical Sciencesand **Vision Sciences,Aston University,Birmingham, UK

Statnote 3: testing the difference between two groups

https://publications.aston.ac.uk/id/eprint/39915/1/Statnote_3.pdf

Statnote 3: testing the difference between two groups

Abstract

Similar works

Full text

Available Versions

Aston Publications Explorer