PowerPoint slides for Hypothesis Testing. Examples are taken from the Medical Literatur

Bryant, Trevor

English

EdShare

©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing1Hypothesis TestingDr Trevor BryantLearning OutcomesFollowing this session you should be able to:• Understand the concept and general procedure of hypothesis testing• Understand the concept and interpretation of P values• Explain the relationship between CI (point estimate ±1.96 x S.E) & Hypothesis Testing• Describe Type I & Type II ErrorsHypothesis testing - milestones• Develop the research question• Develop the research hypothesis• State it as a statistical hypothesis• Test the hypothesis• Was it a good idea?• Next question(s)©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing2The Four Elements of a Research Question• Cells, Patient or Population– What or Who is the question about? • Intervention or Exposure* – What is being done or what is happening to the cells, patients or population? • Outcome(s)– How does the intervention affect the cells, patients or population? • Comparison(s)– What could be done instead of the intervention*Intervention is intentional whereas an exposure is incidentalDefining a Research Hypothesis‘A well-defined hypothesis crystallizes the research question and influences the statistical tests that will be used in analyzing the results’http://intra.som.umass.edu/nakosteen/Topics/Developing%20the%20research%20design.doc [Accessed 17 Feb 2009]You cannot prove a hypothesis• Falisifiability– (Karl Popper, 1902-1994)• Scientific laws cannot be shown to be True or False• They are held as Provisionally True• ‘All Swans are White’– (David Hume,1711-1776)©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing3What is a Hypothesis?• A tentative statement that proposes a possible explanation to some phenomenon or event• A useful hypothesis is a testable statement which may include a prediction• Any procedure you follow without a hypothesis is not an experiment  Formalized Hypothesis • IF and THEN• Specify a tentative relationship• IF skin cancer is related to ultraviolet light, THENpeople with a high exposure to UV light will have a higher frequency of skin cancerDependent variableIndependent variableDisproving a hypothesis• Collect evidence• If evidence supports current hypothesisHold hypothesis to be Provisionally True• If evidence does not support hypothesisReject hypothesis and develop new one• Statistical testing uses Null Hypothesis– No difference unless unlikely event (p)– Alternative hypothesis – a difference?– Swans©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing4Statistical Hypothesis testing -Overview• Define the problem• State null hypothesis (H0)• State alternative hypothesis (H1)• Collect a sample of data to gather evidence• Calculate a test statistic• Relate test statistic to known distribution to obtain P value• Interpret P valuevalue observed of error standardvalue edhypothesis-  value observedstatistic Test Defining the problem• The null hypothesis assumes No EffectH0 : There is no treatment effect in the population of interest• The alternative hypothesis opposite of null hypothesis H1 : There is a treatment effect in the population of interestNote: These are specified before collecting the data, they relate to the population not the sample and usually no direction is specified for the effectCalculating the test statistic• The choice of test statistic will depend on the type of data collected and the hypotheses of interest• ‘Large’ test statistic - more evidence for H1• Values of the test statistic are standardized and can ‘compare to published tables’ – calculatedThe test statistic summarises the data from the sample in a single number.  It’s size indicates the amount of evidence gathered for either hypothesis©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing5How do we choose the test statistic?• What is the measurement of interest?Means, proportions, etc• What is the distribution of the measurementNormal or skewed• How many groups of patients are being studied? 1, 2, 3  or more• Are they independent groups?or pairedInterpretation of the P valueThe P value is the probability of getting a test statistic as large as, or larger than, the one obtained in the sample if the null hypothesis were trueIt is the probability that our results occurred by chanceInterpretation of the P value (2)• By convention, P values of <.05 are often accepted as “statistically significant” in the medical literature• It is an arbitrary cut-off• A cut-off of P <.05 means that in about 5 out of 100(1 in 20) experiments,  a result would appear significant just by chance (“Type I error”)• We can use other P values for example 0.01©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing6Interpretation of the P value (3)• Large P value (usually > 0.05) • Likely to have got results by chance if H0 was true– Accept null hypothesis– Result is non-significant• Small P value (usually < 0.05)• Unlikely to have got results by chance if H0 was true – Reject null hypothesis – accept alternative hypothesis– Result is significantWhere do P >0.05 P >0.01 P >0.001 fit in?Example of a hypothesis test©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing7Example of a hypothesis testRandomised controlled trial of cranberry-lingonberry juice and Lactobacillus GG drink for the prevention of urinary tract infections in women.  Kontiokari et al.  BMJ (2001) 322: 1571-3150 women were randomised to three groups (cranberry-lingonberry juice, lactobacillus drink or control group).  At six months, 8/50 (16%) women in the cranberry group, 19/50 (38%) in the lactobacillus group, and 18/50 (36%) in the controlgroup had had at least one recurrence.Question: Is there any EFFECT of cranberry to prevent infection?Example of a hypothesis testWhat is the Hypothesis?If women drink cranberry-lingoberry juice then there will be a reduction in the recurrence of urinary tract infectionStatistical HypothesisNull H0: There are no differences in recurrence rates among women in the population who drink cranberry-lingoberry juice, lactobacillus drink or neither of theseAlternative H1: There is a difference in the recurrence rates between these three groups in the populationExample of a hypothesis test• Which test should be used?Chi-squared test• What is the test statistic?X2 7.05, P = 0.03• How to interpret the result?Reject null hypothesisThere is a significant difference in recurrence rates between these three groups (based on 5% significance)©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing8Example of a hypothesis testCranberry Lactobacillus ControlLower Bound 0.08 0.26 0.24Point Estimate 0.16 0.38 0.36Upper Bound 0.29 0.52 0.500.20.40.6ProportionTrial GroupsProportions with 95%CIs5 minute breakErrors in Hypothesis testing Jury’s verdictTrue state of Defendant Defendant really is GuiltyDefendant really is InnocentGuiltyCorrect Decision Not guilty  Correct Decision©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing9Statistical DecisionTrue state of null hypothesis - RealityNull hypothesis is TrueNull Hypothesis is FalseAccept H0 accepted correctly Type II error ()Reject Type I error () H0 rejected correctlyTypes of Error in hypothesis testingType I error• The probability that we reject null hypothesis when it is true• ‘False positive’• Rejected H0 because the results occurred by chance• Conclude that there is a significant effect, even though no true effect exists• Probabilities of Type 1 error called – alpha ()Determined in advance,  typically 5%Type 1 Error – Null Hypothesis is TrueShaded areas gives the probability that the Null hypothesis is wrong rejectedAdapted from Kirkwood & Sterne 2nd Ed©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing10Type II error• The probability that we accept null hypothesis when it is false• ‘False Negative’• Accept H0 even though it is not true• Conclude that there is no significant effect, even though a true difference exists• Probabilities of Type II error called – beta ()Type II Error – Null Hypothesis is FalseAdapted from Kirkwood & Sterne 2nd EdShaded area is the probability (b%) that the null hypothesis fails to be rejectedSampling distribution under null hypothesisReal sampling distribution of sample differenceType II error rate• Type II error rate depends on :– the size of the study– the variability of the measurement• The implications of making either a type I or type II error will depend on the context of the study©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing11The power of the study is the probability of correctly detecting a true effectOr the probability of correctly rejecting the null hypothesisPower = 100% - Type II error rate = (1 – ) x 100The Power of the StudyThe Power of the Study (2)• The power will be low if there are only a few observationstaking a larger sample will improve the power• The power will be low if there is variability amongst the observationsreducing variability will improve power• Ideally we would like a power of 100% but this is not feasible usually accept a power of 80%Things to considerWe can never be 100% certain that the correct decision has been reached when carrying out a hypothesis testAn hypothesis test cannot prove that a null hypothesis is true or false. It only gives an indication of the strength of evidence©Dr Trevor BryantUniversity of SouthamptonHypothesis Testing12References:• Altman, D.G. Practical Statistics for Medical Research. Chapman and Hall 1991. Chapter 8• Kirkwood B.R. & Sterne J.A.C. Essential Medical Statistics. 2nd Edition. Oxford: Blackwell Science Ltd 2003. Chapter 8• Machin D. and Campbell M.J. The Design of Studies for Medical Research, John Wiley and Sons 2005 Chapter1Questions

Hypothesis Testing

©Dr Trevor Bryant
University of Southampton
Hypothesis Testing1
Hypothesis Testing
Dr Trevor Bryant
Learning Outcomes
Following this session you should be able to:
• Understand the concept and general procedure of 
hypothesis testing
• Understand the concept and interpretation of P values
• Explain the relationship between CI (point estimate ±
1.96 x S.E) & Hypothesis Testing
• Describe Type I & Type II Errors
Hypothesis testing - milestones
• Develop the research question
• Develop the research hypothesis
• State it as a statistical hypothesis
• Test the hypothesis
• Was it a good idea?
• Next question(s)
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing2
The Four Elements of a Research 
Question
• Cells, Patient or Population
– What or Who is the question about? 
• Intervention or Exposure* 
– What is being done or what is happening to the cells, 
patients or population? 
• Outcome(s)
– How does the intervention affect the cells, patients or 
population? 
• Comparison(s)
– What could be done instead of the intervention
*Intervention is intentional whereas an exposure is incidental
Defining a Research Hypothesis
‘A well-defined hypothesis crystallizes the research 
question and influences the statistical tests that 
will be used in analyzing the results’
http://intra.som.umass.edu/nakosteen/Topics/Developing%20the%20research%20design.doc 
[Accessed 17 Feb 2009]
You cannot prove a hypothesis
• Falisifiability
– (Karl Popper, 1902-1994)
• Scientific laws cannot be shown to be True or False
• They are held as Provisionally True
• ‘All Swans are White’
– (David Hume,1711-1776)
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing3
What is a Hypothesis?
• A tentative statement that proposes a possible 
explanation to some phenomenon or event
• A useful hypothesis is a testable statement which may 
include a prediction
• Any procedure you follow without a hypothesis is not 
an experiment  
Formalized Hypothesis 
• IF and THEN
• Specify a tentative relationship
• IF skin cancer is related to ultraviolet light, THEN
people with a high exposure to UV light will have a 
higher frequency of skin cancer
Dependent variable
Independent variable
Disproving a hypothesis
• Collect evidence
• If evidence supports current hypothesis
Hold hypothesis to be Provisionally True
• If evidence does not support hypothesis
Reject hypothesis and develop new one
• Statistical testing uses Null Hypothesis
– No difference unless unlikely event (p)
– Alternative hypothesis – a difference?
– Swans
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing4
Statistical Hypothesis testing -Overview
• Define the problem
• State null hypothesis (H0)
• State alternative hypothesis (H1)
• Collect a sample of data to gather evidence
• Calculate a test statistic
• Relate test statistic to known distribution to obtain P value
• Interpret P value
value observed of error standard
value edhypothesis-  value observed
statistic Test 
Defining the problem
• The null hypothesis assumes No Effect
H0 : There is no treatment effect in the population of interest
• The alternative hypothesis opposite of null hypothesis 
H1 : There is a treatment effect in the population of interest
Note: These are specified before collecting the data, 
they relate to the population not the sample and 
usually no direction is specified for the effect
Calculating the test statistic
• The choice of test statistic will depend on the type of 
data collected and the hypotheses of interest
• ‘Large’ test statistic - more evidence for H1
• Values of the test statistic are standardized and can 
‘compare to published tables’ – calculated
The test statistic summarises the data from the sample in 
a single number.  It’s size indicates the amount of 
evidence gathered for either hypothesis
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing5
How do we choose the test statistic?
• What is the measurement of interest?
Means, proportions, etc
• What is the distribution of the measurement
Normal or skewed
• How many groups of patients are being studied? 
1, 2, 3  or more
• Are they independent groups?
or paired
Interpretation of the P value
The P value is the probability of getting a test statistic 
as large as, or larger than, the one obtained in the 
sample if the null hypothesis were true
It is the probability that our results occurred by chance
Interpretation of the P value (2)
• By convention, P values of <.05 are often accepted as 
“statistically significant” in the medical literature
• It is an arbitrary cut-off
• A cut-off of P <.05 means that in about 5 out of 100
(1 in 20) experiments,  a result would appear 
significant just by chance (“Type I error”)
• We can use other P values for example 0.01
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing6
Interpretation of the P value (3)
• Large P value (usually > 0.05) 
• Likely to have got results by chance if H0 was true
– Accept null hypothesis
– Result is non-significant
• Small P value (usually < 0.05)
• Unlikely to have got results by chance if H0 was true 
– Reject null hypothesis – accept alternative hypothesis
– Result is significant
Where do P >0.05 P >0.01 P >0.001 fit in?
Example of a hypothesis test
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing7
Example of a hypothesis test
Randomised controlled trial of cranberry-lingonberry juice and 
Lactobacillus GG drink for the prevention of urinary tract infections 
in women.  Kontiokari et al.  BMJ (2001) 322: 1571-3
150 women were randomised to three groups (cranberry-
lingonberry juice, lactobacillus drink or control group).  
At six months, 8/50 (16%) women in the cranberry group, 19/50 
(38%) in the lactobacillus group, and 18/50 (36%) in the control
group had had at least one recurrence.
Question: Is there any EFFECT of cranberry to prevent 
infection?
Example of a hypothesis test
What is the Hypothesis?
If women drink cranberry-lingoberry juice then there 
will be a reduction in the recurrence of urinary tract 
infection
Statistical Hypothesis
Null H0: There are no differences in recurrence rates 
among women in the population who drink cranberry-
lingoberry juice, lactobacillus drink or neither of these
Alternative H1: There is a difference in the recurrence 
rates between these three groups in the population
Example of a hypothesis test
• Which test should be used?
Chi-squared test
• What is the test statistic?
X2 7.05, P = 0.03
• How to interpret the result?
Reject null hypothesis
There is a significant difference in recurrence rates 
between these three groups (based on 5% 
significance)
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing8
Example of a hypothesis test
Cranberry Lactobacillus Control
Lower Bound 0.08 0.26 0.24
Point Estimate 0.16 0.38 0.36
Upper Bound 0.29 0.52 0.5
0
0.2
0.4
0.6
P
ro
p
o
rt
io
n
Trial Groups
Proportions with 95%CIs
5 minute break
Errors in Hypothesis testing 
Jury’s 
verdict
True state of Defendant 
Defendant really is 
Guilty
Defendant really is 
Innocent
Guilty

Correct Decision 
Not guilty 

Correct Decision
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing9
Statistical 
Decision
True state of null hypothesis - Reality
Null hypothesis is 
True
Null Hypothesis is 
False
Accept H0 accepted correctly Type II error ()
Reject Type I error () H0 rejected correctly
Types of Error in hypothesis testing
Type I error
• The probability that we reject null hypothesis when it 
is true
• ‘False positive’
• Rejected H0 because the results occurred by chance
• Conclude that there is a significant effect, even 
though no true effect exists
• Probabilities of Type 1 error called – alpha ()
Determined in advance,  typically 5%
Type 1 Error – Null Hypothesis is True
Shaded areas gives the probability that the Null hypothesis is wrong 
rejected
Adapted from Kirkwood & Sterne 2nd Ed
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing10
Type II error
• The probability that we accept null hypothesis when it 
is false
• ‘False Negative’
• Accept H0 even though it is not true
• Conclude that there is no significant effect, even 
though a true difference exists
• Probabilities of Type II error called – beta ()
Type II Error – Null Hypothesis is False
Adapted from Kirkwood & Sterne 2nd Ed
Shaded area is the probability (b%) that the null hypothesis fails to be rejected
Sampling 
distribution under 
null hypothesis
Real sampling 
distribution of 
sample 
difference
Type II error rate
• Type II error rate depends on :
– the size of the study
– the variability of the measurement
• The implications of making either a type I or type II 
error will depend on the context of the study
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing11
The power of the study is the probability of correctly 
detecting a true effect
Or the probability of correctly rejecting the null 
hypothesis
Power = 100% - Type II error rate = (1 – ) x 100
The Power of the Study
The Power of the Study (2)
• The power will be low if there are only a few 
observations
taking a larger sample will improve the power
• The power will be low if there is variability amongst 
the observations
reducing variability will improve power
• Ideally we would like a power of 100% but this is not 
feasible 
usually accept a power of 80%
Things to consider
We can never be 100% certain that the correct decision 
has been reached when carrying out a hypothesis test
An hypothesis test cannot prove that a null hypothesis 
is true or false. It only gives an indication of the 
strength of evidence
©Dr Trevor Bryant
University of Southampton
Hypothesis Testing12
References:
• Altman, D.G. Practical Statistics for Medical Research. 
Chapman and Hall 1991. Chapter 8
• Kirkwood B.R. & Sterne J.A.C. Essential Medical 
Statistics. 2nd Edition. Oxford: Blackwell Science Ltd 
2003. Chapter 8
• Machin D. and Campbell M.J. The Design of Studies 
for Medical Research, John Wiley and Sons 2005 
Chapter1
Questions


http://edshare.soton.ac.uk/4167/1/10_Hypothesis_Testing.pdf

Hypothesis Testing

Abstract

Similar works

Full text

Available Versions

EdShare

EdShare