In the first part of this presentation, Jyotishka Datta discusses the concepts and calculations behind the efficacy and effectiveness for vaccines from a statistician’s perspective, as well as the different factors contributing to it. Prof. Datta also offers a brief tour through the history of vaccines in preventing infectious diseases and the development of statistical tools such as randomized controlled trials. Lastly, Prof. Datta discusses the direct and indirect effects, the inherent uncertainties involved with the efficacy and efficiency numbers and remark on their comparison. In the second part of this presentation, Angie Ohler discusses the vital role of open access in advancing research on COVID-19

Datta, Jyotishka

Ohler, Angie

English

UARK (University of Arkansas )

University of Arkansas, Fayetteville ScholarWorks@UARK Vaccine Course Honors College Teaching and Learning 5-14-2021 Forum 5: Understanding Vaccine Efficacy and Effectiveness: A Statistician's Perspective/Open Access STEM research Jyotishka Datta Virginia Polytechnic Institute and State University Angie Ohler University of Arkansas, Fayetteville Follow this and additional works at: https://scholarworks.uark.edu/hnrcvac  Part of the Community Health and Preventive Medicine Commons, Influenza Virus Vaccines Commons, and the Public Health Education and Promotion Commons Citation Datta, J., & Ohler, A. (2021). Forum 5: Understanding Vaccine Efficacy and Effectiveness: A Statistician's Perspective/Open Access STEM research. Vaccine Course. Retrieved from https://scholarworks.uark.edu/hnrcvac/7 This Video is brought to you for free and open access by the Honors College Teaching and Learning at ScholarWorks@UARK. It has been accepted for inclusion in Vaccine Course by an authorized administrator of ScholarWorks@UARK. For more information, please contact scholar@uark.edu. Vaccine Efficacy & EffectivenessA Statistician's PerspectiveDr. Jyotishka Datta, Virginia Tech, Statistics."The inspiration of the camel image is that it represents the dedication of the world to bring vaccines to everyone."- Halloran et al. (2009), Design & Analysis of Vaccine StudiesOn May 10, 2021November 20, 2020, New York Times, Carl ZimmerEfficacy • Vaccine efficacy (and vaccine effectiveness), VE, are generally estimated as one minus some measure of relative risk, RR, in the vaccinated group compared to the unvaccinated group:VE = 1 - RR• The groups being compared could be composed of individuals or of populations or communities.Pfizer• Pfizer recruited 43,661 volunteers and waited for 170 people to come down with symptoms of Covid-19 and get a positive test.• Out of these 170, 162 were from the 'placebo’ group and just eight were from the ‘vaccine’ group.• (Groups are randomly allocated)GROUP GROUP SIZENUMBER INFECTEDINFECTION RISKUS POP.Placebo 21,830 162 162/21830= 0.74%2.5 millionVaccine 21830 8 8/21830 = 0.04%131,000Efficacy• Recall, VE = 1 – RR = 1 - (risk for vaccine group)/(risk for placebo)• Placebo group's infection risk: 0.74%• Vaccine group's infection risk: 0.04%• VE = 1 – 0.04/0.74 = (0.74 - 0.04)/0.74 = 0.95• This captures the difference in impact by scaling the percentage point difference in risks by the original infection risk.• What factors influence efficacy? How do we control for age, gender, ethnicity, co-morbidities etc.?Piero Olliaro, “What Does 95% COVID-19 Vaccine Efficacy Really Mean?” Quote from the letter• "The mRNA-based Pfizer and Moderna vaccines were shown to have 94–95% efficacy in preventing symptomatic COVID-19, calculated as 100 × (1 minus the attack rate with vaccine divided by the attack rate with placebo)."• "It means that in a population such as the one enrolled in the trials, with a cumulated COVID-19 attack rate over a period of 3 months of about 1% without a vaccine, we would expect roughly 0·05% of vaccinated people would get diseased."• "It does not mean that 95% of people are protected from disease with the vaccine—a general misconception of vaccine protection also found in a Lancet Infectious Diseases Editorial."Piero Olliaro, “What Does 95% COVID-19 Vaccine Efficacy Really Mean?” Digging deeperHistorical perspective Randomized control trialsCharacterizing UncertaintyEffectivenss vs EfficacyHistory of vaccinesThis Photo by Unknown author is licensed under CC BY-SA-NC.History of vaccines• The word vaccine was derived from Latin word 'vacca' for cow*, when English physician Edward Jenner introduced cowpox-based vaccine against smallpox in 1796.• Apparently, that story was probably not correct. As an NEJM article showed in 2017 (by analyzing historical containers), the vaccine used to prevent small-pox was horse-pox, and maybe ... we should have called it 'equusine'.Historical Perspectives"The fundamental logic behind today’s vaccine trials was worked out by statisticians over a century ago."Through the centuries• After nearly a 100-years of hiatus, at the end of the 19th century, inoculations against cholera, typhoid, plague (caused by bacteria) and rabies (caused by a virus) were developed• By the early 20th century, legendary statisticians Karl Pearson, Major Greenwood, and Udny Yule were deeply engaged in discussions of assessing these vaccines in the field.• 1920's: Pertussis, diptheria, tetanus, and bacille Calmette-Gu´erin against tuberculosis• 1930's: yellow fever, influenza, and rickettsia vaccines• Post-war: polio, measles, mumps, rubella, varicella, and adenovirus.Greenwood & Yule (1915) paper• Famous opening line:• “Hardly any subjects within the range of preventive medicine is of more immediate importance than the methods of prophylaxis which ought to be adopted with respect to typhoid fever and cholera”Why two tables?• Whether to “class as inoculated those who were so at the date of the last returnmade or only those actually inoculated at the time of arrival on the foreign station” ?Why two tables?• In the former case, there may be an exaggeration of the “number of men who were inoculated during the whole exposure to infection,”• In the latter case, one would underestimate it “because many inoculations were done shortly after arrival”• How to adjust for this effect?• Pre-date formal 'randomized studies'Poliomyelitis vaccine• In 1954, an enormous field study - total of 1,829,916 children participated in the nationwide study.• Observed control study.• "to administer vaccine to children in the second grade of school; the corresponding first and third graders would not be inoculated, but would be kept under observation for the occurrence of poliomyelitis in comparison with the inoculated second graders”• not a blinded study + effect of age might lead to bias !!Poliomyelitis vaccine• The plan was changed in mid-stream. In the second plan, called the Placebo Control Study, “children of the first, second, and third grades would be combined. One half would receive vaccine; the other matching half, serving as strict controls, would receive a solution of similar appearance (placebo)”• Despite flaws, this vaccine had 72% efficacy.• The Salk (injected) and Sabin (oral) polio vaccines have been 'transformative' - three polio virus strains has been eliminated in most countries of the world.Randomized Controlled TrialsRandomized clinical trial• Double-blinded randomized controlled trial (RCT)• Group of participants placed either in a control group or experimental, completely at random, people going in are not aware of which group they're in, neither the researchers.• The idea is that the experimental & control groups would be similar in terms of potential factors, such as, age, gender, ethnicity etc.• Keep in mind: There could be issues of ethics ("necessary to know if the vaccine was better than what was available at the time")• Other types of biases, too.Details from Pfizer's website"Efficacy was consistent across age, gender, race and ethnicity demographics; observed efficacy in adults over 65 years of age was over 94%""The Phase 3 clinical trial of BNT162b2 began on July 27 and has enrolled 43,661 participants to date, 41,135 of whom have received a second dose of the vaccine candidate as of November 13, 2020"."Approximately 42% of global participants and 30% of U.S. participants have racially and ethnically diverse backgrounds, and 41% of global and 45% of U.S. participants are 56-85 years of age."Efficacy depends on a lot of things!• Where? J&J conducted trials in US, Latin America, South Africa.• Overall efficacy lower than US-specific efficacy.• In South Africa, trials took place after a new variants B.1.351 emerged, affecting the efficacy.• But it didn't make it useless (SA efficacy ~ 64%)• Also, when do we look at outcomes? J&J has 85% efficacy against severe cases.Direct & Indirect effects• Safe & effective vaccine strategy offers both(1) direct protection (high-risk) and(2) indirect (reduce transmission for those in contact with high-risk).Marc Lipsitch and Natalie E. Dean, “Understanding COVID-19 Vaccine Efficacy”, Science. Strategy• Elderly & people with comorbidities are at greatest risk - age structured mathematical models.• Need to know well the vaccine works in which groups.• Phase 3 trials provide insights about individual level efficacy & safety.• However, assessing subgroup-specific efficacy is often challenging, and needs more work.• For example, blinded follow-up studies can provide evidence on long-term safety, efficacy & age-specific effects.UncertaintyUncertainty • The efficacy numbers you see (e.g., 95%) are point estimates.• For the general population, there will be 'uncertainty' -reflects the difference between the subjects under trial and the large population.• 95% confidence intervals.Uncertainty• One way to think about 95% CI's is that if you conduct 100 such similar studies, 95 of them would contain the efficacy value.• FDA's threshold: efficacy no less than 50% & lower limit of CI cannot be lower than 30%• Fortunately, all the major vaccines surpassed that.Uncertainty• The last thing to note about CI's is that if there's a large overlap between two CI's, then their difference is not statistically significant.• In case of vaccines, it means that the point estimates of the efficacy values might be different, but if the CI's overlap, their efficacies are not really distinguishable.• There are other factors as well.How do vaccines compare?• It is very difficult to compare vaccines.• Vaccines were tested on different groups of people, during different stages of pandemic.• They were also measured in different ways –e.g., J&J 28 days after a single dose, while Moderna 14 days after a second dose.• All these vaccines have a high efficacy against hospitalization & death.Image from Vox videoEffectivenessA clinical trial is not the final destination, but just a start.Researchers follow the effects of vaccine on the large popoulation for a long time.Then, the quantity to look at is called "effectiveness" - the relative reduction of risk in 'real world', millions or billions of people.Early studies show that the vaccines are also quite effective.Final remark"A tale of personal perseverance" • Katalin Karikó was dismissed, ignored, unable to get grants and demoted. Had a cancer scare & her husband was stuck in Hungary sorting out visa issues.• For three decades, she refused to quit.• Ask yourselves, why did it have to be this way, and how many Katalin Karikos have quit?Damian Garde, “The Story of MRNA: How a Once-Dismissed Idea Became a Leading Technology in the COVID Vaccine Race” References• Piero Olliaro, “What Does 95% COVID-19 Vaccine Efficacy Really Mean?” (Blackboard).• Dashiell Young-Saver, “What Does 95% Effective Mean? Teaching the Math of Vaccine Efficacy” (Blackboard).• Marc Lipsitch and Natalie E. Dean, “Understanding COVID-19 Vaccine Efficacy” (Blackboard).• Hodgson et al, “What Defines an Efficacious COVI-19 Vaccine?” (Blackboard).• Baden et al, “Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine” (Blackboard).• Zimmer and Collins, “What Do Vaccine Efficacy Numbers Actually Mean?” (Blackboard).• Kreps et al, “Factors Associated with US Adults’ Likelihood of Acceptng COVID-19 Vaccination” (Blackboard).• Damian Garde, “The Story of MRNA: How a Once-Dismissed Idea Became a Leading Technology in the COVID Vaccine Race” (Blackboard).• Halloran, M. Elizabeth, et al. Design and analysis of vaccine studies. Vol. 18. New York, NY: Springer, 2010.QuestionsThe Role of Open Access in COVID-19 Vaccine ResearchAngie Ohler, Associate Dean for Content and Digital InitiativesOverviewOpen Access• What• WhyCommunity Response• Open Science• ControversyFuture Directions• Public Discourse• Cultural ShiftsOpen AccessWhat is Open Access?• OPEN = IMPACT• Open Access Publishing• Open access (OA) refers to freely available, digital, online information. Open access scholarly literature is free of charge and often carries less restrictive copyright and licensing barriers than traditionally published works, for both the users and the authors. • While OA is a newer form of scholarly publishing, many OA journals comply with well-established peer-review processes and maintain high publishing standards.• Digital Repositories like ScholarWorks@UARK• Funding agencies are getting serious about compliance, and that means publicly funded research must be made available OAWhy Open Access?Higher visibility for University of Arkansas authored works, with higher citation rates and greater impact in the field and beyond.Publishing with established journals and presses using trusted peer review processes.Opportunities to establish a strong early career publication record for tenure track faculty and student authors.Helps those authors in disciplines who do not otherwise have grant funding to pay for publication fees.Community Response to the PandemicOpen Science• Public outcry from the scientific and research community• Taxpayer funded research should be freely available • Data sharing and open peer review• Full access to all research and publicationControversy• Quantity versus quality• Other models for peer review• The Lancet retraction • bioRxiv preprint server retractionFuture DirectionsPublic Discourse• Scholarly Publication in Historical Context• Societal Impact Cultural Shifts• Is this a permanent change?• Should it be?FAQ for  Open Access Publishing Fund• When will the program begin? July 1, 2021• Is this a one time ask? Yes, this is a pilot program, and we will be assessing it for continuing.  • Is there a plan for dedicated funding? Most research-intensive universities have had open access grant programs for a long time. These programs are typically centered within academic libraries and are funded every year by a combination of funding from partners across campus.• Will there be limits on the number of asks, the total amount of requests, the frequency of asks, etc.? We will limit the requests per author to one per year to make sure the funding covers as many authors as possible.• Open to students?  Yes, open to all institutional authors. • Are there caps on the grant amount? Open access grant funds at large universities typically cover between $1500 to $3000 per book or journal article funding request. We will cover up to $2000.• How will the funding be managed?  As with other research institutions, the Libraries will administer the open access funds and the program will have clearly defined rules as to who is eligible, what publications are covered, and guidelines for applying.  These will be posted on the Office of Scholarly Communications page.  Questions? 

Forum 5: Understanding Vaccine Efficacy and Effectiveness: A Statistician\u27s Perspective/Open Access STEM research

https://scholarworks.uark.edu/cgi/viewcontent.cgi?article=1005&context=hnrcvac

Forum 5: Understanding Vaccine Efficacy and Effectiveness: A Statistician\u27s Perspective/Open Access STEM research

Abstract

Similar works

Full text

Available Versions

UARK (University of Arkansas )