The way people respond to messaging from public health organizations on
social media can provide insight into public perceptions on critical health
issues, especially during a global crisis such as COVID-19. It could be
valuable for high-impact organizations such as the US Centers for Disease
Control and Prevention (CDC) or the World Health Organization (WHO) to
understand how these perceptions impact reception of messaging on health policy
recommendations. We collect two datasets of public health messages and their
responses from Twitter relating to COVID-19 and Vaccines, and introduce a
predictive method which can be used to explore the potential reception of such
messages. Specifically, we harness a generative model (GPT-2) to directly
predict probable future responses and demonstrate how it can be used to
optimize expected reception of important health guidance. Finally, we introduce
a novel evaluation scheme with extensive statistical testing which allows us to
conclude that our models capture the semantics and sentiment found in actual
public health responses.Comment: Accepted at ACM WebSci 202