Large language models (LLMs) such as those embedded in 'chatbots' are
accelerating and democratizing research by providing comprehensible information
and expertise from many different fields. However, these models may also confer
easy access to dual-use technologies capable of inflicting great harm. To
evaluate this risk, the 'Safeguarding the Future' course at MIT tasked
non-scientist students with investigating whether LLM chatbots could be
prompted to assist non-experts in causing a pandemic. In one hour, the chatbots
suggested four potential pandemic pathogens, explained how they can be
generated from synthetic DNA using reverse genetics, supplied the names of DNA
synthesis companies unlikely to screen orders, identified detailed protocols
and how to troubleshoot them, and recommended that anyone lacking the skills to
perform reverse genetics engage a core facility or contract research
organization. Collectively, these results suggest that LLMs will make
pandemic-class agents widely accessible as soon as they are credibly
identified, even to people with little or no laboratory training. Promising
nonproliferation measures include pre-release evaluations of LLMs by third
parties, curating training datasets to remove harmful concepts, and verifiably
screening all DNA generated by synthesis providers or used by contract research
organizations and robotic cloud laboratories to engineer organisms or viruses.Comment: 6 pages, 0 figure