We explore the extension of chain-of-thought (CoT) prompting to medical
reasoning for the task of automatic diagnosis. Motivated by doctors' underlying
reasoning process, we present Diagnostic-Reasoning CoT (DR-CoT). Empirical
results demonstrate that by simply prompting large language models trained only
on general text corpus with two DR-CoT exemplars, the diagnostic accuracy
improves by 15% comparing to standard prompting. Moreover, the gap reaches a
pronounced 18% in out-domain settings. Our findings suggest expert-knowledge
reasoning in large language models can be elicited through proper promptings.Comment: Accepted as a Tiny Paper at ICLR 2023 (10 pages, 5 figures