We consider the task of generating designs directly from natural language
descriptions, and consider floor plan generation as the initial research area.
Language conditional generative models have recently been very successful in
generating high-quality artistic images. However, designs must satisfy
different constraints that are not present in generating artistic images,
particularly spatial and relational constraints. We make multiple contributions
to initiate research on this task. First, we introduce a novel dataset,
\textit{Tell2Design} (T2D), which contains more than 80k floor plan designs
associated with natural language instructions. Second, we propose a
Sequence-to-Sequence model that can serve as a strong baseline for future
research. Third, we benchmark this task with several text-conditional image
generation models. We conclude by conducting human evaluations on the generated
samples and providing an analysis of human performance. We hope our
contributions will propel the research on language-guided design generation
forward.Comment: Paper published in ACL2023; Area Chair Award; Best Paper Nominatio