This paper examines to what degree current deep learning architectures for
image caption generation capture spatial language. On the basis of the
evaluation of examples of generated captions from the literature we argue that
systems capture what objects are in the image data but not where these objects
are located: the captions generated by these systems are the output of a
language model conditioned on the output of an object detector that cannot
capture fine-grained location information. Although language models provide
useful knowledge for image captions, we argue that deep learning image
captioning architectures should also model geometric relations between objects.Comment: 15 pages, 10 figures, Appears in CLASP Papers in Computational
Linguistics Vol 1: Proceedings of the Conference on Logic and Machine
Learning in Natural Language (LaML 2017), pp. 41-5
