The ability to jointly understand the geometry of objects and plan actions
for manipulating them is crucial for intelligent agents. We refer to this
ability as geometric planning. Recently, many interactive environments have
been proposed to evaluate intelligent agents on various skills, however, none
of them cater to the needs of geometric planning. We present PackIt, a virtual
environment to evaluate and potentially learn the ability to do geometric
planning, where an agent needs to take a sequence of actions to pack a set of
objects into a box with limited space. We also construct a set of challenging
packing tasks using an evolutionary algorithm. Further, we study various
baselines for the task that include model-free learning-based and
heuristic-based methods, as well as search-based optimization methods that
assume access to the model of the environment. Code and data are available at
https://github.com/princeton-vl/PackIt.Comment: Accepted to ICML 202
