Performing tasks in a physical environment is a crucial yet challenging
problem for AI systems operating in the real world. Physics simulation-based
tasks are often employed to facilitate research that addresses this challenge.
In this paper, first, we present a systematic approach for defining a physical
scenario using a causal sequence of physical interactions between objects.
Then, we propose a methodology for generating tasks in a physics-simulating
environment using these defined scenarios as inputs. Our approach enables a
better understanding of the granular mechanics required for solving
physics-based tasks, thereby facilitating accurate evaluation of AI systems'
physical reasoning capabilities. We demonstrate our proposed task generation
methodology using the physics-based puzzle game Angry Birds and evaluate the
generated tasks using a range of metrics, including physical stability,
solvability using intended physical interactions, and accidental solvability
using unintended solutions. We believe that the tasks generated using our
proposed methodology can facilitate a nuanced evaluation of physical reasoning
agents, thus paving the way for the development of agents for more
sophisticated real-world applications.Comment: The 19th AAAI Conference on Artificial Intelligence and Interactive
Digital Entertainment (AIIDE-23