Despite the successes of recent developments in visual AI, different
shortcomings still exist; from missing exact logical reasoning, to abstract
generalization abilities, to understanding complex and noisy scenes.
Unfortunately, existing benchmarks, were not designed to capture more than a
few of these aspects. Whereas deep learning datasets focus on visually complex
data but simple visual reasoning tasks, inductive logic datasets involve
complex logical learning tasks, however, lack the visual component. To address
this, we propose the visual logical learning dataset, V-LoL, that seamlessly
combines visual and logical challenges. Notably, we introduce the first
instantiation of V-LoL, V-LoL-Trains, -- a visual rendition of a classic
benchmark in symbolic AI, the Michalski train problem. By incorporating
intricate visual scenes and flexible logical reasoning tasks within a versatile
framework, V-LoL-Trains provides a platform for investigating a wide range of
visual logical learning challenges. We evaluate a variety of AI systems
including traditional symbolic AI, neural AI, as well as neuro-symbolic AI. Our
evaluations demonstrate that even state-of-the-art AI faces difficulties in
dealing with visual logical learning challenges, highlighting unique advantages
and limitations specific to each methodology. Overall, V-LoL opens up new
avenues for understanding and enhancing current abilities in visual logical
learning for AI systems