A significant challenge facing current optical flow methods is the difficulty
in generalizing them well to the real world. This is mainly due to the high
cost of hand-crafted datasets, and existing self-supervised methods are limited
by indirect loss and occlusions, resulting in fuzzy outcomes. To address this
challenge, we introduce a novel optical flow training framework: automatic data
factory (ADF). ADF only requires RGB images as input to effectively train the
optical flow network on the target data domain. Specifically, we use advanced
Nerf technology to reconstruct scenes from photo groups collected by a
monocular camera, and then calculate optical flow labels between camera pose
pairs based on the rendering results. To eliminate erroneous labels caused by
defects in the scene reconstructed by Nerf, we screened the generated labels
from multiple aspects, such as optical flow matching accuracy, radiation field
confidence, and depth consistency. The filtered labels can be directly used for
network supervision. Experimentally, the generalization ability of ADF on KITTI
surpasses existing self-supervised optical flow and monocular scene flow
algorithms. In addition, ADF achieves impressive results in real-world
zero-point generalization evaluations and surpasses most supervised methods.Comment: 8 page