Light field cameras can provide rich angular and spatial information to
enhance image semantic segmentation for scene understanding in the field of
autonomous driving. However, the extensive angular information of light field
cameras contains a large amount of redundant data, which is overwhelming for
the limited hardware resource of intelligent vehicles. Besides, inappropriate
compression leads to information corruption and data loss. To excavate
representative information, we propose an Omni-Aperture Fusion model (OAFuser),
which leverages dense context from the central view and discovers the angular
information from sub-aperture images to generate a semantically-consistent
result. To avoid feature loss during network propagation and simultaneously
streamline the redundant information from the light field camera, we present a
simple yet very effective Sub-Aperture Fusion Module (SAFM) to embed
sub-aperture images into angular features without any additional memory cost.
Furthermore, to address the mismatched spatial information across viewpoints,
we present Center Angular Rectification Module (CARM) realized feature
resorting and prevent feature occlusion caused by asymmetric information. Our
proposed OAFuser achieves state-of-the-art performance on the UrbanLF-Real and
-Syn datasets and sets a new record of 84.93% in mIoU on the UrbanLF-Real
Extended dataset, with a gain of +4.53%. The source code of OAFuser will be
made publicly available at https://github.com/FeiBryantkit/OAFuser.Comment: The source code of OAFuser will be made publicly available at
https://github.com/FeiBryantkit/OAFuse