Real-time monocular 3D reconstruction is a challenging problem that remains
unsolved. Although recent end-to-end methods have demonstrated promising
results, tiny structures and geometric boundaries are hardly captured due to
their insufficient supervision neglecting spatial details and oversimplified
feature fusion ignoring temporal cues. To address the problems, we propose an
end-to-end 3D reconstruction network SST, which utilizes Sparse estimated
points from visual SLAM system as additional Spatial guidance and fuses
Temporal features via a novel cross-modal attention mechanism, achieving more
detailed reconstruction results. We propose a Local Spatial-Temporal Fusion
module to exploit more informative spatial-temporal cues from multi-view color
information and sparse priors, as well a Global Spatial-Temporal Fusion module
to refine the local TSDF volumes with the world-frame model from coarse to
fine. Extensive experiments on ScanNet and 7-Scenes demonstrate that SST
outperforms all state-of-the-art competitors, whilst keeping a high inference
speed at 59 FPS, enabling real-world applications with real-time requirements