We present a method for creating 3D obstacle maps in real-time using only a monocular camera and an inertial measurement unit (IMU). We track a large amount of sparse features in the image frame. Then, given scale-accurate pose estimates from a front-end visual-inertial odometry (VIO) algorithm, we estimate the inverse depth to each of the tracked features using a keyframe-based feature-only bundle adjustment. These features are then accumulated within a probabilistic robocentric 3D voxel map that rolls as the camera moves. This local rolling voxel map provides a simple scene representation within which obstacle avoidance planning can easily be done. Our system is capable of running at camera frame rate on a laptop CPU
