In this note we describe a method for recursively estimating the depth of a scene from a sequence of images. The input to the estimator are brightness values at a number of locations of a grid in a video image, and the output is the relative (scaled) depth corresponding to each image-point. The estimator is invariant with respect to the motion of the viewer, in the sense that the motion parameters are not part of the state of the estimator and therefore the estimates do not depend on motion as long as there is enough parallax (the translational velocity is nonzero). This scheme is a "direct" version of an other algorithm previously presented by the authors for estimating depth from point-feature correspondence independent of motion
