In this paper, we present a global scheme for encoding/ decoding natural video sequences with partial 3D models. This technique is based on a robust estimation of a constrained dense field and on the estimation of projection matrices for reference images. Then a sequential encoding of the sequence is performed in order to produce streamed 3D models. The paper presents the global scheme and focus on the most original steps of the encoding method. Some results on real video sequences are given to validate this approach. 1