Tensor completion is a natural higher-order generalization of matrix
completion where the goal is to recover a low-rank tensor from sparse
observations of its entries. Existing algorithms are either heuristic without
provable guarantees, based on solving large semidefinite programs which are
impractical to run, or make strong assumptions such as requiring the factors to
be nearly orthogonal. In this paper we introduce a new variant of alternating
minimization, which in turn is inspired by understanding how the progress
measures that guide convergence of alternating minimization in the matrix
setting need to be adapted to the tensor setting. We show strong provable
guarantees, including showing that our algorithm converges linearly to the true
tensors even when the factors are highly correlated and can be implemented in
nearly linear time. Moreover our algorithm is also highly practical and we show
that we can complete third order tensors with a thousand dimensions from
observing a tiny fraction of its entries. In contrast, and somewhat
surprisingly, we show that the standard version of alternating minimization,
without our new twist, can converge at a drastically slower rate in practice