Approximate message passing (AMP) refers to a class of efficient algorithms
for statistical estimation in high-dimensional problems such as compressed
sensing and low-rank matrix estimation. This paper analyzes the performance of
AMP in the regime where the problem dimension is large but finite. For
concreteness, we consider the setting of high-dimensional regression, where the
goal is to estimate a high-dimensional vector β0 from a noisy
measurement y=Aβ0+w. AMP is a low-complexity, scalable algorithm for
this problem. Under suitable assumptions on the measurement matrix A, AMP has
the attractive feature that its performance can be accurately characterized in
the large system limit by a simple scalar iteration called state evolution.
Previous proofs of the validity of state evolution have all been asymptotic
convergence results. In this paper, we derive a concentration inequality for
AMP with i.i.d. Gaussian measurement matrices with finite size n×N.
The result shows that the probability of deviation from the state evolution
prediction falls exponentially in n. This provides theoretical support for
empirical findings that have demonstrated excellent agreement of AMP
performance with state evolution predictions for moderately large dimensions.
The concentration inequality also indicates that the number of AMP iterations
t can grow no faster than order loglognlogn for the
performance to be close to the state evolution predictions with high
probability. The analysis can be extended to obtain similar non-asymptotic
results for AMP in other settings such as low-rank matrix estimation