For a given m>=1, we consider the finite non-abelian groups G for which
|C_G(g):|<=m for every g in G\Z(G). We show that the order of G can be
bounded in terms of m and the largest prime divisor of the order of G. Our
approach relies on dealing first with the case where G is a non-abelian finite
p-group. In that situation, if we take m=p^k to be a power of p, we show that
|G|<=p^{2k+2} with the only exception of Q_8. This bound is best possible, and
implies that the order of G can be bounded by a function of m alone in the case
of nilpotent groups