Difference-based methods have been attracting increasing attention in
nonparametric regression, in particular for estimating the residual variance.To
implement the estimation, one needs to choose an appropriate difference
sequence, mainly between {\em the optimal difference sequence} and {\em the
ordinary difference sequence}. The difference sequence selection is a
fundamental problem in nonparametric regression, and it remains a controversial
issue for over three decades. In this paper, we propose to tackle this
challenging issue from a very unique perspective, namely by introducing a new
difference sequence called {\em the optimal-k difference sequence}. The new
difference sequence not only provides a better balance between the
bias-variance trade-off, but also dramatically enlarges the existing family of
difference sequences that includes the optimal and ordinary difference
sequences as two important special cases. We further demonstrate, by both
theoretical and numerical studies, that the optimal-k difference sequence has
been pushing the boundaries of our knowledge in difference-based methods in
nonparametric regression, and it always performs the best in practical
situations