Although kernel approximation methods have been widely applied to mitigate the O(n3) cost of the n × n kernel matrix inverse in Gaussian process methods, they still face computational challenges. The ‘residual’ matrix between the covariance and the approximating component is often discarded as it prevents the computational cost reduction. In this paper, we propose a computationally efficient Gaussian process approach that achieves better computational efficiency, O(mn2), compared with standard Gaussian process methods, when using m n data. The proposed approach incorporates the ‘residual’ matrix in its symmetric diagonally dominant form which can be further approximated by the Neumann series. We have validated and compared the approach with full Gaussian process approaches and kernel approximation based Gaussian process variants, both on synthetic and real air quality data
