We propose a method to stabilize illumination color estimation by using the idea of an “Illumination line segment”. It is a finite line where every illumination color exists in the color space. By restricting the estimated illumination colors to be on this line, the method succeeds in making the estimation more robust and accurate. The method is based on our past work, while it uses direct calculation as opposed to the former method that use an iterative process. Experiments show the effectiveness of our method. 