ABSTRACT: This paper addresses the problem of the efficient classification of images in a video stream in cases, where all of the video has to be labeled. Realizing the similarity of consecutive frames, we introduce a set of simple metrics to measure that similarity. To use these observations for decreasing the number of necessary classifications, we propose a scene segmentation algorithm. Performed experiments have evaluated the acquired scene sizes and classification accuracy resulting from the usage of different similarity metrics with our algorithm. As a result, we have identified those metrics from the considered set, which show the best characteristics for usage in scene segmentation