Towards key-frame extraction methods for 3D video: a review

The increasing rate of creation and use of 3D video content leads to a pressing need for methods capable of lowering the cost of 3D video searching, browsing and indexing operations, with improved content selection performance. Video summarisation methods specifically tailored for 3D video content fulfil these requirements. This paper presents a review of the state-of-the-art of a crucial component of 3D video summarisation algorithms: the key-frame extraction methods. The methods reviewed cover 3D video key-frame extraction as well as shot boundary detection methods specific for use in 3D video. The performance metrics used to evaluate the key-frame extraction methods and the summaries derived from those key-frames are presented and discussed. The applications of these methods are also presented and discussed, followed by an exposition about current research challenges on 3D video summarisation methods

An object-based approach to retrieval of image and video content

Promising new directions have been opened up for content-based visual retrieval in recent years. Object-based retrieval which allows users to manipulate video objects as part of their searching and browsing interaction, is one of these. It is the purpose of this thesis to constitute itself as a part of a larger stream of research that investigates visual objects as a possible approach to advancing the use of semantics in content-based visual retrieval. The notion of using objects in video retrieval has been seen as desirable for some years, but only very recently has technology started to allow even very basic object-location functions on video. The main hurdles to greater use of objects in video retrieval are the overhead of object segmentation on large amounts of video and the issue of whether objects can actually be used efficiently for multimedia retrieval. Despite this, there are already some examples of work which supports retrieval based on video objects. This thesis investigates an object-based approach to content-based visual retrieval. The main research contributions of this work are a study of shot boundary detection on compressed domain video where a fast detection approach is proposed and evaluated, and a study on the use of objects in interactive image retrieval. An object-based retrieval framework is developed in order to investigate object-based retrieval on a corpus of natural image and video. This framework contains the entire processing chain required to analyse, index and interactively retrieve images and video via object-to-object matching. The experimental results indicate that object-based searching consistently outperforms image-based search using low-level features. This result goes some way towards validating the approach of allowing users to select objects as a basis for searching video archives when the information need dictates it as appropriate

Video Fade Detection By Discrete Line Identification

The video segmentation problem can be regarded as a problem of detecting the fundamental video units (shots). Due to different ways of linking two consecutive shots this task turns out to be difficult. In this work, we propose a method to detect a type of gradual transition, the fade, by image segmentation tools instead of using dissimilarity measures or mathematical models. Firstly, the video is transformed into a 2D image considering the histogram information, called visual rhythm by histogram. Afterwards, we apply image processing tools to detect specified patterns in this image. © 2002 IEEE.16210131016Guimarães, S.J.F., Couprie, M., Leite, N.J., Araújo, A.A., A method for cut detection based on visual rhythm (2001) Proc. of the IEEE SIBGRAPI, pp. 297-304. , Brazil. ISBN 0769513301Del Bimbo, A., (1999) Visual Information Retrieval, , Morgan KaufmannZabih, R., Miller, J., Mai, K., Feature-based algorithms for detecting and classifying scene breaks (1995) ACM ICMCS, , USA, NovFernando, W.A.C., Canagarajah, C.N., Bull, D.R., Fade and dissolve detection in uncompressed and compressed video sequences (1999) Proc. of the IEEE ICIP, pp. 299-303Lienhart, R., Comparison of automatic shot boundary detection algorithms (1999) SPIE Image and Video Processing VII, 3656, pp. 290-301. , JanTonomura, Y., Akutsu, A., Otsuji, K., Sadakata, T., Videomap and videospaceicon: Tools for anatomizing video content (1993) ACM Interchi, pp. 131-136Chung, M.G., Lee, J., Kim, H., Song, S.M.-H., Kim, W.M., Automatic video segmentation based on spatio-temporal features (1999) Korea Telecom Journal, 4 (1), pp. 4-14Ngo, C.W., Pong, T.C., Chin, R.T., Detection of gradual transitions through temporal slice analysis (1999) Proc. of the IEEE CVPR, pp. 36-41Kong, T.Y., Rosenfeld, A., Digital topology: Introduction and survey (1989) CVGIP, 48, pp. 357-393Dunham, J.G., Optimum uniform piecewise linear aproximation of planar curves (1986) IEEE Trans. on PAMI, 8 (1), pp. 67-7

Video Segementation Based On 2d Image Analysis

The video segmentation problem consists in the identification of the boundary between consecutive shots. The common approach to solve this problem is based on dissimilarity measures between frames. In this work, the video segmentation problem is transformed into a problem of pattern detection, where each video event is transformed into a different pattern on a 2D image, called visual rhythm, obtained by a specific transformation. In our analysis we use topological and morphological tools to detect cuts. Also, we use discrete line analysis and max tree analysis to detect fade transitions and flashes, respectively. We present a comparative analysis of our method for cut detection with respect to some other methods, which shows the better results of our method. © 2002 Elsevier Science B.V. All rights reserved.247947957Bertrand, G., Everat, J.-C., Couprie, M., Image segmentation through operators based upon topology (1997) J. Electron. Imaging, 6, pp. 395-405Chung, M.G., Lee, J., Kim, H., Song, S.M.-H., Kim, W.M., Automatic video segmentation based on spatiotemporal features (1999) Korea Telecom J., 4 (1), pp. 4-14Del Bimbo, A., (1999) Visual Information Retrieval, , Morgan Kaufmann, Los Altos, CADel Bimbo, A., Pala, P., Tanganelli, L., Retrieval of commercials based on dynamics of color flows (2000) J. Visual Lang. Comput., 11, pp. 273-285Demarty, C.-H., (2000) Segmentation Et Structuration D'un Document Vidéo Pour La Caractérisation Et L'Indexation De Son Contenu Sémantiqué, , Ph.D. Thesis, École Nationale Supérieure des Mines de ParisDunham, J.G., Optimum uniform piecewise linear approximation of planar curves (1986) IEEE Trans. PAMI, 8 (1), pp. 67-75Fernando, W.A.C., Canagarajah, C.N., Bull, D.R., Fade and dissolve detection in uncompressed and compressed video sequences (1999) Proc. of the IEEE ICIP, pp. 299-303Guimaraes, S.J.F., Couprie, M., Leite, N.J., Araújo, A.A., Amethod for cut detection based on visual rhythm (2001) Proc. of the XIV Brazilian Symposium on Computer Graphics and Image Processing--SIBGRAPI, pp. 297-304. , Brazil, IEEE Computer Society Press ISBN 0769513301Lienhart, R., Comparison of automatic shot boundary detection algorithms (1999) SPIE Image and Video Processing, 7, pp. 25-30Ngo, C.W., Pong, T.C., Chin, R.T., Detection of gradual transitions through temporal slice analysis (1999) Proc. of the IEEE CVPR, pp. 36-41Salembier, P., Oliveras, A., Garrido, L., Antiextensive connected operators for image and sequence processing (1998) IEEE Trans. Image Process., 7 (4), pp. 555-570Serra, J., (1988) Image Analysis and Mathematical Morphology: Theoretical Advances, 2. , Academic Press, New YorkSoille, P., (1999) Morphological Image Analysis, , Springer, BerlinTonomura, Y., Akutsu, A., Otsuji, K., Sadakata, T., Videomap and videospaceicon: Tools for anatomizing video content (1993) ACM Interchi, pp. 131-136Wang, Y., Liu, Z., Huang, J.-C., Multimedia content analysis (2000) IEEE Signal Process. Mag., pp. 12-36Zabih, R., Miller, J., Mai, K., Feature-based algorithms for detecting and classifying scene breaks (1995) ACM ICMCS, pp. 12-13. , US