Multi-Layer Local Graph Words for Object Recognition

In this paper, we propose a new multi-layer structural approach for the task of object based image retrieval. In our work we tackle the problem of structural organization of local features. The structural features we propose are nested multi-layered local graphs built upon sets of SURF feature points with Delaunay triangulation. A Bag-of-Visual-Words (BoVW) framework is applied on these graphs, giving birth to a Bag-of-Graph-Words representation. The multi-layer nature of the descriptors consists in scaling from trivial Delaunay graphs - isolated feature points - by increasing the number of nodes layer by layer up to graphs with maximal number of nodes. For each layer of graphs its own visual dictionary is built. The experiments conducted on the SIVAL and Caltech-101 data sets reveal that the graph features at different layers exhibit complementary performances on the same content and perform better than baseline BoVW approach. The combination of all existing layers, yields significant improvement of the object recognition performance compared to single level approaches.Comment: International Conference on MultiMedia Modeling, Klagenfurt : Autriche (2012

Sistema de clasificación y reconocimiento de imágenes

Mediante descriptores, se pueden definir los puntos clave que caracterizan una imagen cualquiera, los cuales luego podrán ser localizados en otras escenas en las que existen rotaciones, cambios de escala e iluminación y oclusiones parciales. De esta forma se podrá realizar la búsqueda automática de objetos en distintas imágenes. Durante el desarrollo de este proyecto de grado, se realizará un estudio de diferentes métodos de extracción de características de las imágenes y la utilización de dichos métodos en la implementación de un sistema de clasificación y reconocimiento de imágenes. Para la implementación del sistema de clasificación y reconocimiento de imágenes utilizando la técnica de Bolsa de Palabras Visuales (BofVW), máquinas de vector de soporte y descriptores, inicialmente se parte de la implementación de diversas técnicas para hallar puntos de interés y descriptores sobre algunas imágenes de la base de datos de imágenes levantada por el autor. En segunda instancia se implementaron varios esquemas de clasificación y reconocimiento aplicando los descriptores SIFT y SURF, y realizando la comparación de puntos de interés entre las imágenes para hallar las coincidencias(Esquema de detección de puntos de interés y búsqueda de coincidencias entre imágenes); Primero se hizo la comparación entre dos imágenes y luego una imagen contra un conjunto de imágenes de una base de datos(Esquema de búsqueda de objetos específicos, en un conjunto de imágenes a partir de su grado de coincidencia.). Luego se implementó el sistema de clasificación y reconocimiento de imágenes utilizando la bolsa de palabras visuales (BoVW). Para esta fase del proyecto se implementó el sistema de clasificación de imágenes utilizando bolsa de características personalizada. Sistemas CBIR (CBIR- Sistemas basados en contenido para la recuperación de imágenes) y el sistema de clasificación de imágenes utilizando bolsa de palabras visuales, máquinas de vector de soporte y descriptores (SIFT y SURF)

Image Classification Based On Bag Of Visual Graphs

This paper proposes the Bag of Visual Graphs (BoVG), a new approach to encode the spatial relationships of visual words through a codebook of visual-word arrangements, represented by graphs. This graph-based codebook defines a descriptor for image representations that not only considers the frequency of occurrence of visual words, but also their spatial relationships. Experiments demonstrate that BoVG yields high-accuracy scores in classification tasks on the traditional Caltech-101 and Caltech-256 datasets. © 2013 IEEE.43124316The Institute of Electrical and Electronics Engineers (IEEE) Signal Processing SocietyLazebnik, S., Schmid, C., Ponce, J., Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories (2006) Proc. of IEEE Conf. on Computer Vision and Pattern RecognitionPenatti, O.A.B., Valle, E., Da Torres S, R., Encoding spatial arrangement of visual words (2011) Proc. of Iberoamerican Cong. in Pattern Recognition (CIARP)Boureau, Y.-L., Bach, F., Lecun, Y., Ponce, J., Learning mid-level features for recognition (2010) Proc. IEEE Conf. on Computer Vision and Pattern RecognitionRocha, A., Carvalho, T., Jelinek, H., Goldenstein, S., Wainer, J., Points of interest and visual dictionaries for automatic retinal lesion detection (2012) IEEE Transactions on Biomedical Engineering, 59 (8), pp. 2244-2253Cao, Y., Wang, C., Li, Z., Zhang, L., Zhang, L., Spatial-bag-of-features (2010) Proc of IEEE Conf. on Computer Vision and Pattern RecognitionKaraman, S., Jenny, B.-P., Megret, R., Bugeau, A., Multi-layer local graph words for object recognition (2012) Proc. of Intl. Conf.On Advances in Multimedia Modeling (MMM)Welling, M., Weber, M., Perona, P., Unsupervised learning of models for recognition (2000) Proc. European Conf. Computer VisionFergus, R., Perona, P., Zisserman, A., Object class recognition by unsupervised scale-invariant learning (2003) Proc. of IEEE Conf. Computer Vision and Pattern RecognitionMikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L., A comparison of affine region detectors (2005) Int. J. Comput. Vision, 65 (1-2), pp. 43-72Lowe, D.G., Distinctive image features from scale-invariant keypoints (2004) Int. Journal of Computer Vision, 60 (2), pp. 91-110Van De Sande, K.E.A., Gevers, T., Snoek, C.G.M., Evaluating color descriptors for object and scene recognition (2010) IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (9), pp. 1582-1596Cai, H., Yan, F., Mikolajczyk, K., Learning weights for codebook in image classification and retrieval (2010) Proc of IEEE Conf. Computer Vision and Pattern RecognitionVan Gemert, J.C., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.-M., Visual word ambiguity (2010) IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (7), pp. 1271-1283Viitaniemi, V., Laaksonen, J., Experiments on selection of codebooks for local image feature histograms (2008) Proc. Intl. Conf. on Visual Information Systems: Web-Based Visual Information Search and ManagementHashimoto, M., Cesar Jr., R.M., Object detection by keygraph classification (2009) Proc. of the Intl. Workshop on Graph-Based Representations in Pattern RecognitionJouili, S., Mili, I., Tabbone, S., Attributed graph matching using local descriptions (2009) ACIVS. 5807 of Lecture Notes in Computer Science, pp. 89-99. , SpringerOjala, T., Pietikainen, M., Maenpaa, T., Multiresolution gray-scale and rotation invariant texture classification with local binary patterns (2002) IEEE Transactions on Pattern Analysis and Machine Intelligence, 24 (7), pp. 971-987Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T., Discovering objects and their location in images (2005) IEEE International Conference on Computer VisionLi, F.-F., Fergus, R., Perona, P., Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories (2007) Computer Vision and Image Understanding, 106, pp. 59-70Griffin, G., Holub, A., Perona, P., (2007) Caltech-256 Object Category Dataset, , Tech. Rep. 7694, California Institute of TechnologyVan De Sande, K.E.A., Gevers, T., Snoek, C.G.M., Empowering visual categorization with the gpu (2011) IEEE Transactions on Multimedia, 13 (1), pp. 60-70Chang, C.-C., Lin, C.-J., LIBSVM: A library for support vector machines (2011) ACM Transactions on Intelligent Systems and Technology, 2, pp. 271-2727Huang, T.-K., Weng, R.C., Lin, C.-J., Generalized bradley-terry models and multi-class probability estimates (2006) J. Mach. Learn. Res, 7, pp. 85-115. , De

Visual Word Spatial Arrangement For Image Retrieval And Classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization. © 2013 Elsevier Ltd.472705720Sivic, J., Zisserman, A., Video google: A text retrieval approach to object matching in videos (2003) International Conference on Computer Vision, 2, pp. 1470-1477Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L., A comparison of affine region detectors (2005) International Journal of Computer Vision, 65, pp. 43-72Van De Sande, K.E.A., Gevers, T., Snoek, C.G.M., Evaluating color descriptors for object and scene recognition (2010) Transactions on Pattern Analysis and Machine Intelligence, 32 (9), pp. 1582-1596Lowe, D.G., Distinctive image features from scale-invariant keypoints (2004) International Journal of Computer Vision, 60 (2), pp. 91-110Fei-Fei, L., Fergus, R., Perona, P., Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories (2004) Conference on Computer Vision and Pattern Recognition Workshop, 12, p. 178Andreopoulos, A., Tsotsos, J.K., 50 years of object recognition: Directions forward (2013) Computer Vision and Image Understanding, 117 (8), pp. 827-891Penatti, O.A.B., Valle, E., Torres, R.D.S., Encoding spatial arrangement of visual words (2011) Iberoamerican Congress on Pattern Recognition, 7042, pp. 240-247Hoàng, N.V., Gouet-Brunet, V., Rukoz, M., Manouvrier, M., Embedding spatial information into image content description for scene retrieval (2010) Pattern Recognition, 43, pp. 3013-3024Lazebnik, S., Schmid, C., Ponce, J., Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories (2006) Conference on Computer Vision and Pattern Recognition, 2, pp. 2169-2178Feng, J., Ni, B., Tian, Q., Yan, S., Geometric lp-norm feature pooling for image classification (2011) Conference on Computer Vision and Pattern Recognition, pp. 2609-2704Cao, Y., Wang, C., Li, Z., Zhang, L., Zhang, L., Spatial-bag-of-features (2010) Conference on Computer Vision and Pattern Recognition, pp. 3352-3359Zhou, W., Lu, Y., Li, H., Song, Y., Tian, Q., Spatial coding for large scale partial-duplicate web image search (2010) International Conference on Multimedia, pp. 511-520Jégou, H., Douze, M., Schmid, C., Improving bag-of-features for large scale image search (2010) International Journal of Computer Vision, 87, pp. 316-336Weber, R., Schek, H., Blott, S., A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces (1998) International Conference on Very Large Data Bases, pp. 194-205Traina, J.C., Traina, A., Faloutsos, C., Seeger, B., Fast indexing and visualization of metric data sets using slim-trees (2002) Transactions on Knowledge and Data Engineering, 14 (2), pp. 244-260Kang, H., Hebert, M., Kanade, T., Image matching with distinctive visual vocabulary (2011) IEEE Workshop on Applications of Computer Vision, pp. 402-409Van Gemert, J.C., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.-M., Visual word ambiguity (2010) Transactions on Pattern Analysis and Machine Intelligence, 32, pp. 1271-1283Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A., Lost in quantization: Improving particular object retrieval in large scale image databases (2008) Conference on Computer Vision and Pattern Recognition, pp. 1-8Liu, L., Wang, L., Liu, X., In defense of soft-assignment coding (2011) International Conference on Computer Vision, pp. 1-8Penatti, O.A.B., Torres, R.D.S., Eva-an evaluation tool for comparing descriptors in content-based image retrieval tasks (2010) International Conference on Multimedia Information Retrieval, pp. 413-416Viitaniemi, V., Laaksonen, J., Experiments on selection of codebooks for local image feature histograms (2008) International Conference on Visual Information Systems: Web-Based Visual Information Search and Management, pp. 126-137Jurie, F., Triggs, B., Creating efficient codebooks for visual recognition (2005) International Conference on Computer Vision, 1, pp. 604-610Boureau, Y.-L., Bach, F., Lecun, Y., Ponce, J., Learning mid-level features for recognition (2010) Conference on Computer Vision and Pattern Recognition, pp. 2559-2566Jegou, H., Douze, M., Schmid, C., Hamming embedding and weak geometric consistency for large scale image search (2008) European Conference on Computer Vision. Part i, 5302, pp. 304-317Avila, S., Bossa: Extended bow formalism for image classification (2011) International Conference on Image Processing, pp. 2966-2969Perronnin, F., Improving the fisher kernel for large-scale image classification (2010) European Conference on Computer Vision, 6314, pp. 143-156Mbanya, E., Gerke, S., Ndjiki-Nya, P., Spatial codebooks for image categorization (2011) International Conference on Multimedia Retrieval, pp. 501-507Karaman, S., Multi-layer local graph words for object recognition (2012) Advances in Multimedia Modeling, 7131, pp. 29-39Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R., Content-based image retrieval at the end of the early years (2000) Transactions on Pattern Analysis and Machine Intelligence, 22 (12), pp. 1349-1380Huang, J., Kumar, S.R., Mitra, M., Zhu, W., Zabih, R., Image indexing using color correlograms (1997) Conference on Computer Vision and Pattern Recognition, p. 762Pass, G., Zabih, R., Miller, J., Comparing images using color coherence vectors (1996) ACM Multimedia, pp. 65-73Zhou, W., Li, H., Lu, Y., Tian, Q., Large scale image search with geometric coding (2011) ACM Multimedia, pp. 1349-1352Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T., Discovering objects and their location in images (2005) International Conference on Computer Vision, 1, pp. 370-377Savarese, S., Winn, J., Criminisi, A., Discriminative object class models of appearance and shape by correlations (2006) Conference on Computer Vision and Pattern Recognition, 2, pp. 2033-2040Jegou, H., Douze, M., Schmid, C., Perez, P., Aggregating local descriptors into a compact image representation (2010) Conference on Computer Vision and Pattern Recognition, pp. 3304-3311Torralba, A., Efros, A.A., Unbiased look at dataset bias (2011) Conference on Computer Vision and Pattern Recognition, pp. 1521-152