Modelling semantic context for novelty detection in wildlife scenes

Novelty detection is an important functionality that has found many applications in information retrieval and processing. In this paper we propose a novel framework that deals with novelty detection for multiple-scene image sets. Working with wildlife image data, the framework starts with image segmentation, followed by feature extraction and classification of the image blocks extracted from image segments. The labelled image blocks are then scanned through to generate a co-occurrence matrix of object labels, representing the semantic context within the scene. The semantic co-occurrence matrices then undergo binarization and principal component analysis for dimension reduction, forming the basis for constructing one-class models for each scene category. An algorithm for outlier detection that employs multiple one-class models is proposed. An advantage of our approach is that it can be used for scene classification and novelty detection at the same time. Our experiments show that the proposed approach algorithm gives favourable performance for the task of detecting novel wildlife scenes, and binarization of the label co-occurrence matrices helps to significantly increase the robustness in dealing with the variation of scene statistics.UnpublishedD.E. Berlyne. Stimulus Selection and Conflict. McGraw-Hill Book Company, 1960. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993–1022, 2003. A. Bosch, A. Zisserman, and X. Munoz. Scene classification via pLSA. In Proceedings of the European Conference on Computer Vision, pages 517–530, 2006. V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Computing Surveys, 41:1–58, 2009. Christine Connolly. Wildlife-spotting robots. Sensor Reviews, 27:282–287, 2007. Jia Deng, Wei Dong, R. Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, Los Alamitos, CA, USA, 2009. IEEE Computer Society. Y. Deng, , and B.S. Manjunath. Unsupervised segmentation of color-texture regions in images and video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(8):800–810, Aug 2001. Mark J. Fenske, Elissa Aminoff, Nurit Gronau, and Moshe Bar. Top-down facilitation of visual object recognition: object-based and context-based contributions. Progress in Brain Research, 155:3–21, 2006. R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman. Learning object categories from google’s image search. In International Conference on Computer Vision, volume 2, pages 1816–1823, 2005. L. Fei-Fei and P. Perona. A bayesian hierarchical model for learning natural scence categories. In IEEE Conference on Computer Vision and Pattern Recognition, pages 524–531, 2005. Evgeniy Gabrilovich, Susan Dumais, and Eric Horvitz. Newsjunkie: providing personalized newsfeeds via analysis of information novelty. In WWW’04: Proceedings of the 13th international conference on World Wide Web, pages 482–490, New York, NY, USA, 2004. ACM. K. Hempstalk, E. Frank, and I.H. Witten. One-class classification by combining density and class probability estimation. In Proc. ECML/PKDD’08, volume 5211 of Lecture Notes in Computer Science, pages 505–519, Berlin, September 2008. Springer. R. M. Haralick, K. Shanmugam, and I. Dinstein. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, 3:610–621, 1973. F. Jing, M. Li, L. Zhang, H. Zhang, and B. Zhang. Learning in region-based image retrieval. In International Conference on Image and Video Retrieval, Urbana-Champaign, Illinois, 2003. H. Katti, K. Y. Bin, T. S. Chua, and M. Kankanhalli. Pre-attentive discrimination of interestingness in images. In International Conference of Multimedia and Expo (ICME), Hannover, Germany, June 23-26, 2008. Shehzad Khalid. Motion-based behaviour learning, profiling and classification in the presence of anomalies. Pattern Recognition, 43(1):173 – 186, 2010. Xiaoyan Li and W. Bruce Croft. An information-pattern-based approach to novelty detection. Information Processing and Management, 44(3):1159 – 1188, 2008. L-J. Li, R. Socher, and L. Fei-Fei. Towards total scene understanding:classification, annotation and segmentation in an automatic framework. In Proc. IEEE Computer Vision and Pattern Recognition (CVPR), pages 2036–2043, 2009. B. S. Manjunath, J. R. Ohm, V. V. Vasudevan, and A. Yamada. Color and texture descriptors. Circuits and Systems for Video Technology, IEEE Transactions on, 11(6):703–715, 2001. G. Manson, G. Pierce, and K. Worden. On the long-term stability of normal condition for damage detection in a composite panel. Key Engineering Materials, 204-205:359–370, 2001. Markos Markou and Sameer Singh. Novelty detection: a review—part 1: statistical approaches. Signal Process., 83(12):2481–2497, 2003. Yu-Fei Ma and Hong-Jiang Zhang. Contrast-based image attention analysis by using fuzzy growing. In Multimedia’03: Proceedings of the 11th ACM International Conference on Multimedia, pages 374–381, New York, NY, USA, 2003. ACM. Dragoljub Pokrajac, Aleksandar Lazarevic, and Longin Jan Latecki. Incremental local outlier detection for data streams. In Proceedings of IEEE Symposium on Computational Intelligence and Data Mining, pages 504–515, 2007. Animesh Patcha and Jung-Min Park. An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer Networks, 51(12):3448 – 3470, 2007. Paul J. Silvia. Exploring the Psychology of Interest, volume 56. Oxford University Press, 2006. J. A. Stirk and G. Underwood. Low-level visual saliency does not predict change detection in natural scences. Journal of Vision, 7(10):3:1–10, 2007. J. Wang, J. Li, and G. Wiederhold. Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(9):947–963, 2001

Modelling semantic context for novelty detection in wildlife scenes

Novelty detection is an important functionality that has found many applications in information retrieval and processing. In this paper we propose a novel framework that deals with novelty detection for multiple-scene image sets. Working with wildlife image data, the framework starts with image segmentation, followed by feature extraction and classification of the image blocks extracted from image segments. The labelled image blocks are then scanned through to generate a co-occurrence matrix of object labels, representing the semantic context within the scene. The semantic co-occurrence matrices then undergo binarization and principal component analysis for dimension reduction, forming the basis for constructing one-class models for each scene category. An algorithm for outlier detection that employs multiple one-class models is proposed. An advantage of our approach is that it can be used for scene classification and novelty detection at the same time. Our experiments show that the proposed approach algorithm gives favourable performance for the task of detecting novel wildlife scenes, and binarization of the label co-occurrence matrices helps to significantly increase the robustness in dealing with the variation of scene statistics.UnpublishedD.E. Berlyne. Stimulus Selection and Conflict. McGraw-Hill Book Company, 1960. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993–1022, 2003. A. Bosch, A. Zisserman, and X. Munoz. Scene classification via pLSA. In Proceedings of the European Conference on Computer Vision, pages 517–530, 2006. V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Computing Surveys, 41:1–58, 2009. Christine Connolly. Wildlife-spotting robots. Sensor Reviews, 27:282–287, 2007. Jia Deng, Wei Dong, R. Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, Los Alamitos, CA, USA, 2009. IEEE Computer Society. Y. Deng, , and B.S. Manjunath. Unsupervised segmentation of color-texture regions in images and video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(8):800–810, Aug 2001. Mark J. Fenske, Elissa Aminoff, Nurit Gronau, and Moshe Bar. Top-down facilitation of visual object recognition: object-based and context-based contributions. Progress in Brain Research, 155:3–21, 2006. R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman. Learning object categories from google’s image search. In International Conference on Computer Vision, volume 2, pages 1816–1823, 2005. L. Fei-Fei and P. Perona. A bayesian hierarchical model for learning natural scence categories. In IEEE Conference on Computer Vision and Pattern Recognition, pages 524–531, 2005. Evgeniy Gabrilovich, Susan Dumais, and Eric Horvitz. Newsjunkie: providing personalized newsfeeds via analysis of information novelty. In WWW’04: Proceedings of the 13th international conference on World Wide Web, pages 482–490, New York, NY, USA, 2004. ACM. K. Hempstalk, E. Frank, and I.H. Witten. One-class classification by combining density and class probability estimation. In Proc. ECML/PKDD’08, volume 5211 of Lecture Notes in Computer Science, pages 505–519, Berlin, September 2008. Springer. R. M. Haralick, K. Shanmugam, and I. Dinstein. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, 3:610–621, 1973. F. Jing, M. Li, L. Zhang, H. Zhang, and B. Zhang. Learning in region-based image retrieval. In International Conference on Image and Video Retrieval, Urbana-Champaign, Illinois, 2003. H. Katti, K. Y. Bin, T. S. Chua, and M. Kankanhalli. Pre-attentive discrimination of interestingness in images. In International Conference of Multimedia and Expo (ICME), Hannover, Germany, June 23-26, 2008. Shehzad Khalid. Motion-based behaviour learning, profiling and classification in the presence of anomalies. Pattern Recognition, 43(1):173 – 186, 2010. Xiaoyan Li and W. Bruce Croft. An information-pattern-based approach to novelty detection. Information Processing and Management, 44(3):1159 – 1188, 2008. L-J. Li, R. Socher, and L. Fei-Fei. Towards total scene understanding:classification, annotation and segmentation in an automatic framework. In Proc. IEEE Computer Vision and Pattern Recognition (CVPR), pages 2036–2043, 2009. B. S. Manjunath, J. R. Ohm, V. V. Vasudevan, and A. Yamada. Color and texture descriptors. Circuits and Systems for Video Technology, IEEE Transactions on, 11(6):703–715, 2001. G. Manson, G. Pierce, and K. Worden. On the long-term stability of normal condition for damage detection in a composite panel. Key Engineering Materials, 204-205:359–370, 2001. Markos Markou and Sameer Singh. Novelty detection: a review—part 1: statistical approaches. Signal Process., 83(12):2481–2497, 2003. Yu-Fei Ma and Hong-Jiang Zhang. Contrast-based image attention analysis by using fuzzy growing. In Multimedia’03: Proceedings of the 11th ACM International Conference on Multimedia, pages 374–381, New York, NY, USA, 2003. ACM. Dragoljub Pokrajac, Aleksandar Lazarevic, and Longin Jan Latecki. Incremental local outlier detection for data streams. In Proceedings of IEEE Symposium on Computational Intelligence and Data Mining, pages 504–515, 2007. Animesh Patcha and Jung-Min Park. An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer Networks, 51(12):3448 – 3470, 2007. Paul J. Silvia. Exploring the Psychology of Interest, volume 56. Oxford University Press, 2006. J. A. Stirk and G. Underwood. Low-level visual saliency does not predict change detection in natural scences. Journal of Vision, 7(10):3:1–10, 2007. J. Wang, J. Li, and G. Wiederhold. Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(9):947–963, 2001