Techniques for image classification, object detection and object segmentation

In this paper we document the techniques which we used to participate in the PASCAL NoE VOC Challenge 2007 image analysis performance evaluation campaign. We took part in three of the image analysis competitions: image classification, object detection and object segmentation. In the classification task of the evaluation our method produced comparatively good performance, the 4th best of 19 submissions. In contrast, our detection results were quite modest. Our method's segmentation accuracy was the best of all submissions. Our approach for the classification task is based on fused classifications by numerous global image features, including histograms of local features. The object detection combines similar classification of automatically extracted image segments and the previously obtained scene type classifications. The object segmentations are obtained in a straightforward fashion from the detection results

Code Detection from Control Image

Práce se zabývá problematikou rozpoznání kontrolního kódu v obrázku. Uvádí související oblasti ze zpracování obrazu, mezi které patří odstranění šumu, prahování, barevné modely, segmentace objektů a OCR. Práce dokumentuje výhody a nedostatky dvou vybraných metod segmentace objektů a navrhuje vlastní systém segmentace objektů. Dále je popsán navržený systém pro segmentaci a klasifikaci objektů.Work deals with code detection from control image. The document presents relevant image processing techniques dealing with a noise reduction, thresholding, color models, object segmentation and OCR. This project examines advantages and disadvantages of two selected methods for object segmentation and introduces developed system for object segmentation. The developed system for object segmentation and classification is realized, evaluated and results are discussed in details.

Object Detection in High Resolution Aerial Images and Hyperspectral Remote Sensing Images

With rapid developments in satellite and sensor technologies, there has been a dramatic increase in the availability of remotely sensed images. However, the exploration of these images still involves a tremendous amount of human interventions, which are tedious, time-consuming, and inefficient. To help imaging experts gain a complete understanding of the images and locate the objects of interest in a more accurate and efficient way, there is always an urgent need for developing automatic detection algorithms. In this work, we delve into the object detection problems in remote sensing applications, exploring the detection algorithms for both hyperspectral images (HSIs) and high resolution aerial images. In the first part, we focus on the subpixel target detection problem in HSIs with low spatial resolutions, where the objects of interest are much smaller than the image pixel spatial resolution. To this end, we explore the detection frameworks that integrate image segmentation techniques in designing the matched filters (MFs). In particular, we propose a novel image segmentation algorithm to identify the spatial-spectral coherent image regions, from which the background statistics were estimated for deriving the MFs. Extensive experimental studies were carried out to demonstrate the advantages of the proposed subpixel target detection framework. Our studies show the superiority of the approach when comparing to state-of-the-art methods. The second part of the thesis explores the object based image analysis (OBIA) framework for geospatial object detection in high resolution aerial images. Specifically, we generate a tree representation of the aerial images from the output of hierarchical image segmentation algorithms and reformulate the object detection problem into a tree matching task. We then proposed two tree-matching algorithms for the object detection framework. We demonstrate the efficiency and effectiveness of the proposed tree-matching based object detection framework. In the third part, we study object detection in high resolution aerial images from a machine learning perspective. We investigate both traditional machine learning based framework and end-to-end convolutional neural network (CNN) based approach for various object detection tasks. In the traditional detection framework, we propose to apply the Gaussian process classifier (GPC) to train an object detector and demonstrate the advantages of the probabilistic classification algorithm. In the CNN based approach, we proposed a novel scale transfer module that generates enhanced feature maps for object detection. Our results show the efficiency and competitiveness of the proposed algorithms when compared to state-of-the-art counterparts

Change detection of isolated housing using a new hybrid approach based on object classification with optical and TerraSAR-X data

Optical and microwave high spatial resolution images are now available for a wide range of applications. In this work, they have been applied for the semi-automatic change detection of isolated housing in agricultural areas. This article presents a new hybrid methodology based on segmentation of high-resolution images and image differencing. This new approach mixes the main techniques used in change detection methods and it also adds a final segmentation process in order to classify the change detection product. First, isolated building classification is carried out using only optical data. Then, synthetic aperture radar (SAR) information is added to the classification process, obtaining excellent results with lower complexity cost. Since the first classification step is improved, the total change detection scheme is also enhanced when the radar data are used for classification. Finally, a comparison between the different methods is presented and some conclusions are extracted from the study. © 2011 Taylor & Francis.Vidal Pantaleoni, A.; Moreno Cambroreno, MDR. (2011). Change detection of isolated housing using a new hybrid approach based on object classification with optical and TerraSAR-X data. International Journal of Remote Sensing. 32(24):9621-9635. doi:10.1080/01431161.2011.571297S962196353224BLAES, X., VANHALLE, L., & DEFOURNY, P. (2005). Efficiency of crop identification based on optical and SAR image time series. Remote Sensing of Environment, 96(3-4), 352-365. doi:10.1016/j.rse.2005.03.010Chen, Y., Shi, P., Fung, T., Wang, J., & Li, X. (2007). Object‐oriented classification for urban land cover mapping with ASTER imagery. International Journal of Remote Sensing, 28(20), 4645-4651. doi:10.1080/01431160500444731Dalla Mura, M., Benediktsson, J. A., Bovolo, F., & Bruzzone, L. (2008). An Unsupervised Technique Based on Morphological Filters for Change Detection in Very High Resolution Images. IEEE Geoscience and Remote Sensing Letters, 5(3), 433-437. doi:10.1109/lgrs.2008.917726Dell’Acqua, F., & Gamba, P. (2006). Discriminating urban environments using multiscale texture and multiple SAR images. International Journal of Remote Sensing, 27(18), 3797-3812. doi:10.1080/01431160600557572Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973). Textural Features for Image Classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6), 610-621. doi:10.1109/tsmc.1973.4309314Im, J., Jensen, J. R., & Tullis, J. A. (2008). Object‐based change detection using correlation image analysis and image segmentation. International Journal of Remote Sensing, 29(2), 399-423. doi:10.1080/01431160601075582Lhomme, S., He, D., Weber, C., & Morin, D. (2009). A new approach to building identification from very‐high‐spatial‐resolution images. International Journal of Remote Sensing, 30(5), 1341-1354. doi:10.1080/01431160802509017LOBO, A., CHIC, O., & CASTERAD, A. (1996). Classification of Mediterranean crops with multisensor data: per-pixel versus per-object statistics and image segmentation. International Journal of Remote Sensing, 17(12), 2385-2400. doi:10.1080/01431169608948779Lu, D., Mausel, P., Brondízio, E., & Moran, E. (2004). Change detection techniques. International Journal of Remote Sensing, 25(12), 2365-2401. doi:10.1080/0143116031000139863Shimabukuro, Y. E., Almeida‐Filho, R., Kuplich, T. M., & de Freitas, R. M. (2007). Quantifying optical and SAR image relationships for tropical landscape features in the Amazônia. International Journal of Remote Sensing, 28(17), 3831-3840. doi:10.1080/01431160701236829Stramondo, S., Bignami, C., Chini, M., Pierdicca, N., & Tertulliani, A. (2006). Satellite radar and optical remote sensing for earthquake damage detection: results from different case studies. International Journal of Remote Sensing, 27(20), 4433-4447. doi:10.1080/01431160600675895Yuan, D., & Elvidge, C. D. (1996). Comparison of relative radiometric normalization techniques. ISPRS Journal of Photogrammetry and Remote Sensing, 51(3), 117-126. doi:10.1016/0924-2716(96)00018-

Extending SSL patches spatial relations in Vision Transformers for object detection and instance segmentation tasks

Vision Transformer (ViT) architecture has become a de-facto standard in computer vision, achieving state-of-the-art performances in various tasks. This popularity is given by a remarkable computational efficiency and its global processing self-attention mechanism. However, in contrast with convolutional neural networks (CNNs), ViTs require large amounts of data to improve their generalization ability. In particular, for small datasets, their lack of inductive bias (i.e. translational equivariance, locality) can lead to poor results. To overcome the issue, SSL techniques based on the understanding of spatial relations among image patches without human annotations (e.g. positions, angles and euclidean distances) are extremely useful and easy to integrate in ViTs architecture. The correspondent model, dubbed RelViT, showed to improve overall image classification accuracy, optimizing tokens encoding and providing new visual representation of the data. This work proves the effectiveness of SSL strategies also for object detection and instance segmentation tasks. RelViT outperforms standard ViT architecture on multiple datasets in the majority of the related benchmarking metrics. In particular, testing on a small subset of COCO, results showed a gain of +2.70%, +2.20% in mAP for image segmentation and object detection respectively.Vision Transformer (ViT) architecture has become a de-facto standard in computer vision, achieving state-of-the-art performances in various tasks. This popularity is given by a remarkable computational efficiency and its global processing self-attention mechanism. However, in contrast with convolutional neural networks (CNNs), ViTs require large amounts of data to improve their generalization ability. In particular, for small datasets, their lack of inductive bias (i.e. translational equivariance, locality) can lead to poor results. To overcome the issue, SSL techniques based on the understanding of spatial relations among image patches without human annotations (e.g. positions, angles and euclidean distances) are extremely useful and easy to integrate in ViTs architecture. The correspondent model, dubbed RelViT, showed to improve overall image classification accuracy, optimizing tokens encoding and providing new visual representation of the data. This work proves the effectiveness of SSL strategies also for object detection and instance segmentation tasks. RelViT outperforms standard ViT architecture on multiple datasets in the majority of the related benchmarking metrics. In particular, testing on a small subset of COCO, results showed a gain of +2.70%, +2.20% in mAP for image segmentation and object detection respectively

ORGAN LOCALIZATION AND DETECTION IN SOW’S USING MACHINE LEARNING AND DEEP LEARNING IN COMPUTER VISION

The objective of computer vision research is to endow computers with human-like perception to enable the capability to detect their surroundings, interpret the data they sense, take appropriate actions, and learn from their experiences to improve future performance. The area has progressed from using traditional pattern recognition and image processing technologies to advanced techniques in image understanding such as model-based and knowledge-based vision. In the past few years there has been a surge of interest in machine learning algorithms for computer vision-based applications. Machine learning technology has the potential to significantly contribute to the development of flexible and robust vision algorithms that will improve the performance of practical vision systems with a higher level of competence and greater generality. Additionally, the development of machine learning-based architectures has the potential to reduce system development time while simultaneously achieving the above-stated performance improvements. This work proposes the utilization of a computer vision-based approach that leverages machine and deep learning systems to aid the detection and identification of sow reproduction cycles by segmentation and object detection techniques. A lightweight machine learning system is proposed for object detection to address dataset collection issues in one of the most crucial and potentially lucrative farming applications. This technique was designed to detect the vulvae region in pre-estrous sows using a single thermal image. In the first experiment, the support vector machine (SVM) classifier was used after extracting features determined by 12 Gabor filters. The features are then concatenated with the features obtained from the Histogram of oriented gradients (HOG) to produce the results of the first experiment. In the second experiment, the number of distinct Gabor filters used was increased from 12 to 96. The system is trained on cropped image windows and uses the Gaussian pyramid technique to look for the vulva in the input image. The resulting process is shown to be lightweight, simple, and robust when applied to and evaluated on a large number of images. The results from extensive qualitative and quantitative testing experiments are included. The experimental results include false detection, missing detection and favorable detection rates. The results indicate state-of-the-art accuracy. Additionally, the project was expanded by utilizing the You Only Look Once (YOLO) deep learning Object Detection models for fast object detection. The results from object detection have been used to label images for segmentation. The bounding box from the detected area was systematically colored to achieve the segmented and labeled images. Then these segmented images are used as custom data to train U-Net segmentation. The first step involves building a machine learning model using Gabor filters and HOG for feature extraction and SVM for classification. The results discovered the deficiency of the model, therefore a second stage was suggested in which the dataset was trained using YOLOv3-dependent deep learning object detection. The resulting segmentation model is found to be the best choice to aid the process of vulva localization. Since the model depends on the original gray-scale image and the mask of the region of interest (ROI), a custom dataset containing these features was obtained, augmented, and used to train a U-Net segmentation model. The results of the final approach shows that the proposed system can segment sow\u27s vulva region even in low rank images and has an excellent performance efficiency. Furthermore, the resulting algorithm can be used to improve the automation of estrous detection by providing reliable ROI identification and segmentation and enabling beneficial temporal change detection and tracking in future efforts

Object detection and segmentation using discriminative learning

Object detection and segmentation algorithms need to use prior knowledge of objects' shape and appearance to guide solutions to correct ones. A promising way of obtaining prior knowledge is to learn it directly from expert annotations by using machine learning techniques. Previous approaches commonly use generative learning approaches to achieve this goal. In this dissertation, I propose a series of discriminative learning algorithms based on boosting principles to learn prior knowledge from image databases with expert annotations. The learned knowledge improves the performance of detection and segmentation, leading to fast and accurate solutions. For object detection, I present a learning procedure called a Probabilistic Boosting Network (PBN) suitable for real-time object detection and pose estimation. Based on the law of total probability, PBN integrates evidence from two building blocks, namely a multiclass classifier for pose estimation and a detection cascade for object detection. Both the classifier and detection cascade employ boosting. By inferring the pose parameter, I avoid the exhaustive scan over pose parameters, which hampers real-time detection. I implement PBN using a graph-structured network that alternates the two tasks of object detection and pose estimation in an effort to reject negative cases as quickly as possible. Compared with previous approaches, PBN has higher accuracy in object localization and pose estimation with noticeable reduced computation. For object segmentation, I cast deformable object segmentation as optimizing the conditional probability density function p(C|I), where I is an image and C is a vector of model parameters describing the object shape. I propose a regression approach to learn the density p(C|I) discriminatively based on boosting principles. The learned density p(C|I) possesses a desired unimodal, smooth shape, which can be used by optimization algorithms to efficiently estimate a solution. To handle the high-dimensional learning challenges, I propose a multi-level approach and a gradient-based sampling strategy to learn regression functions efficiently. I show that the regression approach consistently outperforms state-of-the-art methods on a variety of testing datasets. Finally, I present a comparative study on how to apply three discriminative learning approaches - classification, regression, and ranking - to deformable shape segmentation. I discuss how to extend the idea of the regression approach to build discriminative models using classification and ranking. I propose sampling strategies to collect training examples from a high-dimensional model space for the classification and the ranking approach. I also propose a ranking algorithm based on Rankboost to learn a discriminative model for segmentation. Experimental results on left ventricle and left atrium segmentation from ultrasound images and facial feature localization demonstrate that the discriminative models outperform generative models and energy minimization methods by a large margin