Effective and efficient visual description based on local binary patterns and gradient distribution for object recognition

Cette thèse est consacrée au problème de la reconnaissance visuelle des objets basé sur l'ordinateur, qui est devenue un sujet de recherche très populaire et important ces dernières années grâce à ses nombreuses applications comme l'indexation et la recherche d'image et de vidéo , le contrôle d'accès de sécurité, la surveillance vidéo, etc. Malgré beaucoup d'efforts et de progrès qui ont été fait pendant les dernières années, il reste un problème ouvert et est encore considéré comme l'un des problèmes les plus difficiles dans la communauté de vision par ordinateur, principalement en raison des similarités entre les classes et des variations intra-classe comme occlusion, clutter de fond, les changements de point de vue, pose, l'échelle et l'éclairage. Les approches populaires d'aujourd'hui pour la reconnaissance des objets sont basé sur les descripteurs et les classiffieurs, ce qui généralement extrait des descripteurs visuelles dans les images et les vidéos d'abord, et puis effectue la classification en utilisant des algorithmes d'apprentissage automatique sur la base des caractéristiques extraites. Ainsi, il est important de concevoir une bonne description visuelle, qui devrait être à la fois discriminatoire et efficace à calcul, tout en possédant certaines propriétés de robustesse contre les variations mentionnées précédemment. Dans ce contexte, l objectif de cette thèse est de proposer des contributions novatrices pour la tâche de la reconnaissance visuelle des objets, en particulier de présenter plusieurs nouveaux descripteurs visuelles qui représentent effectivement et efficacement le contenu visuel d image et de vidéo pour la reconnaissance des objets. Les descripteurs proposés ont l'intention de capturer l'information visuelle sous aspects différents. Tout d'abord, nous proposons six caractéristiques LBP couleurs de multi-échelle pour traiter les défauts principaux du LBP original, c'est-à-dire, le déffcit d'information de couleur et la sensibilité aux variations des conditions d'éclairage non-monotoniques. En étendant le LBP original à la forme de multi-échelle dans les différents espaces de couleur, les caractéristiques proposées non seulement ont plus de puissance discriminante par l'obtention de plus d'information locale, mais possèdent également certaines propriétés d'invariance aux différentes variations des conditions d éclairage. En plus, leurs performances sont encore améliorées en appliquant une stratégie de l'image division grossière à fine pour calculer les caractéristiques proposées dans les blocs d'image afin de coder l'information spatiale des structures de texture. Les caractéristiques proposées capturent la distribution mondiale de l information de texture dans les images. Deuxièmement, nous proposons une nouvelle méthode pour réduire la dimensionnalité du LBP appelée la combinaison orthogonale de LBP (OC-LBP). Elle est adoptée pour construire un nouveau descripteur local basé sur la distribution en suivant une manière similaire à SIFT. Notre objectif est de construire un descripteur local plus efficace en remplaçant l'information de gradient coûteux par des patterns de texture locales dans le régime du SIFT. Comme l'extension de notre première contribution, nous étendons également le descripteur OC-LBP aux différents espaces de couleur et proposons six descripteurs OC-LBP couleurs pour améliorer la puissance discriminante et la propriété d'invariance photométrique du descripteur basé sur l'intensité. Les descripteurs proposés capturent la distribution locale de l information de texture dans les images. Troisièmement, nous introduisons DAISY, un nouveau descripteur local rapide basé sur la distribution de gradient, dans le domaine de la reconnaissance visuelle des objets. [...]This thesis is dedicated to the problem of machine-based visual object recognition, which has become a very popular and important research topic in recent years because of its wide range of applications such as image/video indexing and retrieval, security access control, video monitoring, etc. Despite a lot of e orts and progress that have been made during the past years, it remains an open problem and is still considered as one of the most challenging problems in computer vision community, mainly due to inter-class similarities and intra-class variations like occlusion, background clutter, changes in viewpoint, pose, scale and illumination. The popular approaches for object recognition nowadays are feature & classifier based, which typically extract visual features from images/videos at first, and then perform the classification using certain machine learning algorithms based on the extracted features. Thus it is important to design good visual description, which should be both discriminative and computationally efficient, while possessing some properties of robustness against the previously mentioned variations. In this context, the objective of this thesis is to propose some innovative contributions for the task of visual object recognition, in particular to present several new visual features / descriptors which effectively and efficiently represent the visual content of images/videos for object recognition. The proposed features / descriptors intend to capture the visual information from different aspects. Firstly, we propose six multi-scale color local binary pattern (LBP) features to deal with the main shortcomings of the original LBP, namely deficiency of color information and sensitivity to non-monotonic lighting condition changes. By extending the original LBP to multi-scale form in different color spaces, the proposed features not only have more discriminative power by obtaining more local information, but also possess certain invariance properties to different lighting condition changes. In addition, their performances are further improved by applying a coarse-to-fine image division strategy for calculating the proposed features within image blocks in order to encode spatial information of texture structures. The proposed features capture global distribution of texture information in images. Secondly, we propose a new dimensionality reduction method for LBP called the orthogonal combination of local binary patterns (OC-LBP), and adopt it to construct a new distribution-based local descriptor by following a way similar to SIFT.Our goal is to build a more efficient local descriptor by replacing the costly gradient information with local texture patterns in the SIFT scheme. As the extension of our first contribution, we also extend the OC-LBP descriptor to different color spaces and propose six color OC-LBP descriptors to enhance the discriminative power and the photometric invariance property of the intensity-based descriptor. The proposed descriptors capture local distribution of texture information in images. Thirdly, we introduce DAISY, a new fast local descriptor based on gradient distribution, to the domain of visual object recognition.LYON-Ecole Centrale (690812301) / SudocSudocFranceF

Malware Image Classification using Machine Learning with Local Binary Pattern

Malware classification is a critical part in the cybersecurity. Traditional methodologies for the malware classification typically use static analysis and dynamic analysis to identify malware. In this paper, a malware classification methodology based on its binary image and extracting local binary pattern (LBP) features are proposed. First, malware images are reorganized into 3 by 3 grids which is mainly used to extract LBP feature. Second, the LBP is implemented on the malware images to extract features in that it is useful in pattern or texture classification. Finally, Tensorflow, a library for machine learning, is applied to classify malware images with the LBP feature. Performance comparison results among different classifiers with different image descriptors such as GIST, a spatial envelope, and the LBP demonstrate that our proposed approach outperforms others

Query Region Determination based on Region Importance Index and Relative Position for Region-based Image Retrieval

An efficient Region-Based Image Retrieval (RBIR) system must consider query region determination techniques and target regions in the retrieval process. A query region is a region that must contain a Region of Interest (ROI) or saliency region. A query region determination can be specified manually or automatically. However, manual determination is considered less efficient and tedious for users. The selected query region must determine specific target regions in the image collection to reduce the retrieval time. This study proposes a strategy of query region determination based on the Region Importance Index (RII) value and relative position of the Saliency Region Overlapping Block (SROB) to produce a more efficient RBIR. The entire region is formed by using the mean shift segmentation method. The RII value is calculated based on a percentage of the region area and region distance to the center of the image. Whereas the target regions are determined by considering the relative position of SROB, the performance of the proposed method is tested on a CorelDB dataset. Experimental results show that the proposed method can reduce the Average of Retrieval Time to 0.054 seconds with a 5x5 block size configuration

Fusion Iris and Periocular Recognitions in Non-Cooperative Environment

The performance of iris recognition in non-cooperative environment can be negatively impacted when the resolution of the iris images is low which results in failure to determine the eye center, limbic and pupillary boundary of the iris segmentation. Hence, a combination with periocular features is suggested to increase the authenticity of the recognition system. However, the texture feature of periocular can be easily affected by a background complication while the colour feature of periocular is still limited to spatial information and quantization effects. This happens due to different distances between the sensor and the subject during the iris acquisition stage as well as image size and orientation. The proposed method of periocular feature extraction consists of a combination of rotation invariant uniform local binary pattern to select the texture features and a method of color moment to select the color features. Besides, a hue-saturation-value channel is selected to avoid loss of discriminative information in the eye image. The proposed method which consists of combination between texture and colour features provides the highest accuracy for the periocular recognition with more than 71.5% for the UBIRIS.v2 dataset and 85.7% for the UBIPr dataset. For the fusion recognitions, the proposed method achieved the highest accuracy with more than 85.9% for the UBIRIS.v2 dataset and 89.7% for the UBIPr dataset

Performances of the LBP based algorithm over CNN models for detecting crops and weeds with similar morphologies

Weed invasions pose a threat to agricultural productivity. Weed recognition and detection play an important role in controlling weeds. The challenging problem of weed detection is how to discriminate between crops and weeds with a similar morphology under natural field conditions such as occlusion, varying lighting conditions, and different growth stages. In this paper, we evaluate a novel algorithm, filtered Local Binary Patterns with contour masks and coefficient k (k-FLBPCM), for discriminating between morphologically similar crops and weeds, which shows significant advantages, in both model size and accuracy, over state-of-the-art deep convolutional neural network (CNN) models such as VGG-16, VGG-19, ResNet-50 and InceptionV3. The experimental results on the “bccr-segset” dataset in the laboratory testbed setting show that the accuracy of CNN models with fine-tuned hyper-parameters is slightly higher than the k-FLBPCM method, while the accuracy of the k-FLBPCM algorithm is higher than the CNN models (except for VGG-16) for the more realistic “fieldtrip_can_weeds” dataset collected from real-world agricultural fields. However, the CNN models require a large amount of labelled samples for the training process. We conducted another experiment based on training with crop images at mature stages and testing at early stages. The k-FLBPCM method outperformed the state-of-the-art CNN models in recognizing small leaf shapes at early growth stages, with error rates an order of magnitude lower than CNN models for canola–radish (crop–weed) discrimination using a subset extracted from the “bccr-segset” dataset, and for the “mixed-plants” dataset. Moreover, the real-time weed–plant discrimination time attained with the k-FLBPCM algorithm is approximately 0.223 ms per image for the laboratory dataset and 0.346 ms per image for the field dataset, and this is an order of magnitude faster than that of CNN models

A Novel Adaptive LBP-Based Descriptor for Color Image Retrieval

In this paper, we present two approaches to extract discriminative features for color image retrieval. The proposed local texture descriptors, based on Radial Mean Local Binary Pattern (RMLBP), are called Color RMCLBP (CRMCLBP) and Prototype Data Model (PDM). RMLBP is a robust to noise descriptor which has been proposed to extract texture features of gray scale images for texture classification. For the first descriptor, the Radial Mean Completed Local Binary Pattern is applied to channels of the color space, independently. Then, the final descriptor is achieved by concatenating the histogram of the CRMCLBP_S/M/C component of each channel. Moreover, to enhance the performance of the proposed method, the Particle Swarm Optimization (PSO) algorithm is used for feature weighting. The second proposed descriptor, PDM, uses the three outputs of CRMCLBP (CRMCLBP_S, CRMCLBP_M, CRMCLBP_C) as discriminative features for each pixel of a color image. Then, a set of representative feature vectors are selected from each image by applying k-means clustering algorithm. This set of selected prototypes are compared by means of a new similarity measure to find the most relevant images. Finally, the weighted versions of PDM is constructed using PSO algorithm. Our proposed methods are tested on Wang, Corel-5k, Corel-10k and Holidays datasets. The results show that our proposed methods makes an admissible tradeoff between speed and retrieval accuracy. The first descriptor enhances the state-of-the-art color texture descriptors in both aspects. The second one is a very fast retrieval algorithm which extracts discriminative features

Image region description using orthogonal combination of local binary patterns enhanced with color information

International audienceVisual content description is a key issue for machine-based image analysis and understanding. A good visual descriptor should be both discriminative and computationally efficient while possessing some properties of robustness to viewpoint changes and lighting condition variations. In this paper, we propose a new operator called the orthogonal combination of local binary patterns (denoted as OC-LBP) and six new local descriptors based on OC-LBP enhanced with color information for image region description. The aim is to increase both discriminative power and photometric invariance properties of the original LBP operator while keeping its computational efficiency. The experiments in three different applications show that the proposed descriptors outperform the popular SIFT, CS-LBP, HOG and SURF, and achieve comparable or even better performances than the state-of-the-art color SIFT descriptors. Meanwhile, the proposed descriptors provide complementary information to color SIFT, because a fusion of these two kinds of descriptors is found to perform clearly better than either of the two separately. Moreover, the proposed descriptors are about 4 times faster to compute than color SIFT

Image region description using orthogonal combination of local binary patterns enhanced with color information

International audienceVisual content description is a key issue for machine-based image analysis and understanding. A good visual descriptor should be both discriminative and computationally efficient while possessing some properties of robustness to viewpoint changes and lighting condition variations. In this paper, we propose a new operator called the orthogonal combination of local binary patterns (denoted as OC-LBP) and six new local descriptors based on OC-LBP enhanced with color information for image region description. The aim is to increase both discriminative power and photometric invariance properties of the original LBP operator while keeping its computational efficiency. The experiments in three different applications show that the proposed descriptors outperform the popular SIFT, CS-LBP, HOG and SURF, and achieve comparable or even better performances than the state-of-the-art color SIFT descriptors. Meanwhile, the proposed descriptors provide complementary information to color SIFT, because a fusion of these two kinds of descriptors is found to perform clearly better than either of the two separately. Moreover, the proposed descriptors are about 4 times faster to compute than color SIFT

Local Binary Pattern based algorithms for the discrimination and detection of crops and weeds with similar morphologies

In cultivated agricultural fields, weeds are unwanted species that compete with the crop plants for nutrients, water, sunlight and soil, thus constraining their growth. Applying new real-time weed detection and spraying technologies to agriculture would enhance current farming practices, leading to higher crop yields and lower production costs. Various weed detection methods have been developed for Site-Specific Weed Management (SSWM) aimed at maximising the crop yield through efficient control of weeds. Blanket application of herbicide chemicals is currently the most popular weed eradication practice in weed management and weed invasion. However, the excessive use of herbicides has a detrimental impact on the human health, economy and environment. Before weeds are resistant to herbicides and respond better to weed control strategies, it is necessary to control them in the fallow, pre-sowing, early post-emergent and in pasture phases. Moreover, the development of herbicide resistance in weeds is the driving force for inventing precision and automation weed treatments. Various weed detection techniques have been developed to identify weed species in crop fields, aimed at improving the crop quality, reducing herbicide and water usage and minimising environmental impacts. In this thesis, Local Binary Pattern (LBP)-based algorithms are developed and tested experimentally, which are based on extracting dominant plant features from camera images to precisely detecting weeds from crops in real time. Based on the efficient computation and robustness of the first LBP method, an improved LBP-based method is developed based on using three different LBP operators for plant feature extraction in conjunction with a Support Vector Machine (SVM) method for multiclass plant classification. A 24,000-image dataset, collected using a testing facility under simulated field conditions (Testbed system), is used for algorithm training, validation and testing. The dataset, which is published online under the name “bccr-segset”, consists of four subclasses: background, Canola (Brassica napus), Corn (Zea mays), and Wild radish (Raphanus raphanistrum). In addition, the dataset comprises plant images collected at four crop growth stages, for each subclass. The computer-controlled Testbed is designed to rapidly label plant images and generate the “bccr-segset” dataset. Experimental results show that the classification accuracy of the improved LBP-based algorithm is 91.85%, for the four classes. Due to the similarity of the morphologies of the canola (crop) and wild radish (weed) leaves, the conventional LBP-based method has limited ability to discriminate broadleaf crops from weeds. To overcome this limitation and complex field conditions (illumination variation, poses, viewpoints, and occlusions), a novel LBP-based method (denoted k-FLBPCM) is developed to enhance the classification accuracy of crops and weeds with similar morphologies. Our contributions include (i) the use of opening and closing morphological operators in pre-processing of plant images, (ii) the development of the k-FLBPCM method by combining two methods, namely, the filtered local binary pattern (LBP) method and the contour-based masking method with a coefficient k, and (iii) the optimal use of SVM with the radial basis function (RBF) kernel to precisely identify broadleaf plants based on their distinctive features. The high performance of this k-FLBPCM method is demonstrated by experimentally attaining up to 98.63% classification accuracy at four different growth stages for all classes of the “bccr-segset” dataset. To evaluate performance of the k-FLBPCM algorithm in real-time, a comparison analysis between our novel method (k-FLBPCM) and deep convolutional neural networks (DCNNs) is conducted on morphologically similar crops and weeds. Various DCNN models, namely VGG-16, VGG-19, ResNet50 and InceptionV3, are optimised, by fine-tuning their hyper-parameters, and tested. Based on the experimental results on the “bccr-segset” dataset collected from the laboratory and the “fieldtrip_can_weeds” dataset collected from the field under practical environments, the classification accuracies of the DCNN models and the k-FLBPCM method are almost similar. Another experiment is conducted by training the algorithms with plant images obtained at mature stages and testing them at early stages. In this case, the new k-FLBPCM method outperformed the state-of-the-art CNN models in identifying small leaf shapes of canola-radish (crop-weed) at early growth stages, with an order of magnitude lower error rates in comparison with DCNN models. Furthermore, the execution time of the k-FLBPCM method during the training and test phases was faster than the DCNN counterparts, with an identification time difference of approximately 0.224ms per image for the laboratory dataset and 0.346ms per image for the field dataset. These results demonstrate the ability of the k-FLBPCM method to rapidly detect weeds from crops of similar appearance in real time with less data, and generalize to different size plants better than the CNN-based methods