In this work, a novel content-based image retrieval (CBIR) method is presented. It has been implemented and run on “Qatris
IManager” [14], a system belonging to SICUBO S.L. (spin-off from University of Extremadura, Spain). The system offers
some innovative visual content search tools for image retrieval from databases. It searches, manages and classifies images
using four kinds of features: colour, texture, shape and user description.
In a typical CBIR system, query results are a set of images sorted by feature similarities with respect to the query. However,
images with high feature similarities to the query may be very different from the query in terms of semantics. This discrepancy
between low-level features and high-level concepts is known as the semantic gap.
The search method presented here, is a novel supervised image retrieval method, based in Bayesian Logistic Regression, which
uses the information from the characteristics extracted from the images and from the user’s opinion who sets up the search. The
procedure of search and learning is based on a statistical method of aggregation of preferences given by Arias-Nicolás et al. [1]
and is useful in problems with both a large number of characteristics and few images.
The method could be specially helpful for those professionals who have to make a decision based in images, such as doctors to
determine the diagnosis of patients, meteorologists, traffic police to detect license plate, etc

Arias-Nicolás, J. P.

Calle-Alonso, F.

English

DSpace at University of West Bohemia

POSTER: A novel Content-based Image Retrieval systembased on Bayesian Logistic RegressionJ.P. Arias-NicolásUniversity of Extremadura, Spainjparias@unex.esF. Calle-AlonsoUniversity of Extremadura, Spainfcalonso@unex.esABSTRACTIn this work, a novel content-based image retrieval (CBIR) method is presented. It has been implemented and run on “QatrisIManager” [14], a system belonging to SICUBO S.L. (spin-off from University of Extremadura, Spain). The system offerssome innovative visual content search tools for image retrieval from databases. It searches, manages and classifies imagesusing four kinds of features: colour, texture, shape and user description.In a typical CBIR system, query results are a set of images sorted by feature similarities with respect to the query. However,images with high feature similarities to the query may be very different from the query in terms of semantics. This discrepancybetween low-level features and high-level concepts is known as the semantic gap.The search method presented here, is a novel supervised image retrieval method, based in Bayesian Logistic Regression, whichuses the information from the characteristics extracted from the images and from the user’s opinion who sets up the search. Theprocedure of search and learning is based on a statistical method of aggregation of preferences given by Arias-Nicolás et al. [1]and is useful in problems with both a large number of characteristics and few images.The method could be specially helpful for those professionals who have to make a decision based in images, such as doctors todetermine the diagnosis of patients, meteorologists, traffic police to detect license plate, etc.Keywords: Computer Vision, CBIR, Logistic Regression, Similarity, Pairwise Comparisons.1 INTRODUCTIONResearch in Content-Based Image Retrieval (CBIR) istoday a very active discipline, concentrating on depthissues, such as learning or management access to infor-mation content in images. Two fundamental problemsremain largely unsolved: how to best learn from users’query concepts, and how to measure perceptual simi-larity. A very popular framework in the 1970s was tofirst annotate the images by text, and then to use text-based Database Management Systems (DBMS) to per-form image retrieval. The essential difficulty in thismethod results from rich content in the images and sub-jectivity in human perception: the same image contentmay be perceived differently by other users, [16].Classification techniques are usually applied in CBIRsystems. Image categorization contributes to performmore effective searches. In the repertoire of imagesunder consideration there is a gradual distinction be-tween narrow and broad domains. A broad domain hasan unlimited and unpredictable variability in its appear-ance even for the same semantic meaning [15]. Thegood performance of classifiers has been proved whenthe image domain is specific, i.e, it is a narrow domainwhich has a limited and predictable variability in rele-vant aspects for the specific purpose, [6, 7].Actually, there are lots of works directly related toCBIR, and QBIC system is one of the first [8]. It wasdeveloped by IBM Corporation and is commerciallyavailable. This kind of CBIR systems focus on workwith images over a broad domain. Another system isPicHunter [5] which uses Bayesian learning based on aprobabilistic model of user’s behaviour. This systemworks with features from images and introduce rele-vance feedback from the users’ opinions. Most recentlyLiu et al. [12] focused on a powerful feature selectionmethod always addressed to cover the semantic gap;however, these authors do not develop a good methodto analyze similarity, but do so for metric distance.When classification methods are applied to general-purpose image collections the results are not positive,even less if we hope that the performance of the clas-sifier match with the classification developed by non-expert humans. We find some examples in [17, 18].We thus aim to develop a pairwise comparisonmethod based on the Binary Logistic Regression inorder to determine the images that can match onelacking some information. The proposed frameworkis based on [1], which focused on how to aggregatepersonal preferences to arrive at an optimal groupdecision. We are interested in searching for similarimages with respect to several features, when weonly know the similarity between some pairs of theimages. The method for CBIR proposed, combininginformation of both computational features and user’sknowledge.2 THE METHOD2.1 Feature ExtractionEach image is represented by a feature vector of fea-tures. We have considered three kinds of features:color, texture and shape features.WSCG 2010 Poster papers 19• Color Features. The color features used in thiswork are based on HLS model (Hue, Saturation, Lu-minosity), since the human perception is quite simi-lar to this model. On the other hand, local color fea-tures are used in order to achieve information aboutthe spatial distribution [4].• Texture Features. They have been obtained apply-ing two well known methods. The first one workson a global processing of images, it is based on theGray Level Co-ocurrenceMatrix proposed by Haral-ick [10]. The second method is focused on detectingonly linear texture primitives. It is based on featuresobtained from Run Length Matrix proposed by [9].• Shape Features. The images are processed us-ing Active Contours as segmentation method andthen some shape features are obtained from thesecontours. Shape features are based on Hu’s mo-ments(first and second moments), centroid (centerof gravity), angle of minimum inertia, area, perime-ter, ratio of area and perimeter, and major and minoraxis of fitted ellipse. The methods to obtain thesefeatures are explained in [3].2.2 ClasificationOnce features vectors are obtained, we apply our pro-posed search method. An efficient images supervisedclassification method based on bayesian logistic regres-sion has been implemented, which stands out for hishigh probability of wise move and his facility of incor-porating the user’s opinion in the learning phase.The method needs a training stage and a posteriortesting one. In the first phase, we require a set of pre-viously classified images to determine the pattern thatwill be used in the phase of test for classification of newimages. One of the important advantages of using thebayesan logistic regression, with regard to others meth-ods, is to obtain a progressive process of the reliabilityof classifier incorporating the user’s opinions and cor-rections in the successive phases of learning.The multinomial logistic regression model is adirect generalization of the binary logistic regressionfor K classes. So, we can classify a new elementx in a class k ∈ {1,2, . . . ,K}, for what we can as-sign to it a K-dimensional vector with values 0− 1,y = (y1,y2, . . . ,yK)t where yk = 1 and the othersones 0. Multinomial logistic regression is a model ofconditional probability of the form:p(yk = 1|x, B) = e(Btkx)K∑i=1e(Btix), (1)and standardize by the matrix B = (B1,B2, . . . ,BK).Every column of B is a vector of param-eters corresponding to each of the classes:Bk = (βk1,βk2, . . . ,βkm)t .The most widely used bayesian approach to themodel of logistic regression is to impose a gaussiandistribution with mean 0 and variance σ2k j, for everyparameter βk j ([2, 13]):p(βk j|σk j) = 1√2piσk jexp(−β 2k j2σ2k j). (2)The classification of a new image is based on the vec-tor of conditional probabilities estimated by the model.For this, simply the image is assigned to the class withthe highest estimated probability. The maximum like-lihood estimation of the parameters B is equivalent tomaximize:L(B) = l(B|X)+ ln p(B), (3)beingl(B|X) =−n∑i=1[m∑k=1yikBtkxi − lnm∑k=1e(Btkxi)](4)and p(B) is the joint distribution of vector B.L(B) is optimized by iteratively maximizing a surro-gate function Q, thus (see e.g. [11]):Bˆ(t+1) = argmaxBQ(B|Bˆ(t)). (5)Once the classifier has been trained, it will be appliedto new images whose classification is unknown. The re-sultant model is applied on a new vector of characteris-tics to obtain a vector of K probabilities, where K is thenumber of classes. The k−element of the vector repre-sents the probability of the new image belongs to classk. Therefore, the resultant value to apply the classifierwill be the class with major probability of belonging.2.3 Similarity measureHaving a prior knowledge about the similarity betweenimages, our objective is to find the most similar imagesto the one given (with respect to the obtained features).First, we sample r pairs of images from the imagedatabase. The idea is to determine a unique discrepancydistance (dg) for all images, that models the similaritybetween images.Then for every pair (a,b) of images, we compute:• The independent variables, i.e., the distance betweentheir features:xab = (d1(a,b),d2(a,b), ...,dn(a,b)), (6)where di is a distance function that models the simi-larity with respect to the feature i (i = 1, . . . ,n).Notethat, since di are distances, these variables are nonnegative.WSCG 2010 Poster papers 20• The answer variable or dependent variable, Yab, isa boolean set of variables, with a value of 0 if theimages (a and b) belong to the same class, or 1 ifnot.This is the training stage of the system.We will apply modified Logistic Regression to(x j1,x j2, . . . ,x jn,y j), j = 1, . . . ,r.Since our objective is to determine a measurementof discrepancy among images, and as the independentvariables are non-negative, the linear predictor will benon-negative. Thus we consider a link function thattransforms pi into a quantity that takes values in the in-terval [0,+∞). Then we consider the link function:g(pi) = log(1+pi1−pi). (7)In this case,pi =ext β −1ext β +1and 1−pi = 2ext β +1. (8)We estimate that parameters β = (β1,β2, . . . ,βn),which maximize the likelihood function:L(β ) =r∑i=1yi log(0.5(extiβ −1))− log(0.5(extiβ +1)),being β ≥ 0.Unfortunately there is no analytical solution for βˆ(estimated β ), but we may resort to a Newton-Raphsoniterative procedure. Each cycle in this procedure pro-vides an updating formula given by:βˆ (k+1) = βˆ (k) +(XtWX)−1XtZˆ(Y− Yˆ), (9)where Y denotes the vector of response values, X de-notes a matrix with each row by xti , Yˆ the vector of es-timated values at that iteration and W, Z denotes thediagonal matrix with elements:zˆi =exti βˆ(k)exti βˆ(k) −1, andwˆi = yiexti βˆ(k)(exti βˆ(k) −1)2+exti βˆ(k)(exti βˆ(k)+1)2, (10)respectively. This formula is used until the estimatesconverge.We obtain:pˆiab = Pˆ[Yab = 1] =extabβˆ −1extabβˆ +1, (11)the probability of a and b are different.Note that for all a ∈A , it holds that di(a,a) = 0 (i =1, . . . ,n). Therefore, pˆiaa = 0. Given a,b ∈ A , we saythat a is similar to b if pˆiab is close to 0. However if twoimages a and b are very different with respect to somefeature (di(a,b) tends to infinite) and βi > 0, then theprobability that a and b be different, pˆiab tends to 1.We aim to determine the most similar images to theimage under study by applying this method. For a newimage c, we can compute pˆica j , that can be interpretedas the discrepancy degree between c and a j. Note that ifc and a j are very similar, the probability of discrepancyis near 0. Then, we must look for the images a j so thatthe probability of this one being different from c is near0. Observe that pˆiab near 0 is equivalent to xtabβˆ near 0.Then, we could consider:dg(·) =n∑i=1βˆidi(·), (12)as the measure of discrepancy between images.2.4 Relevance feedback to searchingThe method may improve by providing user’s opinionabout retrieved images at each step. The β parameteris then updated by applying the Bayesian Logistic Re-gression method, and by showing new images to theuser. A measure is thus obtained for each query imageand user. This step corresponds to the learning processin the methodology, see Figure 1.Figure 1: Searching and relevance feedback.The system displays the most similar images on data-base to the Query Image (QI). The user can interactwith the system, indicating which of the retrieval im-ages (RI) correspond to their search conditions (RI-YES) and which do not (RI-NO). Therefore, the opin-ion of the user in every query allows to add new pairsby means of a process of feedback, see Table 1.The system can learn from the answers provided bythe user and update vector β with the above mentionedinformation (feedback). Once optimized, the system of-fers new similar images. The process repeats itself untilthe user demonstrates an agreement with the result.WSCG 2010 Poster papers 21Image QI RI-YES RI-NOQI . . . Y = 0 Y = 1RI-YES Y = 0 Y = 0 Y = 1RI-NO Y = 1 Y = 1 . . .Table 1: Learning process (QI: Query Image; RI: Re-trieval Image; RI-YES: RI corresponds to the QI; RI-NO: RI does not correspond to the QI).3 CONCLUSIONSIn this paper we have tested a novel pairwise compar-ison method to find most similar images and a classi-fication method based on Bayesian logistic regression.The method has sorted query results by similarity, so,we can say that it works like a similarity measure. Thetechnique is based on the Logistic Regression method,and it is useful to get information for Decision Making.In addition the method is integrated in a commercialimage database system. It is also particularly useful tosolve ranking issues involving a large number of fea-tures and few images. Finally, the method is easily ap-plied in practice. By extracting color, texture and shapefeatures from the digital images, acceptable results areobtained for some real CBIR problems. This methodproposes an approach to the semantic gap.ACKNOWLEDGEMENTSThis research was partially supported by projectTIN2008-06796-C04-03 from MEC and projectPDT09A009 from Junta de Extremadura, Spain.REFERENCES[1] Arias-Nicolás, J.P., Martín, J., Pérez, C.: A logis-tic regression-based pairwise comparison methodto aggregate preferences. Group Decision and Ne-gotiation, 17(3), 237–247 (2008).[2] Berger, J.O.: Statistical Decision Theory andBayesian Analysis. Springer (1985).[3] Caro, A., Rodríguez, P.G., Antequera, T., Pala-cios, R.: Feasible application of shape-based clas-sification. Pattern Recognition and Image Analy-sis. LNCS Vol.4477, pp.588-595 IbPRIA (2007).[4] Cinque L., Levialdi S., Pellicano A., Olsen K.A.:Color-Based Image Retrieval Using Spatial-Chromatic Histograms. IEEE International Con-ference on Multimedia Computing and Systems(ICMCS’99), Vol.2, pp. 969-973, (1999).[5] Cox, I.J., Miller, M.L., Minka, T., Papathornas,T., Yianilos, P.: The Bayesian Image Retrival Sys-tem, PicHunter: Theory, Implementation and Psy-chophysical experiments. IEEE Transactions onImage Processing, 9, 20–37 (2000).[6] Dumais, S., Platt, J., Heckerman, D., Sahami, M.:Inductive learning algorithms and representationsfor text categorization. Proc. of 7th Internationalconference on Information and Knowledge Man-agement. (1998).[7] El-Naqa, I., Yang, Y., Galatsanos, N.P.,Nishikawa, R.M., Wernick, M.N.: A simi-larity learning approach to content-based imageretrieval: application to digital mammography.IEEE Transactions on Medical Imaging 23(10),1233–1244 (2004).[8] Flickner, M.D., Sawhney, H.S., Niblack, C.W.,Ashley, J., Huang, Q., Dom, B., Gorkani, M.,Hafner, J., Lee, D., Petkovic, D., Steele, D.,Yanker, P.C.: Query by Image and Video Content:The QBIC System. IEEE Computer 28(9), 23–32(1995).[9] Galloway M. M.: Texture Analysis using GreyLevel Run Lengths. Computer Graphics and Imag.Processing, vol. 4, pp. 172-179 (1975).[10] Haralick, R.M., Shapiro, L.G.: Computer andRobot Vision. Vol.I Addison-Wesley (1993).[11] Krishnapuram, B., Carin, L., Figueiredo, M.,Hartemink, A.J.: Sparse Multinomial LogisticRegression: Fast Algorithms and GeneralizationBounds. IEEE Transactions on Pattern Analysisand Machine Intelligence, 27, 6, 957–968 (2005).[12] Liu, W., Peng, F., Feng, S., You, J., Chen, Z., Wu,J., Yuan, K., Ye, D.: Semantic Feature Extractionfor Brain CT Image Clustering Using Nonnega-tive Matrix Factorization. Lecture Notes in Com-puter Science 4901: Medical Biometrics, 41–48(2008).[13] McCullagh, P., Nelder, J.A.: Generalized LinearModels, second ed.. Chapman & Hall (1989).[14] Qatris Imanager, SICUBO S.L.(www.sicubo.com), Cáceres, Spain.[15] Smeulders, A., Worring, M., Santini, S., Gupta,A., Jain, R.: Content-Based Image Retrieval atthe End of the Early Years. IEEE Transactions OnPattern Analisys andMachine Intelligence 22(12),1349–1380 (2000).[16] Tversky, A.: Features of similarity. PhychologicalReview 84(4), 327–352 (1977).[17] Vailaya, A., Figueiredo, M., Jain, A., Hong Jiang,Z.: Content-based hierarchical classification ofvacation images. Multimedia Computing and Sys-tems, IEEE International Conference. Vol. 1, pp.518-523 (1999).[18] Li, J., Wang, J.Z.: ALIP: the Automatic Linguis-tic Indexing of Pictures system. Computer Visionand Pattern Recognition. IEEE Computer SocietyConference. Vol 2, pp. 1208-1209 CVPR (2005).WSCG 2010 Poster papers 22

A novel content-based image retrieval system based on Bayesian logistic regression

University of West Bohemia Digital Library

http://dspace5.zcu.cz/bitstream/11025/1106/1/Arias-Nicol%c3%a1s.pdf

A novel content-based image retrieval system based on Bayesian logistic regression

Abstract

Similar works

Full text

Available Versions

DSpace at University of West Bohemia

University of West Bohemia Digital Library