Direct immunofluorescence (DIF) images are used by clinical experts for the diagnosis of autoimmune blistering diseases. The analysis of serration patterns in DIF images concerns two types of patterns, namely n- and u-serrated. Manual analysis is time-consuming and challenging due to noise. We propose an algorithm for the automatic classification of serrated patterns in DIF images. We first segment the epidermal basement membrane zone (BMZ) where n- and u-serrated patterns are typically found. Then, we apply a bank of B-COSFIRE filters to detect ridges and determine their orientations with respect to the BMZ. Finally, we classify an image by comparing its normalized histogram of relative orientations with those of the training images using a nearest neighbor approach. We achieve a recognition rate of 84.4% on a UMCG data set of 416 DIF images, which is comparable to 83.4% by clinical experts.peer-reviewe

Azzopardi, George

Guo, Jiapan

Jonkman, Marcel F.

Meijer, Joost M.

Petkov, Nicolai

Shi, Chenyu

Direct immunofluorescence (DIF) images are used by clinical experts for the diagnosis of autoimmune blistering diseases. The analysis of serration patterns in DIF images concerns two types of patterns, namely n- and u-serrated. Manual analysis is time-consuming and challenging due to noise. We propose an algorithm for the automatic classification of serrated patterns in DIF images. We first segment the epidermal basement membrane zone (BMZ) where n- and u-serrated patterns are typically found. Then, we apply a bank of B-COSFIRE filters to detect ridges and determine their orientations with respect to the BMZ. Finally, we classify an image by comparing its normalized histogram of relative orientations with those of the training images using a nearest neighbor approach. We achieve a recognition rate of 84.4% on a UMCG data set of 416 DIF images, which is comparable to 83.4% by clinical experts

Meijer, Joost

ARTS repository - University of Groningen

Automatic Classification of Serrated Patterns in Direct Immunofluorescence Images

Chenyu Shi

Joost M. Meijer

Jiapan Guo

George Azzopardi

Marcel F. Jonkman

Nicolai Petkov

CiteSeerX

1Automatic classification of serrated patterns in direct immunofluorescence

University of Groningen

NARCIS 

OAR@UM

1Automatic classification of serrated patterns in directimmunofluorescence imagesChenyu Shi1, Joost M. Meijer3, Jiapan Guo1, George Azzopardi1,2,Marcel F. Jonkman3 and Nicolai Petkov11Johann Bernoulli Institute for Mathematics and Computer Science, Universityof Groningen, The Netherlands2Intelligent Computer Systems, University of Malta, Malta3Dermatology for Medical Sciences, University Medical Center Groningen(UMCG), University of Groningen, The NetherlandsAbstract: Direct immunofluorescence (DIF) images are used by clini-cal experts for the diagnosis of autoimmune blistering diseases. Theanalysis of serration patterns in DIF images concerns two types of pat-terns, namely n- and u-serrated. Manual analysis is time-consumingand challenging due to noise. We propose an algorithm for the auto-matic classification of serrated patterns in DIF images. We first seg-ment the epidermal basement membrane zone (BMZ) where n- andu-serrated patterns are typically found. Then, we apply a bank of B-COSFIRE filters to detect ridges and determine their orientations withrespect to the BMZ. Finally, we classify an image by comparing itsnormalized histogram of relative orientations with those of the train-ing images using a nearest neighbor approach. We achieve a recog-nition rate of 84.4% on a UMCG data set of 416 DIF images, which iscomparable to 83.4% by clinical experts.1 IntroductionManual analysis of serration patterns in direct immunouorescence (DIF) im-ages is used by clinical experts for the diagnosis of certain skin diseases [5, 12–14]. One such disease is epidermolysis bullosa acquisita (EBA) [15], which is achronic subepidermal blistering disease of the skin and mucus membranes. Theserration pattern analysis concerns two types of patterns, which are referred toas n- and u-serrated, along the epidermal basement membrane zone (BMZ). Then-serrated patterns, an example of which is shown in Fig. 1a, are characterized2 C. Shi, J. Guo, G. Azzopardi, J.M. Meijer, M.F. Jonkman and N. Petkov(a) (b)Fig. 1: Examples of (a) n-serrated and (b) u-serrated patterns.by undulating n-shapes. On the other hand, the u-serrated patterns comprisefinger-like shapes, as shown in Fig. 1b. The manual analysis of serration pat-terns is very challenging mainly due to noise and lack of well-trained clinicalexperts and immunofluorescence microscopists [7, 12].We propose an improvement of the method that we proposed in [11] for theclassification of n- and u-serrated patterns. That method uses as a feature vec-tor a histogram of absolute orientations of ridges. It does not consider the localorientation of the BMZ. Here, we first segment the BMZ and then we computeits local orientations. Subsequently, we apply a bank of B-COSFIRE filters [3] todetect ridges and determine their relative orientations with respect to the BMZ.Finally, we classify a test image by comparing its normalized histogram of rel-ative orientations with those of the training images using a nearest neighborapproach.The rest of the paper is organized as follows. In Section 2 we explain the pro-posed method. We evaluate it and report experimental results in Section 3. InSection 4, we draw conclusions.2 Proposed method2.1 OverviewIn the following, we explain the main idea of the proposed method and subse-quently we provide a detailed description of each step.Fig. 2(a-f) show six examples of RGB DIF images. Fig. 2(a-c) and Fig. 2(d-f)contain n-serrated and u-serrated patterns, respectively. The dashed red linesenclose the BMZ. This is the area where n- or u-serrated patterns are located.For each DIF image, we first segment the BMZ by applying the preprocessingAutomatic classification of serrated patterns in DIF images 3(a) (b) (c)(d) (e) (f)Fig. 2: Examples of RGB DIF images that contain (a-c) n-serrated and (d-f) u-serrated pat-terns. The red dashed lines indicate the BMZ.algorithm, which we proposed in [11], to the green channel of the DIF image.Then, we compute the local orientation for each location along the BMZ by con-sidering a small and a large neighbourhood, respectively. For the small neigh-bourhood we use B-COSFIRE ridge detectors and for the larger neighbourhoodwe use the orientation measurement approach introduced in [4]. The formermeasurement gives the absolute orientations of the serrated pattern, while thelatter measurement determines the orientation of the baseline. Finally, we com-pute the relative orientations and represent every DIF image by a normalizedhistogram of such orientations.2.2 Segmentation of the BMZWe use the algorithm that we proposed in [11] to segment the BMZ from thegreen channel of a DIF image (of size 1392× 1040 pixels), Fig. 3b. First, we use adisk-shaped structuring element (radius of 30 pixels) to perform morphologicalclosing followed by a flood-fill operation, of which the result is shown in Fig. 3c.Then, we use Canny edge detector [6] to extract the upper-most boundary of thelargest connected component. In order to avoid the border effect, we remove 50pixels from each side of the image. Fig. 3d shows the extracted boundary. Weobtain the BMZ mask (Fig. 3e) by dilating the extracted boundary using a disk-shaped structuring element (radius of 30 pixels). Finally, we use the resultingBMZ mask to extract the corresponding region of the green channel of a DIF im-age (Fig. 3f).4 C. Shi, J. Guo, G. Azzopardi, J.M. Meijer, M.F. Jonkman and N. Petkov(a) (b) (c)(d) (e) (f)Fig. 3: (a) An example of a RGB DIF image (of size 1392× 1040 pixels) and (b) its greenchannel image. (c) The resulting binary image of a morphological closing operationusing a disk-shaped structuring element (radius of 30 pixels) applied to the greenchannel image. (d) The extracted boundary between the dark and white regions ofthe binary image. (e) Binary mask of the BMZ and (f) the product of the greenchannel image and the mask.2.3 Determination of the local BMZ orientationWe compute the local BMZ orientation φ(x, y) at a given location (x, y) by an ori-entation measurement approach introduced in [4]. Fig. 4 illustrates this method.For a point A on the BMZ mask (Fig. 3e), we consider the intensity pixels of thecorresponding boundary map (Fig. 3d) along a circle of a given radius r = 50pixels. Two positions (B and C) are chosen as the points that characterize thelocal orientations. We determine the angles of the polar coordinates for suchpoints with respect to the point A. For instance, the leg from B to A forms aninbound angle φ1 = 0.72 radians, and the leg from A to C forms an outboundangle φ2 = 3.40 radians. Then we compute the local orientation φ(x, y) of Awith respect to B and C as:φ(x, y) = ||pi/2− (φ1 + φ2)/2||pi (1)where || · ||pi represents the modulus of pi. For this example, φ(A) = 0.49 radi-ans.Automatic classification of serrated patterns in DIF images 5Fig. 4: Part of a BMZ mask from Fig. 3e.2.4 Determination of ridge orientationWe use B-COSFIRE filters1 [3] to detect ridges and determine their orientations.A B-COSFIRE filter is a ridge detector, which is based on the COSFIRE ap-proach [2] and the CORF computational model [1]. Its response is achievedby computing the geometric mean of a group of linearly aligned responses of aDifference-of-Gaussians (DoG) filter. A B-COSFIRE filter has three main param-eters: the standard deviation σ of the outer Gaussian function in the involvedDoG filter2, the radius l and the orientation θ. Further to the experiments wedid in [11], we use σ = 1.2 and l = 4. Here, we only mention that we use 16B-COSFIRE filters with θ ∈ (0,pi/16, . . . , 15pi/16). Fig. 5a shows an exampleof a B-COSFIRE filter that is selective for ridges with an orientation θ = pi/6radians and Fig. 5c shows its response map to the input image shown in Fig. 5b.These filters have other parameters which we skip here for brevity. We refer theinterested reader to [1–3, 8–11] and to an online implementation3.For each BMZ, we apply a bank of 16 B-COSFIRE filters that are selective fordifferent orientations. We denote byΘ(x, y) the local orientation of a ridge pointas:Θ(x, y) = ||pi/2 + θ(x, y)||pi (2)where θ(x, y) represents the orientation of the B-COSFIRE filter that exhibits themaximum response at the location (x, y).1Matlab scripts: http://www.mathworks.com/matlabcentral/fileexchange/491722The standard deviation of the inner Gaussian function is 0.5σ3http://matlabserver.cs.rug.nl6 C. Shi, J. Guo, G. Azzopardi, J.M. Meijer, M.F. Jonkman and N. Petkov(a) (b) (c)l=42σθ= pi6Fig. 5: (a) Structure of a B-COSFIRE filter which is used for detecting the ridges with anorientation θ = pi/6 radians. The cross marker indicates the support center of thefilter. The radius l indicates the farthest distance from the center of the filter thattakes as inputs five responses from a center-on DoG filter with σ = 1.2. The locationsof the afferent inputs are equally spaced in intervals of 2 pixels. (b) A cropped regionof a DIF image and (c) the corresponding B-COSFIRE response map.2.5 Descriptor: normalized histogram of relative orientationsWe denote by Φ(x, y) the relative orientation of a ridge at position (x,y) withrespect to the BMZ orientation φ(x, y):Φ(x, y) ={pi − ||φ(x, y)−Θ(x, y)||pi, if ||φ(x, y)−Θ(x, y)||pi > pi/2||φ(x, y)−Θ(x, y)||pi, otherwise (3)In this way, we obtain a map of the relative orientations for the BMZ of eachDIF image. We construct the L1-normalized histogram of these orientations anduse it as the feature vector of a DIF image. Fig. 6(a-f) show the normalizedhistograms of the images in Fig. 2.3 Evaluation3.1 Data setWe use a data set provided by the University Medical Center Groningen(UMCG). It comprises 26 DIF images from different pemphigoid patients, 15of which contain n-serrated patterns and the rest contain u-serrated ones. Weexpand the data set to 416 DIF images by rotating each of the 26 images frompi/8 to 2pi radians in intervals of pi/8 radians. All images are taken with magni-fications of ×40, which is the most commonly used scale in hospitals. The dataset is publicly available as an online test4.4http://www.nversusu.umcg.nl/Automatic classification of serrated patterns in DIF images 7(a) (b) (c)(d) (e) (f)0.050.100.150.200.050.100.150.200 pi8pi43pi8pi2 0pi8pi43pi8pi2 0pi8pi43pi8pi2Orientation Orientation OrientationL1-normalizefrequencyFig. 6: (a-f) Six L1-normalized histograms of the relative orientations of the DIF images inFig. 2(a-f), respectively.3.2 ExperimentsWe apply the proposed method to all 416 DIF images of the UMCG data set andfor every DIF image we compute the L1-normalized histogram of the relativeorientations. For the evaluation of the proposed method, we use the nearestneighbor algorithm to classify each image as u-serrated or n-serrated by com-paring its histogram with the ones of the remaining DIF images in the data set.In Table 1, we report the results that we obtain for different similarity measuresbetween the histograms. By using leave-one-out cross-validation, we achieve arecognition rate of 84.4% on the expanded UMCG data set.Table 1: Recognition rates achieved for different distance measures.Chi-square Euclidean Cosine Correlation CityblockMethod in [11] 69.9% 69.0% 63.0% 63.7% 65.6%Proposed method 84.4% 84.4% 75.5% 73.3% 83.4%4 ConclusionsWe achieve the recognition rate of 84.4% on the expended UMCG data set of 416DIF images, which is significantly better than the 69.9% that we achieve by theprevious method [11] and that it is comparable to the 83.4% achieved by clinicalexperts [12].8 C. Shi, J. Guo, G. Azzopardi, J.M. Meijer, M.F. Jonkman and N. PetkovReferences[1] Azzopardi, G., Petkov, N.: A CORF computational model of a simple cellthat relies on LGN input outperforms the Gabor function model. BiologicalCybernetics 106(3), 177–189 (2012)[2] Azzopardi, G., Petkov, N.: Trainable COSFIRE filters for keypoint detec-tion and pattern recognition. IEEE Transactions on Pattern Analysis andMachine Intelligence 35(2), 490–503 (2013)[3] Azzopardi, G., Strisciuglio, N., Vento, M., Petkov, N.: Trainable COSFIREfilters for vessel delineation with application to retinal images. Medical Im-age Analysis 19(1), 46–57 (2015)[4] Brinka, A., Smitb, J., Bulacua, M., Schomakera, L.: Writer identificationusing directional ink-trace width measurements. Pattern Recognition 45(1),162–171 (2012)[5] Buijsrogge, J.J.A., Diercks, G.F.H., Pas, H.H., Jonkman, M.F.: The manyfaces of epidermolysis bullosa acquisita after serration pattern analysisby direct immunofluorescence microscopy. British Journal of Dermatology165(1), 92–98 (2011)[6] Canny, J.: A computational approach to edge detection. IEEE Transactionson Pattern Analysis and Machine Intelligence 8(6), 679–698 (1986)[7] Gammon, W., Kowalewski, C., Chorzelski, T., Kumar, V., Briggaman,R., Beutner, E.: Direct immunofluorescence studies of sodium chloride-separated skin in the differential diagnosis of bullous pemphigoid and epi-dermolysis bullosa acquisita. Journal of the American Academy of Derma-tology 22(4), 664–670 (1990)[8] Petkov, N.: Biologically motivated computationally intensive approachesto image pattern-recognition. Future Generation Computer Systems 11(4-5), 451–465 (1995)[9] Petkov, N., Kruizinga, P.: Computational models of visual neurons spe-cialised in the detection of periodic and aperiodic oriented visual stimuli:Bar and grating cells. Biological Cybernetics 76(2), 83–96 (1997)[10] Petkov, N., Westenberg, M.: Suppression of contour perception by band-limited noise and its relation to non-classical receptive field inhibition. Bio-logical Cybernetics 88(10), 236–246 (2003)[11] Shi, C., Guo, J., Azzopardi, G., Meijer, J.M., Jonkman, M.F., Petkov, N.: Au-tomatic differentiation of u- and n-serrated patterns in direct immunofluo-rescence images. Computer Analysis of Images and Patterns (CAIP 2015),Lecture Notes in Computer Science 9256 (Springer), in print, (2015)[12] Terra, J.B., Meijer, J.M., Jonkman, M.F., Diercks, G.F.H.: The n- vs. u-Automatic classification of serrated patterns in DIF images 9serration is a learnable criterion to differentiate pemphigoid from epider-molysis bullosa acquisita in direct immunofluorescence serration patternanalysis. British Journal of Dermatology 169(1), 100–105(2013)[13] Terra, J.B., Pas, H.H., Hertl, M., Dikkers, F.G., Kamminga, N., Jonkman,M.F.: Immunofluorescence serration pattern analysis as a diagnostic cri-terion in antilaminin-332 mucous membrane pemphigoid: immunopatho-logical findings and clinical experience in 10 Dutch patients. British Journalof Dermatology 165(4), 815–822 (2011)[14] Vodegel, R., Jonkman, M., Pas, H., De Jong, M.: U-serrated immunode-position pattern differentiates type VII collagen targeting bullous diseasesfrom other subepidermal bullous autoimmune diseases. British Journal ofDermatology 151(1), 112–118 (2004)[15] Woodley, D.T., Briggaman, R.A., O’Keefe, E.J., Inman, A.O., Queen, L.L.,Gammon, W.R.: Identification of the skin basement-membrane autoanti-gen in epidermolysis bullosa acquisita. New England Journal of Medicine310(16), 1007–1013 (1984)