A two-component model is proposed for perceptual image coding. For the first component of the model, the watershed operator is used to detect strong edge features. Then, an efficient morphological interpolation algorithm reconstructs the smooth areas of the image from the extracted edge information, also known as sketch data. The residual component, containing fine textures, is separately coded by a subband coding scheme. The morphological operators involved in the coding of the primary component perform very efficiently compared to conventional techniques like the LGO operator, used for the edge extraction, or the diffusion filters, iteratively applied for the interpolation of smooth areas in previously reported sketch-based coding schemes.Peer ReviewedPostprint (published version

Casas Pla, Josep Ramon

Torres Urgell, Lluís

English

A two-component model is proposed for perceptual image coding. For the first component of the model, the watershed operator is used to detect strong edge features. Then, an efficient morphological interpolation algorithm reconstructs the smooth areas of the image from the extracted edge information, also known as sketch data. The residual component, containing fine textures, is separately coded by a subband coding scheme. The morphological operators involved in the coding of the primary component perform very efficiently compared to conventional techniques like the LGO operator, used for the edge extraction, or the diffusion filters, iteratively applied for the interpolation of smooth areas in previously reported sketch-based coding schemes.Peer Reviewe

UPCommons

Strong edge features for image coding

UPCommons. Portal del coneixement obert de la UPC

STRONG EDGE FEATURES FOR IMAGE CODINGJOSEP R. CASAS and LUIS TORRESDepartment of Signal Theory and CommunicationsUniversitat Politecnica de CatalunyaCampus Nord UPC, edici D5Gran Capita s/n, 08034 Barcelona, SpainE-mail: josep@gps.tsc.upc.esAbstract. A two-component model is proposed for perceptual image coding. For the rst com-ponent of the model, the watershed operator is used to detect strong edge features. Then, anecient morphological interpolation algorithm reconstructs the smooth areas of the image fromthe extracted edge information, also known as sketch data. The residual component, containingne textures, is separately coded by a subband coding scheme. The morphological operators in-volved in the coding of the primary component perform very eciently compared to conventionaltechniques like the LGO operator, used for the edge extraction, or the diusion lters, iterativelyapplied for the interpolation of smooth areas in previously reported sketch-based coding schemes.Key words: morphological interpolation, perceptual models, feature-based image coding1. IntroductionThe coding of visual information cannot be performed eciently without consideringperceptual factors of the human vision [1]. Current standards for image compression[2] exploit some aspects of visual perception {for instance, in the design of quanti-zation tables for DCT coecients{ but it is generally accepted that only the studyof image models strongly related to the Human Visual System [3] will lead to thehighest compression ratios needed for very low bit-rate applications. The so-calledSecond Generation models permit a graceful degradation of the perceived qualityof reconstructed images, without the unnatural artifacts (blockiness, ringing andblurring) of waveform coding techniques at low bit-rates.In a recent paper, Ran and Farvardin [4] stressed the role of strong edges in ourperception of the images. They put forward a model for image compression based ona three component decomposition: namely a strong edge component, a smooth com-ponent and a texture component. The performance of such model was thoroughlyinvestigated [5], proving its utility for most coding applications and showing a sig-nicant improvement over JPEG at low bit-rates. However, one of the drawbacksof their method, as well as of previously reported works in the same eld [6, 7], isthe large computation time spent in the edge extraction and interpolation processesapplied to obtain the rst and second components of the model. These are based onthe solution of an energy minimization (variational) problem, governed by the heator diusion equation, that converges rather slowly to the nal solution by means ofan iterative space-variant ltering operation.This work was supported by the European RACE/Morpheco project 20532 JOSEP R. CASAS AND LUIS TORRESMathematical Morphology provides powerful operators to perform shape anal-ysis. Morphological operators are very useful for the detection of edge features in`perceptually motivated' Second Generation image coding applications, as has beenshown in segmentation-based coding schemes [8]. Furthermore, a new morpholog-ical interpolation technique intended for the scattered data interpolation problemhas been described in [9]. It has proven to be faster than linear diusion techniquesin order to generate the smooth component from the extracted edge features, withsimilar quality of the interpolation result.The aim of this paper is to present a two-component model for perceptual imagecoding that strongly relies on morphological operators. For the rst component, thewatershed operator is used to detect strong edge features. Then, a morphologicalinterpolation algorithm reconstructs the smooth textures from the edge information.This results in the primary component of the perceptual model, formed by strongedges and smooth textures. The residual component, or texture component, con-tains the ne textures and is separately coded by a subband coding scheme alreadypresented in [10]. The morphological operators involved in the coding of the primarycomponent perform very eciently compared to previously reported techniques foredge extraction, like the LGO operator [6] or the diusion lters iteratively appliedfor the interpolation of smooth areas in [6, 7, 4].2. Strong Edge Extraction from the Morphological LaplacianThe morphological Laplacian, L(f), is dened as the residue of the gradient bydilation, g+(), and the gradient by erosion, g (), that is:g+(f) = (f)   f (1)g (f) = f   "(f) (2)L(f) = g+(f)   g (f) (3)The morphological Laplacian is greater than zero at the lower edge of the tran-sitions and smaller than zero at the upper edge. In at surfaces or slanted planeswithout convexity changes, it cancels out. Indeed, it can be shown that the mor-phological Laplacian is an approximation of the signal second derivative.The extrema of the second derivative locate the points with largest curvaturevalues. These points occur at the upper and lower sides of the transitions bring-ing information about the transition width and the intensity change. The readeris kindly referred to the drawing in gure 1 (left) for an illustration of the one-dimensional case. In the two-dimensional case of gure 1 (right), the set of pointswhere the morphological Laplacian reaches signicant values mainly corresponds tothe perceived image contours. The lines of largest curvature are called upper andlower edge brims by some authors [4]. Robinson [11] claims that these lines, whichmay be obtained as the `crest' and `valley' lines of a second derivative operator, areless noisy than Laplacian zero-crossings that follow the edge midpoints. Edge brimsdo not show so many random uctuations because they do not represent a very rapidchange in value with respect to position as edge midpoints do.STRONG EDGE FEATURES FOR IMAGE CODING 3gradientsignallaplacianoooupper brimlowerbrimzero crossingFig. 1. Laplacian extrema locates edge brims: left, upper and lower brims; right, morpho-logical Laplacian of the cameraman image (Note: mid grey corresponds to zero level)A smooth approximation of an image may be obtained by interpolation fromthe set of pixels with large curvature values. The following experiment has beencarried out in order to investigate its possible application to image compression.In the left image of gure 2, a set of pixels having large absolute values of themorphological Laplacian is shown. If we attempt to interpolate the remaining pixelsof the smooth areas in between, the result will be the one presented in the rightimage {the interpolation technique will be briey explained in the following section.About one tenth of the pixels of the image have been used as initial points forthe interpolation algorithm. The peak to noise ratio of the interpolated image ofgure 2 is only 23 dB but its subjective quality is not bad, because our attentionis primarily drawn to the strong transitions which have been correctly placed andreproduced. Edge brims do look promising for a primitive-based characterization ofvisual information from a perceptual point of view. This experiment proves that itis possible to obtain a fair approximation of the original image from the amplitudesand positions of some pixels in the neighborhood of the strong edges. Moreover,the morphological Laplacian performs as an eective enhancement operator for thedetection of such set of initial pixels.Obviously, the application of this idea to image coding relies on the selection ofa proper set of initial pixels for the interpolation process. The initial set should leadto a compact representation and, at the same time, allow a good approximation ofthe original image by interpolation. A good strategy could be to select the crestand valley lines of the Laplacian so that a chain code technique may be appliedfor the coding of the pixels' positions at low cost. The amplitudes of the initialpixels should also may be coded with a few number of bits. Given that intensityvalues along the edge brims should keep rather constant, some approximationmay beemployed to code these values within each brim line. In the current work, polynomialapproximations will be used.In the left image of gure 3, the white and black lines correspond, respectively, to4 JOSEP R. CASAS AND LUIS TORRESFig. 2. Morphological interpolation from pixels with large Laplacian values: left, initialimage (about 10% pixels); right, interpolation resultthe crest and valley lines of the Laplacian or, likewise, to the positions of the lowerand upper edge brims of the initial image. Edge brims may be detected by computingthe watershed of the Laplacian and of its dual with an appropriate set of markers.In order to obtain the lower brims (crest lines of the Laplacian), the set of markersis formed by the union of two sets: the at areas of the original image larger than agiven size and some connected components with negative Laplacian values indicatingthe presence of valleys. For the upper brims (valley lines of the Laplacian), the secondset is formed by connected components with positive Laplacian values indicatingpeaks and ridges. Please notice that some pieces of contour have been removed fromthe watershed result, either because the Laplacian was not signicant enough atthese positions or because the lines were too short. The necessary thresholds havebeen chosen on an empirical basis.If the initial set is composed of the pixels at the positions indicated by the water-shed lines shown in gure 3 with the approximated intensity values, the interpolationresults in the right image of the same gure.3. Reconstruction of the Strong Edge Componentby Morphological InterpolationThe interpolation technique used in the previous examples has been already pre-sented in [9] but, for completeness purposes, it will be briey described in the presentsection.Starting from the set of initial pixels the, morphological interpolation techniqueapproximates the amplitudes of the unknown pixels of the image by an ecientalgorithm that consists of two steps, namely a geodesic propagation step followed bya smoothing step. This two steps are successively iterated.STRONG EDGE FEATURES FOR IMAGE CODING 5Fig. 3. Interpolation from lower and upper edge brims: left, brims' positions; right,interpolation result at 0.18 bpp3.1. Geodesic propagation stepIn the geodesic propagation step, the amplitude values of the known pixels are prop-agated by geodesic dilation to ll the empty areas of the image. This is performedby a fast algorithm, using FIFO queues. Some intermediate images correspondingto the geodesic propagation step are shown in gure 4 for a synthetic initial imageconsisting of two small geometric gures.3.2. Progressive smoothing stepAt the positions where two or more propagation fronts originated from initial pixelsof dierent amplitudes meet, the process stops and a false transition is created. Thefalse transitions appearing outside the set of initial pixels will be smoothed in thesecond step. The morphological Laplacian is used as a transition detector in order toobtain these false transitions. Pixels on both sides of the false transitions composethe set of secondary pixels. A grey level value equal to the average of the intensityvalues on both sides of the transition is assigned to each secondary pixel. This is thesmoothing step. Secondary pixels will be used in the next iteration of the algorithmin order to smooth out these transitions.Initial image prop. dist. 4 prop. dist. 12 prop. dist. 24 prop. dist. 72Fig. 4. Geodesic propagation step: initial pixels and some intermediate images6 JOSEP R. CASAS AND LUIS TORRESinitial prop. 1st iteration 2nd iteration 3rd iteration 16th iteration...Fig. 5. Smoothing step: rst row, initial and secondary pixels; second row: propagation3.3. IterationNext, a second iteration is performed: the propagation step propagates the greylevel values from the sets of initial as well as secondary pixels. The propagationcreates new false transitions which dene a new set of secondary pixels where greylevel values are smoothed. Note that this new set of secondary pixels generally doesnot include the rst set of secondary pixels. This process of 1) propagation of valuesfrom the initial and secondary pixels, and 2) smoothing of the grey levels of the newsecondary pixels, is iterated until idempotence. Figure 5 illustrates several iterationsof the algorithm. After a few number of iterations, the algorithm quickly convergesto the nal interpolated image. For instance, only twelve iterations were necessaryfor the morphological interpolation result presented in the example of gure 3. TheCPU time was of 9 seconds in a SUN SparcClassic workstation. Applying iteratedspace variant linear diusion lters to solve the problem of interpolation from sparsesets, and even using multigrid techniques to speed up convergence, execution timesabout two orders of magnitude higher are reported [4].4. Two-component Image CodingThe interpolation result of gure 3 (right) corresponds to the rst component ofour perceptual model, formed by strong edges and smooth areas generated from theinitial pixels' positions and amplitudes. A derivative chain code is used to code thepixels' positions, whereas the amplitude values have been coded by polynomial ap-proximation. More precisely, the network of brim lines is broken at each triple point(points with more than two branches). Then, the amplitudes of the pixels locatedunder the resulting curves are approximated by a rst order polynomial. The twocoecients dening each polynomial are quantized, entropy coded and transmitted.The overall bit-rate for this coded image is 0.18 bits per pixel. About 16% of thisrate is spent in the coding of amplitudes, 70% for the chain-code information andthe remaining 9% for the initial positions of each brim line.STRONG EDGE FEATURES FOR IMAGE CODING 7Fig. 6. Texture component: left, coding residue; right, subband coded texture at 0.15 bppThe coding residue of the rst component {computed as the dierence betweenthe original image and the interpolation result{ mainly consists of ne textures. Thissecond component of the model is shown in the left image of gure 6. It lacks ofsignicant transitions and may be approximated by a waveform coding technique.A coded reconstruction at low bit-rate (0.15 bpp) is shown in the right image of thesame gure. It has been obtained by the application of the linear subband codingscheme presented in [10]. Information about the edge structure {available from therst component{ is used for the texture coding of the second component, so thatthe masking eect of strong transitions may be considered. Amplitude errors in theneighborhood of these areas are less noticeable for the human eye than in other partsof the image [12]. Therefore, the quantization process is allowed to introduce largeerrors near the transitions by employing adaptive quantizers and bit allocations overarbitrarily shaped sub-edge regions in order to reduce the total number of bits.5. Results and ConclusionThe compression ratio achieved with the above strategy is equal to 24 (0.33 bpp) forthe addition of the strong edge and the ne-texture components of gures 3 and 6.The result is shown in the right image of gure 7. For comparison, the applicationof the JPEG standard [2] at the same bit-rate is shown in the left image of thesame gure. The subjective quality of the described technique is signicantly betterbecause of the good rendition of the strong edges. The PSNR value (25.5 dB) isalso larger than for the JPEG reconstruction (24 dB). The artifacts produced bythe block-based DCT coding {blockiness in smooth areas and ringing in the neigh-borhood of strong transitions{ are not present in the result of the two-componentcoding scheme. However, a dierent kind of visual artifacts may be observed. Acertain smoothing eect is visible in some parts of the image and there are somemissing objects, for instance the neck of the shirt has been almost removed.The separate coding of strong edges permits the adaptation of the coding schemeto the visual perception of the images, avoiding unnatural degradations produced8 JOSEP R. CASAS AND LUIS TORRESFig. 7. Results: left, JPEG at 0.35 bpp; right, described technique at 0.33 bppby waveform coding techniques at high compression ratios. A number of dierentartifacts are introduced by the new method at low bit-rates. It is hoped that sucheects are more naturally perceived than those of waveform coding techniques by thesubjective judgement of the observer. Furthermore, the application of morphologicaltechniques for the extraction and reconstruction of the rst component {watershedand morphological interpolation{ perform very eciently compared to previouslyreported techniques for edge extraction and scattered data interpolation.References1. N. Jayant, J. Johnston, and R. Safranek. Signal compression based on models of humanperception. Proceedings of the IEEE, 81(10):1383{1421, October 1993.2. G. K. Wallace. The JPEG still picture compression standard. Communications of the ACM,34(4):30{44, April 1991.3. M. Kunt, A. Ikonomopoulos, and M. Kocher. Second generation image coding techniques.Proceedings of the IEEE, 73(4):549{575, April 1985.4. X. Ran and N. Farvardin. A perceptually motivated three-component image model. Part I:Description of the model. IEEE Transactions on Image Processing, 4(4):401{415, April 1995.5. X. Ran and N. Farvardin. A perceptually motivated three-component image model. Part II:Application to image compression. IEEE Transactions on Image Processing, 4(4):430{447,April 1995.6. S. Carlsson. Sketch based coding of grey level images. EURASIP, Signal Processing, 15(1):57{83, July 1988.7. P. Grattoni and A. Guiducci. Contour coding for image description. Pattern RecognitionLetters, 11:95{105, February 1990.8. P. Salembier, L. Torres, F. Meyer, and C. Gu. Region-based video coding using mathematicalmorphology. Proceedings of IEEE (Invited Paper), 83(6):843{857, June 1995.9. J.R. Casas, P. Salembier, and L. Torres. Morphological interpolation for texture coding. InIEEE International Conference on Image Processing, volume I, pages 526{529, WashingtonDC, USA, October 1995.10. J.R. Casas and L. Torres. A feature-based subband coding scheme. To be presented atICASSP'96, May 1996.11. John A. Robinson. Image coding with ridge and valley primitives. IEEE Transactions onCommunications, 43(6):2095{2102, June 1995.12. T.N. Cornsweet. Visual Perception. Academic Press, New York, 1970.

https://upcommons.upc.edu/bitstream/2117/102877/1/10.1.1.51.670%20%282%29.pdf

Strong edge features for image coding

Abstract

Similar works

Full text

Available Versions

UPCommons

UPCommons. Portal del coneixement obert de la UPC