After a short introduction into traditional image transform coding, multirate systems and multiscale signal coding the paper focuses on the subject of image encoding by a neural network. Taking also noise into account a network model is proposed which not only learns the optimal localized basis functions for the transform but also learns to implement a whitening filter by multi-resolution encoding. A simulation showing the multi-resolution capabilitys concludes the contribution

Brause, Rüdiger W.

Glitsch, Jürgen

English

Hochschulschriftenserver - Universität Frankfurt am Main

A Spectral multi-resolution Image Encoding NetworkRüdiger W. Brause, Jürgen GlitschJ. W. Goethe-University, Computer Science Dep., Frankfurt a.M., Germanybrause@informatik.uni-frankfurt.de1  AbstractAfter  a  short  introduction  into  traditional  imagetransform  coding,  multirate  systems  and  multiscalesignal coding the paper focuses on the subject of imageencoding  by  a  neural  network.  Taking  also  noise  intoaccount  a  network  model  is  proposed  which  not  onlylearns  the  optimal  localized  basis  functions  for  thetransform but also learns to implement a whitening filterby multi-resolution encoding. A simulation showing themulti-resolution capabilitys concludes the contribution.2  IntroductionIn  the  sensor  encoding  area,  all  approaches  try  tominimize  the  necessary  information  for  a  givenreproduction error. One actual, important approach is thetransform coding concept that is the base for the JPEGand  MPEG  image  encoding  standards  [3],  [5].  Thisconcept sees the pixels of  an image as parallel signalswhich have to be encoded. For this purpose, the picture issubdivided  into  subimages    (e.g.  8x8  pixels)  andtransformed  by  a  linear  transform  into  coefficients.Afterwards, the code coefficients are quantized accordingto a quantization table. For the  reconstruction  process,these  procedures  are  inversely  done.  In  figure  1  theencoding and decoding situation is visualized.The transforms are based on the decompositionx x == åyi iinj ( )1(2.1)for a vectored signal x and the basis functions ji. It iswell known that transform coding minimizing the leastmean squared error (LMSE) for the reproduced imagescan be obtained by the Karhunen-Loève expansion (KLT)or  principal  component  analysis  (PCA),  which  can  beimplemented by lateral inhibited neural networks, e.g [1].Nevertheless,  the  JPEG  and  MPEG  standard  uses  thecosine  transform,  (DCT)  a  special  kind  of  Fouriertransform  (DFT)  as  basis  functions  ji  which  becomebasis vectors in the descrete case.They  all  share  the  property  that  the  part  of  thesampled data, the block, is of equal length for all basisfunctions. Since the different basis functions are sinus orcosines  of  different  frequencies,  this  means  that  wesample  the  sensor  data  with  the  same  inter-sampledistance  for  different  frequencies.  According  to  thesample theorem of Shannon [10] , a signal containing acertain highest frequency f can only be recontructed if thesample frequeny fs (i.e. the sampling distance), is doubledfs³2f (i.e. the sampling interval is smaller than half of theperiod of f). So, by choosing a small sampling intervalfor a reliable reconstruction we oversample all the lowerfrequency components which results in highly correlatedand thus redundant....x1x2xny1y2ym+1ymyn.. ... ......y1ym....x1x2xnEncoding Decoding storage orlinear transformation quantization inverse transformation....y2 .. ..inverse quantizationtransmissionFig. 1 The transform coding approachProceedings of the IEEE 7th Int. Conference on Tools withArtificial Intelligence, Nov.5-8, 1995, Herndon, Virginia, pp. 138-141coefficients  even  for  the  PCA  case.  This  can  bechanged by  a spectral multi-resolution  approach  whereeach frequency component has ist own sampling blocksize. This spectral multiresolution idea is also reflectedby the well known subject of multirate filter systems, seee.g. the book of Vaidyanathan[11]. Here, we start with theapproach of deviding the power spectrum |y(f)| of a signalx(t) into several overlapping intervals or subbands by thelinear  decomposition  of  basis  functions  with  differentfrequency  characteristics  (filter  banks).  In  figure  2a  afilter  bank  system  and  in  figure  2b  the  frequencyresponses of the different encoded signals yi are shown.           y0         |y|x(t  )            y1           y2             . . .(a) (b)Fig.2 Filter banks and subbands         for multirate samplingAccording  to  the  Shannon  theorem,  we  can  savesample  points  (and  therefore  encoding  coefficients)  bydecreasing  the  sampling  rates  for  the  lower  frequencybands, e.g. by a factor of two. If we arrange this parallelscheme in a sequential manner using only one type  offilter bank which consists of two symmetrically aranged,overlapping filter banks (a high-pass filter and a low-passfilter: Quadrature Mirror Filter QMF-Filter[11] ) we getthe wavelet approach [4].Since  each  filtered  signal  is  subsampled,  thecorresponding  part  of  the  original  signal  is  scaled(compressed) on the time scale. Thus, the correspondingbasis function have to be rescaled (expanded) to representthe  real  basis  function.  The  corresponding  samplinginterval  is  therefore  also  expanded,  resulting  in  adifferent interval, i.e. in image encoding a different areasurface, for each basis function.Additionally,  in  many  sensor  encoding  tasks  theability  to  deal  with  the  noisy  environment  of  thetransmission or storage is of crucial importance. It can beshown [2] that for many parallel channels the informationflow is maximized if the linear transformation producesdecorrelated  and  normalized  channel  output  whichcoincidents well with the classical result of a whiteningfilter by Shannon [9] for one channel. This means that wehave no longer to implement a PCA, but to decorrelateand normalize the output data which can be done by aninfinity of base vector systems. Among them, a PCA withscaled eigenvectors is just one sufficient solution, not anecessary one.So, we can finally conclude that we are looking for anencoding  scheme  that  uses  non-uniform  sampling  andorthonormalizes the output data  at  the  same  time.  Forthis task, neural  networks are good candidates.3  A multi-resolution network modelFor our multi-resolution model, let us first specify theactivity in the network.3.1  The activity modelThe  linear  activity  due  to  eq.(1.1)  can  be  easilyimplemented  by  a  net  of  n  linear  neurons  eachimplementing a scalar producty =  w x j jjn= å1= wTx.  (2.1)As a network they implement a linear transform by theirparallel actiony   =  y   =  1TnTw xw xWx1￿nüýïþï= (2.2)This is shown in figure 3 by the solid lines.w 1w 2w my1y2ym......x1x2xninput vector x outputvector yw eightvectorslateralconnections...Fig. 3 A symmetrically lateral inhibited network modelNow, how can we get the vectors orthogonal ?The ordinary PCA networks minimize the error of thereconstruction  by  assigning  to  the  weight  vectors  theeigenvectors of  the input covariance matrix CXX whichforms an orthogonal base. The main difference betweenthe different models is given how  the assignment takesplace.  This  is  done  by  learning  rules  which  induce  aspecial  learning  network,  i.e.  a network  for  the  signalflow exclusive for the learning phase.3.2  The learning networkThe  learning  rules  of  the  models  can  be  devided  intothree  categories:  models  using  symmetric  lateralinhibitions  (as  e.g.  [1],  see  dotted  lines  in  figure  3),models using asymmetric lateral inhibiton influence (e.g.[7])  and models  without  lateral  inhibition,  but  using  aH0(f)H1(f)H2(f):1:2:4f2 f1 f0    fkind  of  activity  backpropagation  through  the  weightsinfluencing the signal source (e.g. [6],[8]). The first oneshave the advantage that they  are  biologically  plausible(lateral  inhibition  connections  are  found  also  inbiological  nervous  systems)  and  can  easily  beimplemented by analog circuits, because it uses only oneline  of  feedback  per  neuron  contrary  to  the  signalinfluencing feedback of the latter ones.So, let us choose a lateral inhibition model, for instancethe one of Brause  [1]. Here, our objective function thatimplements a PCA for the learning model is( ) R y y ymi jj j imimRiimRw w w1 2 21 12114121 2, , ,,￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿= -= ¹ = = å å å b(2.3a)accompanied by the restriction for obtaining normalizedweights|wk| = 1 (2.3.b)The objective function is composed by two terms R1 andR2.  The  first  term  becomes  zero  only  when  allcrosscorrelation terms cij are zero while the second termbecome  strongly  negative  for  the  variance  gettingmaximal.  Since  the  extrema  of  the  objective  function,even scaled by an arbitrary factor, remains the same thefactor  b  denotes  only  the  relative  influence  of  thecrosscorrelations  with  respect  to  the  autocorrelationinfluence.Now we let the weights of this feedforward networklearn by the simple gradient descent learning rule( ) ( ) ( ) ( ) w w w w wk kwm t t t R k = - - Ñ 11 2 g , , ,￿ k = 1, 2,¼, mwith  the learning rate g and the Nabla operator Ñ for thegradient.With  the  gradient  we  can  directly  compute  thedeterministic  learning  rule  for  the  k-th  neuron.Introducing  lateral  coupling  weights  which  are  oftenobserved in biological nervous circuitryuij = - áyiyjñ      lateral inhibition  (2.4)between the neurons for the learning process we finallyget as the stochastic learning rules( ) ( ) ( ) ~,w w xk kki ii i kmk t t t u y y = - + +éëêêùûúú = ¹ å 11g b   (2.5a)and  wwwkkk ttt( )~ ( )~ ( )= normalization    (2.5b)Please  note  that  all  weight  vectors  have  a  differentnumber of dimensions; since they are only coupled by thescalar output yi, the different dimensions do not implyany problems.For  noise  suppression,  the  covariance  coefficientshave to become [2]c y yi j decorrelationP i j normalizationij i j = =¹=ìíî0 ( )( )   (2.6)In contrast to the ordinary orthonormal networks, wedo not change the length of the weight vectors to fit thevariance P by a proper objective function (see [2] , but weintroduce  a  second  learning  mechanism  for  the  area(number of components) that the weight vector covers.By correspondingly increasing the image area, also thevariance represented by the neuronal output is increased.Thus, we  change  from  the  equi-resolution  encoding  tothe multi-resolution approach.The mechanism can be realized by a simple gradientdescent on the mean squared error( ) n(t+1)  =  n(t)  -  (t) g¶¶y n Pynk ( )2 -         (2.7)For  an  unknown  input  statistic,  we  do  not  knowexplicitely  y(n).  Since  we  can  assume  that  it  is  amonotoneous  increasing  function,  we  might  as  wellreplace  it  by  a  positive  constant  for  the  stochasticlearning, including it in the (heuristic) learning rate g(t).( ) n(t+1)  =  n(t) -  (t) g y P k2 -         (2.8)So, the system starts with all neurons having an equalinput area. For a stable input statistic, by using a small geach neuron should only slowly change the input area,much more slowly than the weights themselves.  The growing  mechanism  of  (2.8)  can  also  have  abiological counterpart: the receptive field of the neuronmight grow as long as it senses a difference between itsown,  small  activity  and the  neighboured  activity.  Thiscan  also  be  interpretated  as  a  kind  of  load  balancingmechanism for a pool of processors.The lateral inhibition weights should also be updatedand reflect an average of the most recent patterns. Pleasenote that the standard stochastic approximation approachyields some problems in this case because the distributionof the yi is not stationery; they are subject for change ofthe weights.It  should  be  noted  that  the  goal  of  the  combineddecorrelation  and  growing  mechanism  is  no  longer  aPCA  in  the  classical  sense;  the  decorrelation  of  theoutput is produced by  vectors which are no longer theeigenvectors  of  the  input  correlation  matrix;  they  aresomething different. The exact analytical expression forthe new multi-resolution goal is not easily to obtain; thisis a subject for current research.4  Simulating multi-resolution imageencodingFor a small picture called „Nikita“, shown in figure 4, weimplemented a multi-resolution encoding.Fig. 4 The image „Nikita“For this purpose we divided the 35x18 pixel imageinto blocks of different, but fixed size, input sequentiallythe blocks to one network with the input dimension n ofthe  blocksize,  and  let  the  network  converge  by  thelearning rules. After convergence, we  can compare thenumber  of  encoding  coefficients  and  the  measuredreproduction error of an equal block size encoding, i.e. aclassical  KLT,  with  the  ones  of  the  multi-resolutionapproach which certainly does not result in a KLT butstill  decorrelates  the  output  data.  The  following  tablecompares the two encodings.EncodingtypeNo. of diff.comp.comp. size Totalno. ofcomp.Reprod.errorKLT 2 4x2 162 0.1413MR 2 4x2, 5x3 123 0.115KLT 3 4x2 243 0.1215MR 3 4x2, 5x3, 6x4 147 0.113As we can see, the multi-resolution scheme MR hasalready advantages over the classical KLT encoding evenfor just two components. For three KLT components, thenumber  of  necessary  encoding  components  is  198%compared to the MR encoding, even involving a higherreproduction error.This simulation has shown the principal advantage ofmulti-resolution  encoding  over  the  classical  encodingscheme of  the KLT encoding. Nevertheless,  for  higherareas the model encounters heavy convergence problemsdue to the fact that the conditions (2.5c) are only valid forthe actual, unknown variance (which is different for eacharea) and a KLT expansion, which is not given here.For  bigger  pictures  and  more  neurons  ourbiologically-inspired approach  of  symmetrical  networksposes  too  much  problems.  Thus,  we  have  to  devise  adifferent algorithm for technical applications.5  ConclusionWe presented a  multi-resolution encoding scheme forimage encoding and showed that the underlieing conceptof  non-uniform  sampling  yields  higher  compression  atlower error compared with conventional methods.Additionally, for the implementation of this conceptwe presented an adaptive algorithm for the generation ofa  noise-immune  multi-resolution  architecture  of  anneural network.Nevertheless,  for  the  multi-resolution  encoding  fortechnical applications other algorithms are needed whichwill be presented elsewhere.AcknowledgmentWe want to thank A. Schmid for providing us withthe „Nikita“ simulation data.References[1] R. Brause: Transform Coding by Lateral Inhibited NeuralNets; Proc. IEEE Tools for Art. Intell. TAI-93[2] R. Brause, M.Rippl: Image Encoding in the Presence ofNoise; submitted to Int. Conf. on  Art.  Neural  NetworksICANN95, Paris Oct. !995[3] G.Wallace: The JPEG Still Picture Compression Standard;Comm. of the ACM, Vol.34, No.4, pp.31-44, April 1991[4]  S.  Mallat:  A  Theory  for  multiresolution  signaldecomposition: The  wavelet  representation;  IEEE  Trans.Pattern Anal. and Mach. Intell., vol.31, pp.674-693, 1989[5]  D.  LeGall:  MPEG:  A  Video  Compression  Standard  forMultimedia  Applications;  Comm.  of  the  ACM,  Vol.34,No.4, pp. 47-58, April 1991[6] E.Oja:  Neural  Networks,  Principal  Components,  andSubspaces;  Int.    Journ.  Neural  Syst.  Vol1/1,  pp.  61-68,World Scientific, London 1989[7]  J.  Rubner,  K.  Schulten,  P.  Tavan:  A  Self-OrganizingNetwork for Complete Feature Extraction; in R.Eckmiller,G.Hartmann,  G.Hauske  (eds.):  Parallel  Processing  inNeural  Systems  and  Computers;  North  Holland,Amsterdam 1990, pp. 365-368[8] Terence D.  Sanger:  Optimal  Unsupervised  Learning  in aSingle-Layer Linear  Feedforward  Neural  Network;  Proc.of the Int. Conf. Neural Networks, Boston 1988, Pergamonpress; und in Neural Networks, Vol. 2, pp.459-473 (1989)[9]  C.E.  Shannon,  W.Weaver:  The  Mathematical  Theory  ofInformation; University of Illinois Press, Urbana 1949[10] C.E. Shannon: Communication in the presence of noise;Proc. of the IRE, Vol. 37, pp.10-21, Jan. 1949[11] P.P. Vaidyanathan: Multirate  Systems  and  Filter  Banks;Prentice Hall PTR, Englewood Cliffs, New Jersey 1993

A Theory for multiresolution signal decomposition: The wavelet representation;

Communication in the presence of noise;

G.Wallace: The JPEG Still Picture Compression Standard;

M.Rippl: Image Encoding in the Presence of Noise; submitted to Int.

MPEG: A Video Compression Standard for Multimedia Applications;

Multirate Systems and Filter Banks;

Optimal Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network;

Tavan: A Self-Organizing Network for Complete Feature Extraction;

Transform Coding by Lateral Inhibited Neural Nets;

W.Weaver: The Mathematical Theory of Information;

A spectral multi-resolution image encoding network

A spectral multi-resolution image encoding network

Abstract

Similar works

Full text

Available Versions

Hochschulschriftenserver - Universität Frankfurt am Main