Wavelet transforms using matrix-valued wavelets (MVWs) can process the components of vector-valued signals jointly, and thus offer potential advantages over scalar wavelets. For every matrix-valued scaling filter, there are infinitely many matrix-valued wavelet filters corresponding to rotated bases. We show how the arbitrary orthogonal factor in the choice of wavelet filter can be selected adaptively with a modified SIMPLIMAX algorithm. The 3×3 orthogonal matrix-valued scaling filters of length 6 with 3 vanishing moments have one intrinsic free scalar parameter in addition to three scalar rotation parameters. Tests suggest that even when optimising over these parameters, no significant improvement is obtained when compared to the naive scalar-based filter. We have found however in an image compression test that, for the naive scaling filter, adaptive basis rotation can decrease the RMSE by over 20%

Ginzberg, P

Walden, AT

Spiral - Imperial College Digital Repository

ADAPTIVE ORTHOGONAL MATRIX-VALUED WAVELETS AND COMPRESSION OFVECTOR-VALUED SIGNALSP. Ginzberg and A. T. WaldenDepartment of Mathematics, Imperial College London,180 Queen’s Gate, London SW7 2BZ, UK.(e-mail: paul.ginzberg05@imperial.ac.uk and a.walden@imperial.ac.uk)ABSTRACTWavelet transforms using matrix-valued wavelets (MVWs)can process the components of vector-valued signals jointly,and thus offer potential advantages over scalar wavelets. Forevery matrix-valued scaling filter, there are infinitely manymatrix-valued wavelet filters corresponding to rotated bases.We show how the arbitrary orthogonal factor in the choiceof wavelet filter can be selected adaptively with a modifiedSIMPLIMAX algorithm. The3×3 orthogonal matrix-valuedscaling filters of length6 with 3 vanishing moments haveone intrinsic free scalar parameter in addition to three scalarrotation parameters. Tests suggest that even when optimis-ing over these parameters, no significant improvement is ob-tained when compared to the naive scalar-based filter. Wehave found however in an image compression test that, forthe naive scaling filter, adaptive basis rotation can decreasethe RMSE by over 20%.Index Terms— multichannel wavelet, vector-valuedwavelet, matrix-valued wavelet, basis rotation, SIMPLIMAX,compression, scalar thresholding1. INTRODUCTIONThe naive approach for applying wavelet-based methodsto vector-valued data is to transform each component in-dependently with a scalar wavelet transform. Ann × nmatrix-valued wavelet (MVW) is a type of wavelet which isspecifically designed to jointly transform the components ofn-vector-valued signals [6, 14]. The coefficients of a matrix-valued scaling filter (MVSF) or matrix-valued wavelet filter(MVWF) aren × n matrices. The increased number of de-grees of freedom offered by such filters allows one, for exam-ple, to build finite impulse-response (FIR) MVSFs which areorthogonal, symmetric, and have high vanishing moments,such as the quaternion (4× 4) construction in [7].In a search of the MVW literature, we have come acrossonly four explicit MVW designs of practical interest. [4] de-vised a procedure based on multichannel lifting to constructPaul Ginzberg thanks the EPSRC (UK) for financial support.biorthogonal MVWs, and gives coefficients for the2×2 case.A 3×3 example based on the same method is given in [2]. [8]construct two examples of biorthogonal MVWs by solving aset of design equations symbolically. The construction from[2] has been applied to the compression, denoising [1] andwatermarking [3] of colour images. The construction from[8] has been applied to the compression and denoising of 2-Dvector wind fields [9, 13]. In addition to these, the authorshave constructed2 × 2, and quaternion (4 × 4) orthogonalMVWs [7].One characteristic which all these constructions share isthat they contain free parameters which must be specified. In[8], the free parameters are chosen such that the scaling andwavelet filters resemble ideal lowpass and highpass filters asclosely as possible. In [2, Fig. 7] the performance for a fewparameter choices are compared. In this paper, we develop amethod which allows us to systematically select the free pa-rameter in the orthogonal3 × 3 construction based on [7],in order to optimise its performance for signal compression.Since the optimisation can be performed for a specific sig-nal, this can be considered as a method for implementing anadaptive wavelet transform. However, whilst the adaptive op-timisation of the wavelet filter for a given scaling filter canbedone in a computationally efficient manner, we will use bruteforce (trying a large number of parameter values) to optimisethe scaling filter.In Section 2 we introduce MVWs. In Sections 3, 4 and 5we classify the three types of free parameters. These are, re-spectively, an arbitrary orthogonal similarity transformationof the scaling filter, an intrinsic parameter in the scaling filterdesign, and an arbitrary rotation of the wavelet filter whichcontrols the wavelet coefficient basis. We suggest an algo-rithm for the adaptive optimisation of the latter in Section6.Section 4 describes the set of all3× 3 orthogonal MVSFs oflength6 with 3 vanishing moments. In Section 7 we systemat-ically test the effects of parameter choices for these filters ona test image, which leads to some insights on MVW design.2. MATRIX-VALUED WAVELETSA (discrete) MVW transform decomposes a vector-valuedsignalf(t) ∈ L2(R,R1×n) into a linear combinationf(t) =∑k∈Zsk2-J/2Φ(2-J t-k) +∑k∈Z,j<Jwj,k2-j/2Ψ(2-jt-k)(1)of the translations and dilations of a matrix-valued (MV)scaling functionΦ(t) ∈ L2(R,Rn×n), and a MV waveletfunctionΨ(t) ∈ L2(R,Rn×n), with coefficientssk,wj,k ∈R1×n.Φ andΨ satisfy the dilation equationsΦ(t) =√2∑k∈ZGkΦ(2t−k), Ψ(t) =√2∑k∈ZHkΦ(2t−k),where{Gk} and{Hk} aren × n matrix-valued sequences,called the matrix-valued scaling filter (MVSF) and matrix-valued wavelet filter (MVWF) respectively.MVWs are a type of generalized multiwavelet. Indeed,the MVW transform can be implemented as a fast multi-wavelet transform. There is however no need for vector-ization, pre-filtering or post-fitlering steps since the sig-nal is already in vector form. MVSF coefficients satisfy2−12∑k∈Z Gk = In. This sets them apart from standardmultiwavelets, for which the sum has one eigenvalue equalto 1, and all other eigenvalues strictly less than1 in absolutevalue.In this paper, we will deal only with orthogonal MVWs,i.e. MVWs for which the basis ofL2(R,R1×n) used in thedecomposition (1) is orthonormal. Also, we will only dealwith MVSFs{Gk} having finite lengthL, i.e. Gk 6= 0n×nonly for 0 ≤ k < L. Particular attention will be given to thecasen = 3 andL = 6.3. ORTHOGONAL SIMILARITYTRANSFORMATIONSDefinition 1 Two filters {Gk} and {Jk} are orthogonallysimilar iffJk = OGkOT , ∀k ∈ Z (2)for some orthogonal matrixO, (i.e.,OOT = In).The map{Gk} 7→ {OGkOT } is called an orthogonalsimilarity transformation (OST).OSTs preserve orthogonality, filter length and vanishing mo-ments [7].For a given MVSF{Gk}, we can generate a whole fam-ily of MVSFs {OGkOT } by taking OSTs. It is convenient togroup MVSFs into such orthogonally similar families, whichcan be described by an arbitrarily chosen representative ele-ment.Given a scaling and wavelet filter pair{Gk}, {Hk}, wewill apply any OST to both filters, to obtain a valid scalingand wavelet filter pair{OGkOT }, {OHkOT }.Let O(3) denote the set of3× 3 orthogonal matrices andSO(3) = {O ∈ O(3) : det(O) = 1} denote the set of3 × 3rotation matrices. ThenO(3) = SO(3) ∪ (−SO(3)). How-ever, for anyGk,O ∈ R3×3, (−O)Gk(−O)T = OGkOT .Hence, we only need to consider OSTs with rotationsO ∈SO(3).We can parameterise the rotationsO ∈ SO(3) using 3Euler angles(θ1, θ2, θ3) ∈]−π, π]× [0, π]×]−π, π]. Given aMVSF {Gk}, we will want to select an optimal MVSF withinthe family of orthogonally similar filters that it generates, bychoosing appropriate values for the parametersθ1, θ2, θ3.If we wish to select these parametersbeforeobserving thesignal to be compressed (i.e. non-adaptively), and the prop-erties of the unknown signal are a-priori invariant under ro-tations (e.g. the coordinate system used for the signal is un-known and arbitrary), then the choice of OST is irrelevant andwe may arbitrarily setθ1 = θ2 = θ3 = 0, (O = I3).4. INTRINSIC PARAMETERSAfter considering OSTs, there may still be additional free pa-rameters in the design of the MVSF. For the set of3 × 3MVSFs of length6 with 3 vanishing moments there is onesuch free parameter, denotedx⋆.We can describe the set of all orthogonal3× 3 MVSFs oflength6 with 3 vanishing moments as follows:There are two naive filters. One is given by{gkI3}, where{gk} is the scalar minimum phase Daubechies scaling filter,and the other by its time-reversal{g5−kI3}.The non-naive filters are either orthogonally similar togk 0 000Jk(x), (3)or to its time-reversal, wheregk is as above, andJk is the non-trivial 2× 2 MVSF construction of length6 with 3 vanishingmoments, given in [7] with free parameter0 ≤ x ≤ C =[5 + 2√10]1/2;We treat the non-naive filters as a single family, parame-terised by−1 ≤ x⋆ ≤ 1 as follows: If 0 ≤ x⋆ ≤ 1 thenselect (3)., withx = Cx⋆. If −1 ≤ x⋆ < 0 then take thetime-reversal of (3) withx = −Cx⋆.5. BASIS SELECTION AND ROTATION OF THEWAVELET FILTERFor a given MVSF{Gk}, a corresponding MVWF{Hk} canbe computed using the method described in [10, Thm. 10.2,Coroll. 9.2] (see also [7]). However, the choice of MVWFfor a given MVSF is not unique. Indeed, any filter of theform {RHk} whereR is an orthogonal matrix is also valid.Hence, we may wish to optimise the choice ofR.Consider the matrixW whose rows are given by the var-ious wavelet coefficientswj,k obtained from (1). (We as-sume that in practice the signal being transformed is finiteand discrete, so that there are only finitely many wavelet co-efficients.) Then the matrix of wavelet coefficients obtainedby using{RHk} as our wavelet filter instead of{Hk} (andhenceRΨ instead ofΨ) will simply be WRT . In otherwords, choosingR is equivalent to selecting the orthonormalbasis under which we will encode the wavelet coefficients.When applying the MVW transform to images (or moregenerally using transforms with more than one time dimen-sion or wavelet packet transforms) the effects of rotating thewavelet filter or rotating the wavelet coefficient basis are sub-tly different due to further filtering being applied after thewavelet filter. Thus, treating this situation in its full gener-ality requires that we consider two separate rotation parame-tersR. We will avoid this complication resulting from thenon-commutative interaction between vertical and horizon-tal transform components by considering only the problemof finding an optimal rotation of the wavelet coefficient basis.This is the more tractable rotation to optimise, since rotatedwavelet coefficients can be obtained without recomputing theMVW transform.For certain applications, such as those based on vectorthresholding, the choice of basis is irrelevant. We can thenarbitrarily chooseR = In. In the context of compressionby scalar thresholding however, selecting an appropriate bsiscan significantly improve performance. Whenn = 3, sincechoosingR = −I3 will not affect results, we again needonly considerR ∈ SO(3), parameterised by three Euler an-glesθ̃1, θ̃2, θ̃3. (Since inversions and permutations of the axeswill not affect results, we could decide to restrict the 3D rangeof (θ̃1, θ̃2, θ̃3) by a factor of24. This is done by “quotientingout” the rotation group of the cube fromSO(3).)6. MODIFIED SIMPLIMAX ALGORITHMLet τp : Rm×n → Rm×n denote the hard scalar thresh-olding operator which sets the100p% smallest entries of awavelet coefficient matrixW to 0. We wish to minimisetheL2 distance between the original signal and the signal re-constructed from the thresholded coefficients. We call thisquantity the root mean squared error (RMSE). Since the or-thogonal wavelet transform is an isometry, this is given byRMSE =∣∣∣∣τp(WRT )−WRT∣∣∣∣2, where||•||2denotes theFrobenius norm. The problem of minimizing this quantityoverR ∈ O(3) can be solved by a simpler orthogonal vari-ant of the SIMPLIMAX algorithm used in factor analysis, ashinted at in [11, p. 578]. The algorithm is based on [5, Case II]and proceeds as follows:Start from an initial guessR0 and recursively setRk+1 =UkVTk Rk, whereUk and Vk are obtained from the sin-gular value decompositionMk = UkDkV Tk of Mk =RkWT τp(WRTk ). The RMSE decreases at each iteration,until convergence.Like many non-convex optimisation routines, this proce-dure suffers from the fact that it may converge to a local min-imum. To mitigate this problem, random initial guesses areused in addition to the default choiceR0 = In. In our ap-plications, we computed the RMSE for 2000 randomR, andselected the best 4 rotations as additional random startingval-uesR0.Uniformly distributed random rotations are generated us-ing the rotation-invariant (Haar) measure [12].Remark 1 The same algorithm can be applied with quanti-zation operators other thanτp.7. NUMERICAL RESULTS AND INTERPRETATIONWe will take as our signalf the well known512 × 512 testcolour imageLenain 24-bit RGB format.We considered56 values ofx⋆ and (due to computationaltime constraints) 100 OSTs (O = I3 and a further 99 uni-formly distributed random OSTs). For each combination ofx⋆ andO, we computed the full MVW transform of the im-age and optimised the choice of wavelet coefficient basis ro-tationR through the modified SIMPLIMAX algorithm. Therelative RMSE,rRMSE = RMSE ||W ||−12, was computedafter thresholdingp = 90% of wavelet coefficients.The naive filter built from the minimum phase scalarDaubechies scaling filter of length 6 gives an rRMSE of8.75%. WhenO = R = I3, the lowest rRMSE is obtainedfor the diagonal MVSF corresponding tox⋆ = −1 and equals8.74%. To remove the influence of our choice of represen-tative element amongst orthogonally similar wavelets, weaverage results over the 100 OSTs. Then the lowest averagerRMSE obtained from non-naive filters is8.82%, for x⋆ = 0.Hence the unoptimised MVSFs are generally underperform-ing relative to the naive filter. We see from Fig. 1 that evenafter optimising the choice of bothO andx⋆, the decreasein rRMSE relative to the naive filter is less than2%. Againx⋆ = 0 is optimal.Optimisation overR on the other hand can provide a sig-nificant improvement in performance at a much lower compu-tational cost. This optimisation is however particularly effec-tive for the naive filter, leading to a12.9% decrease in rRMSEto 7.62%. Again, MVSFs underperform.Experiments on the512 × 512 imagesmandrill, peppersandairplanegive qualitatively similar results to Fig. 1, exceptfor different ranges of rRMSE. The values for the naive filterbefore and after optimisation are given in Table 1.We believe that optimisation overR is particularly effec-tive for the naive filters because the phases of the filters ap-plied to each component match, leading to better alignmentof the large wavelet coefficients across the 3 columns ofW .This explanation is consistent with the fact that optimisationoverR is more effective forx⋆ = 0 andx⋆ = ±1, values at−1 −0.5 0 0.5 17.67.888.28.48.68.899.29.4x*relative RMSE (%)Fig. 1. Relative RMSE after settingp = 90% of coefficientsin the MVW transform ofLenato 0, for varyingx⋆ and var-ious degrees of optimisation. From top to bottom, the dash-dotted curve is for no optimisation (averaged over OSTs), thedotted curve is after optimisingO, the dashed curve is af-ter optimisingR (averaged over OSTs), the full curve is af-ter jointly optimising bothO andR. The horizontal linescorrespond to the naive minimum-phase filter, before (squaremarkers) and after (round markers) optimisation ofR.which two out of the three filter dimensions will have match-ing phases, in some sense. Lack of proper alignment of thewavelet coefficients is also problematic for applications ba edon vector thresholding, and may be at the root of the overalldisappointing performance of the non-naive3 × 3 wavelets.Although symmetric (zero phase) MVWs exist forn = 2, 4,currently no example exists for oddn.Optimisation ofR is useful because the distribution ofwavelet coefficients inR3 is anisotropic. Indeed, for naivewavelet filters, the wavelet coefficients which encode a sharpedge between two uniformly coloured regions will lie alonga line through the origin. One of the reasons for the lessereffectiveness of basis selection for MVWs may be that theydo not exhibit this behavior. If we treat the anisotropy as el-lipsoidal, then the major and minor axes provide a heuristicchoice of basis. In other words we may chooseR such thatW TW = RTDR, with D diagonal. This heuristic can alsobe used as a starting guess for the SIMPLIMAX algorithm.8. REFERENCES[1] S. Agreste and A. Vocaturo, “Multichannel waveletscheme for color image processing.” InApplied andIndustrial Mathematics in Italy III: Selected Contribu-image rRMSE optimised improvement(%) rRMSE (%) (%)Lena 8.75 7.65 12.9mandrill 25.3 19.7 22.1peppers 8.25 8.02 2.86airplane 7.88 5.71 27.5Table 1. rRMSE obtained for the naive Daubechies filter be-fore and after optimisation ofR. p = 90% of the waveletcoefficients are set to 0.tions from the 9th SIMAI Conference, Rome, Italy 15-19September 2008,pp. 1–12, 2009.[2] S. Agreste and A. Vocaturo, “A new class of full rankfilters in the context of digital color image processing.”In Proceedings of the 10th European Congress of ISS,Bologna, Italy, 2009, pp. 1–6.[3] S. Agreste and A. Vocaturo, “Wavelet and multichannelwavelet based watermarking algorithms for digital colorimages.” InCommunications to SIMAI Congress, vol. 3,2009, pp. 242.1–242.11.[4] S. Bacchelli, M. Cotronei & T. Sauer, “Multifilters withand without prefilters,”BIT Numerical Mathematics,vol. 42, pp. 231–261, 2001.[5] N. Cliff, “Orthogonal rotation to congruence,”Psychome-trika, vol. 31, pp. 33–42, 1966.[6] J. E. Fowler and L. Hua, “Wavelet transforms for vectorfields using omnidirectionally balanced multiwavelets,”IEEE Trans. Signal Process., vol. 50, pp. 3018–3027,2002.[7] P. Ginzberg and A. T. Walden, “Matrix-valued andquaternion wavelets,” submitted toIEEE Trans. SignalProcess, 2012.[8] L. Hua and J. E. Fowler, “Technical details on a familyof omnidirectionally balanced symmetric-antisymmetricmultiwavelets.” Technical report, Mississippi StateUniversity. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.11.3802, 2002.[9] L. Hua and J. E. Fowler, “Wavelet-based coding oftime-varying vector fields of ocean-surface winds,”IEEETrans. Geosci. Remote Sensing,vol. 42, pp. 1283–1290.[10] F. Keinert,Wavelets and Multiwavelets. Chapman andHall/CRC, 2003.[11] H. A. L. Kiers, “Simplimax: Oblique rotation to an opti-mal target with simple structure,”Psychometrika,vol. 59,pp. 567–579, 1994.[12] R. E. Miles, “On random rotations inR3,” Biometrika,vol. 52, pp. 636–639, 1965.[13] M. A. Westenberg and E. Thomas, “Denoising 2-Dvector fields by vector wavelet thresholding,” J. WSCGvol. 13, pp. 33–40, 2005.[14] X.-G. Xia and B. W. Suter, “Vector-valued wavelets andvector filter banks,”IEEE Trans. Signal Process., vol. 44,pp. 508–18, 1996.

Adaptive Orthogonal Matrix-Valued Wavelets and Compression of Vector-Valued Signals

Abstract

Similar works

Full text

Available Versions

Spiral - Imperial College Digital Repository