Hierarchical fusion using vector quantization for visualization of hyperspectral images

Visualization of hyperspectral images that combines the data from multiple sensors is a major challenge due to huge data set. An efficient image fusion could be a primary key step for this task. To make the approach computationally efficient and to accommodate a large number of image bands, we propose a hierarchical fusion based on vector quantization and bilateral filtering. The consecutive image bands in the hyperspectral data cube exhibit a high degree of feature similarity among them due to the contiguous and narrow nature of the hyperspectral sensors. Exploiting this redundancy in the data, we fuse neighboring images at every level of hierarchy. As at the first level, the redundancy between the images is very high we use a powerful compression tool, vector quantization, to fuse each group. From second level onwards, each group is fused using bilateral filtering. While vector quantization removes redundancy, bilateral filter retains even the minor details that exist in individual image. The hierarchical fusion scheme helps in accommodating a large number of hyperspectral image bands. It also facilitates the midband visualization of a subset of the hyperspectral image cube. Quantitative performance analysis shows the effectiveness of the proposed method

Assessment of the performance of image fusion for the mapping of snow

The assessment of the performance of multi-resolution image fusion, or image sharpening methods, is difficult. In the context of binary classification of snow targets in mountainous terrain, fusion methods were applied to help achieve more accurate mapping. To quantify objectively the gain of information that can be attributed to an increase in spatial resolution, we investigate the Mean Euclidean Distance (MED) between the snowline obtained from the classification, and a reference snowline (or a ground truth line), as a relevant indicator to measure both the discrepancy between datasets at different spatial resolutions, and the accuracy of the mapping process. First, a theoretical approach based on aggregating detailed reference images from the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) showed that the MED has a linear relationship with the pixel size that makes it suitable to assess images of different resolutions. Secondly, we tested the MED to snow maps obtained ‘with’ or ‘without’ applying a fusion method to the MODerate Resolution Imaging Spectroradiometer (MODIS). We demonstrated that the MED identified a significant value added, in terms of mapping accuracy, which can be attributed to the fusion process. When the fusion method was applied to four different images, the MED overall decreased by more than 30%. Finally, such a ‘feature based’ quality indicator can also be interpreted as a statistical assessment of the planimetric accuracy of natural pattern outlines.PublishedPeer ReviewedAiazzi, B., Alparone, L., Argenti, F. & Baronti, S. (1999). “Wavelet and pyramid techniques for multisensor data fusion: a performance comparison varying with scale ratios” In S. B. Serpico (ed.), Proceedings of the SPIE Image Signal Process. For Remote Sensing V. Vol. 3871 SPIE SPIE. pp. 251–262. Alparone, L., Baronti, S., Garzelli, A. & Nencini, F. (2004). “A global quality measurement of pan-sharpened multispectral imagery” IEEE Geoscience and Remote Sensing Letters. 1(4): 313–317. Alparone, L., Wald, L., Chanussot, J., Thomas, Gamba, P. & Bruce, L. M. (2007). “Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data fusion contest” IEEE Transactions on Geoscience and Remote Sensing. 45(10): 3012–3021. Amolins, K., Zhang, Y. & Dare, P. (2007). “Wavelet based image fusion techniques – An introduction, review and comparison” Journal of Photogrammetry and Remote Sensing. 62(4): 249–263. Bendjoudi, H. (2002). “The gravelius compactness coefficient: critical analysis of a shape index for drainage basins” Hydrological Sciences Journal. 47(6): 921–930. Chen, H. & Varshney, P. K. (2007). “A human perception inspired quality metric for image fusion based on regional information” Information Fusion. 8(2): 193–207. Crane, R. G. & Anderson, M. R. (1984). “Satellite discrimination of snow / cloud surfaces” International Journal of Remote Sensing. 5: 213–223. Du, Y., Vachon, P. W. & van der Sanden, J. J. (2003). “Satellite image fusion with multiscale wavelet analysis for marine applications: preserving spatial information and minimizing artifacts (PSIMA)” Canadian Journal of Remote Sensing. 29(1): 14–23. Gao, X., Wang, T. & Li, J. (2005). “A content-based image quality metric” Lecture Notes in Computer Science. 3642: 231–240. to get. Garguet-Duport, B., Girel, J., Chassery, J.-M. & Pautou, G. (1996). “The use of multi-resolution analysis and wavelet transform for merging SPOT panchromatic and multispectral imagery data” Photogrammetric Engineering and Remote Sensing. 62(9): 1057–1066. González-Audicana, M., Otazu, X., Fors, O. & Alvarez-Mozos, J. (2006). “A low computational-cost method to fuse IKONOS images using the spectral response function of its sensors” IEEE Transactions on Geoscience and Remote Sensing. 44(6): 1683–1691. González-Audicana, M., Saleta, J. L., Catalán, R. G. & Garc´ıa, R. (2004). “Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition” IEEE Transactions on Geoscience and Remote Sensing. 42(6): 1291–1299. Goodchild, M. (1980). “Fractals and the accuracy of geographical measures” Mathematical Geology. 12(2): 85–98. Goodchild, M. F. & Mark, D. M. (1987). “The fractal nature of geographic phenomena” Annals of Association of American Geographers. 77(2): 265–278. Jin, X. Y. & Davis, C. H. (2005). “Automated building extraction from high-resolution satellite imagery in urban areas using structural, contextual, and spectral information” EURASIP Journal of Applied Signal Processing. 2005(14): 2196–2206. Keshava, N. (2003). “A survey of spectral unmixing algorithms” Lincoln Laboratory Journal. 14(1): 55–78. Lam, N. S.-N. & Quattrochi, D. A. (1992). “On the issues of scale, resolution, and fractal analysis in the mapping sciences” Professional Geographer. 44(1): 88–98. Laporterie-Déjean, F., de Boissezon, H., Flouzat, G. & Lefèvre-Fonollosa, M.-J. (2005). “Thematic and statistical evaluations of five panchromatic/multispectral fusion methods on simulated PLEIADES-HR images” Information Fusion. 6(3): 193–212. Lasaponara, R. & Masini, N. (2005). “QuickBird-based analysis for the spatial characterization of archaeological sites: case study of the Monte Serico medieval village” Geophysical Research Letter. 32(12): L12313. Li, X., He, H. S., Bu, R., Wen, Q., Chang, Y., Hu, Y. & Li, Y. (2005). “The adequacy of different landscape metrics for various landscape patterns” Pattern Recognition. 38(12): 2626–2638. Malpica, J. A. (2007). “Hue adjustment to IHS pan-sharpened IKONOS imagery for vegetation enhancement” IEEE Geoscience and Remote Sensing Letters. 4(1): 27–31. Nichol, J. & Wong, M. S. (2005). “Satellite remote sensing for detailed landslide inventories using change detection and image fusion” International Journal of Remote Sensing. 26(9): 1913–1926. Pasqualini, V., Pergent-Martini, C., Pergent, G., Agreil, M., Skoufas, G., Sourbes, L. & Tsirika, A. (2005). “Use of SPOT 5 for mapping seagrasses: An application to Posidonia oceanica” Remote Sensing of Environment. 94(1): 39–45. Petrovic, V. (2007). “Subjective tests for image fusion evaluation and objective metric validation” Information Fusion. 8(2): 208–216. Pohl, C. & Genderen, J. L. V. (1998). “Multisensor image fusion in remote sensing: concepts, methods and applications” International Journal of Remote Sensing. 19(5): 823–854. Ranchin, T., Aiazzi, B., Alparone, L., Baronti, S. & Wald, L. (2003). “Image fusion–the ARSIS concept and some successful implementation schemes” Journal of Photogrammetry and Remote Sensing. 58: 4–18. Ranchin, T. & Wald, L. (2000). “Fusion of high spatial and spectral resolution images : the ARSIS concept and its implementation” Photogrammetric Engineering and Remote Sensing. 66(1): 49–61. Richter, R. (1998). “Correction of satellite imagery over mountainous terrain” Applied Optics. 37(18): 4004–4015. Shi, W., Zhu, C., Tian, Y. & Nichol, J. (2005). “Wavelet-based image fusion and quality assessment” International Journal of Applied Earth Observation and Geoinformation. 6: 241–251. Sirguey, P., Mathieu, R., Arnaud, Y., Khan, M. M. & Chanussot, J. (2008). “Improving MODIS spatial resolution for snow mapping using wavelet fusion and ARSIS concept” IEEE Geoscience and Remote Sensing Letters. 5(1): doi:10.1109/LGRS.2007.908884. In Press. Toet, A. & Franken, E. M. (2003). “Perceptual evaluation of different image fusion schemes” Displays. 24(1): 25–37. Tu, T.-M., Huang, P. S., Hung, C.-L. & Chang, C.-P. (2004). “A fast Intensity–Hue–Saturation fusion technique with spectral adjustment for IKONOS imagery” IEEE Geoscience and Remote Sensing Letters. 1(4): 309–312. Wald, L. (2000). “Quality of high resolution synthesized images: is there a simple criterion?” Proceedings of the International Conference on Fusion of Earth Data. SEE Gr éca Sophia Antipolis, France pp. 99–105. Wald, L., Ranchin, T. & Mangolini, M. (1997). “Fusion of satellite images of different spatial resolution: assessing the quality of resulting images” Photogrammetric Engineering and Remote Sensing. 63(6): 691–699. Wang, Z. & Bovik, A. C. (2002). “A universal image quality index” IEEE Signal Processing Letters. 9(3): 81–84. Woodcock, C. E. & Strahler, A. H. (1987). “The factor of scale in remote sensing” Remote Sensing of Environment. 21(3): 311–332. Xydeas, C. S. & Petrovic, V. (2000). “Objective image fusion performance measure” Electronics Letters. 36(4): 308–309. Zhai, G., Zhang, W., Yang, X. & Xu, Y. (2005). “Image quality assessment metrics based on multi-scale edge presentation” Proceedings of the IEEE Workshop on Signal Processing Systems Design and Implementation. IEEE IEEE. pp. 331–336. Zhan, Q., Molenaar, M., Tempfli, K. & Shi, W. (2005). “Quality assessment for geo-spatial objects derived from remotely sensed data” International Journal of Remote Sensing. 26(14): 2953–2974. Zhang, W. J. & Kang, J. Y. (2006). “QuickBird panchromatic and multi-spectral image fusion using wavelet packet transform” Intelligent Control and Automation. 344: 976–981. Zhang, Y. (2004). “Understanding image fusion” Photogrammetric Engineering and Remote Sensing. 70(6): 657–661

Assessment of the performance of image fusion for the mapping of snow

The assessment of the performance of multi-resolution image fusion, or image sharpening methods, is difficult. In the context of binary classification of snow targets in mountainous terrain, fusion methods were applied to help achieve more accurate mapping. To quantify objectively the gain of information that can be attributed to an increase in spatial resolution, we investigate the Mean Euclidean Distance (MED) between the snowline obtained from the classification, and a reference snowline (or a ground truth line), as a relevant indicator to measure both the discrepancy between datasets at different spatial resolutions, and the accuracy of the mapping process. First, a theoretical approach based on aggregating detailed reference images from the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) showed that the MED has a linear relationship with the pixel size that makes it suitable to assess images of different resolutions. Secondly, we tested the MED to snow maps obtained ‘with’ or ‘without’ applying a fusion method to the MODerate Resolution Imaging Spectroradiometer (MODIS). We demonstrated that the MED identified a significant value added, in terms of mapping accuracy, which can be attributed to the fusion process. When the fusion method was applied to four different images, the MED overall decreased by more than 30%. Finally, such a ‘feature based’ quality indicator can also be interpreted as a statistical assessment of the planimetric accuracy of natural pattern outlines.PublishedPeer ReviewedAiazzi, B., Alparone, L., Argenti, F. & Baronti, S. (1999). “Wavelet and pyramid techniques for multisensor data fusion: a performance comparison varying with scale ratios” In S. B. Serpico (ed.), Proceedings of the SPIE Image Signal Process. For Remote Sensing V. Vol. 3871 SPIE SPIE. pp. 251–262. Alparone, L., Baronti, S., Garzelli, A. & Nencini, F. (2004). “A global quality measurement of pan-sharpened multispectral imagery” IEEE Geoscience and Remote Sensing Letters. 1(4): 313–317. Alparone, L., Wald, L., Chanussot, J., Thomas, Gamba, P. & Bruce, L. M. (2007). “Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data fusion contest” IEEE Transactions on Geoscience and Remote Sensing. 45(10): 3012–3021. Amolins, K., Zhang, Y. & Dare, P. (2007). “Wavelet based image fusion techniques – An introduction, review and comparison” Journal of Photogrammetry and Remote Sensing. 62(4): 249–263. Bendjoudi, H. (2002). “The gravelius compactness coefficient: critical analysis of a shape index for drainage basins” Hydrological Sciences Journal. 47(6): 921–930. Chen, H. & Varshney, P. K. (2007). “A human perception inspired quality metric for image fusion based on regional information” Information Fusion. 8(2): 193–207. Crane, R. G. & Anderson, M. R. (1984). “Satellite discrimination of snow / cloud surfaces” International Journal of Remote Sensing. 5: 213–223. Du, Y., Vachon, P. W. & van der Sanden, J. J. (2003). “Satellite image fusion with multiscale wavelet analysis for marine applications: preserving spatial information and minimizing artifacts (PSIMA)” Canadian Journal of Remote Sensing. 29(1): 14–23. Gao, X., Wang, T. & Li, J. (2005). “A content-based image quality metric” Lecture Notes in Computer Science. 3642: 231–240. to get. Garguet-Duport, B., Girel, J., Chassery, J.-M. & Pautou, G. (1996). “The use of multi-resolution analysis and wavelet transform for merging SPOT panchromatic and multispectral imagery data” Photogrammetric Engineering and Remote Sensing. 62(9): 1057–1066. González-Audicana, M., Otazu, X., Fors, O. & Alvarez-Mozos, J. (2006). “A low computational-cost method to fuse IKONOS images using the spectral response function of its sensors” IEEE Transactions on Geoscience and Remote Sensing. 44(6): 1683–1691. González-Audicana, M., Saleta, J. L., Catalán, R. G. & Garc´ıa, R. (2004). “Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition” IEEE Transactions on Geoscience and Remote Sensing. 42(6): 1291–1299. Goodchild, M. (1980). “Fractals and the accuracy of geographical measures” Mathematical Geology. 12(2): 85–98. Goodchild, M. F. & Mark, D. M. (1987). “The fractal nature of geographic phenomena” Annals of Association of American Geographers. 77(2): 265–278. Jin, X. Y. & Davis, C. H. (2005). “Automated building extraction from high-resolution satellite imagery in urban areas using structural, contextual, and spectral information” EURASIP Journal of Applied Signal Processing. 2005(14): 2196–2206. Keshava, N. (2003). “A survey of spectral unmixing algorithms” Lincoln Laboratory Journal. 14(1): 55–78. Lam, N. S.-N. & Quattrochi, D. A. (1992). “On the issues of scale, resolution, and fractal analysis in the mapping sciences” Professional Geographer. 44(1): 88–98. Laporterie-Déjean, F., de Boissezon, H., Flouzat, G. & Lefèvre-Fonollosa, M.-J. (2005). “Thematic and statistical evaluations of five panchromatic/multispectral fusion methods on simulated PLEIADES-HR images” Information Fusion. 6(3): 193–212. Lasaponara, R. & Masini, N. (2005). “QuickBird-based analysis for the spatial characterization of archaeological sites: case study of the Monte Serico medieval village” Geophysical Research Letter. 32(12): L12313. Li, X., He, H. S., Bu, R., Wen, Q., Chang, Y., Hu, Y. & Li, Y. (2005). “The adequacy of different landscape metrics for various landscape patterns” Pattern Recognition. 38(12): 2626–2638. Malpica, J. A. (2007). “Hue adjustment to IHS pan-sharpened IKONOS imagery for vegetation enhancement” IEEE Geoscience and Remote Sensing Letters. 4(1): 27–31. Nichol, J. & Wong, M. S. (2005). “Satellite remote sensing for detailed landslide inventories using change detection and image fusion” International Journal of Remote Sensing. 26(9): 1913–1926. Pasqualini, V., Pergent-Martini, C., Pergent, G., Agreil, M., Skoufas, G., Sourbes, L. & Tsirika, A. (2005). “Use of SPOT 5 for mapping seagrasses: An application to Posidonia oceanica” Remote Sensing of Environment. 94(1): 39–45. Petrovic, V. (2007). “Subjective tests for image fusion evaluation and objective metric validation” Information Fusion. 8(2): 208–216. Pohl, C. & Genderen, J. L. V. (1998). “Multisensor image fusion in remote sensing: concepts, methods and applications” International Journal of Remote Sensing. 19(5): 823–854. Ranchin, T., Aiazzi, B., Alparone, L., Baronti, S. & Wald, L. (2003). “Image fusion–the ARSIS concept and some successful implementation schemes” Journal of Photogrammetry and Remote Sensing. 58: 4–18. Ranchin, T. & Wald, L. (2000). “Fusion of high spatial and spectral resolution images : the ARSIS concept and its implementation” Photogrammetric Engineering and Remote Sensing. 66(1): 49–61. Richter, R. (1998). “Correction of satellite imagery over mountainous terrain” Applied Optics. 37(18): 4004–4015. Shi, W., Zhu, C., Tian, Y. & Nichol, J. (2005). “Wavelet-based image fusion and quality assessment” International Journal of Applied Earth Observation and Geoinformation. 6: 241–251. Sirguey, P., Mathieu, R., Arnaud, Y., Khan, M. M. & Chanussot, J. (2008). “Improving MODIS spatial resolution for snow mapping using wavelet fusion and ARSIS concept” IEEE Geoscience and Remote Sensing Letters. 5(1): doi:10.1109/LGRS.2007.908884. In Press. Toet, A. & Franken, E. M. (2003). “Perceptual evaluation of different image fusion schemes” Displays. 24(1): 25–37. Tu, T.-M., Huang, P. S., Hung, C.-L. & Chang, C.-P. (2004). “A fast Intensity–Hue–Saturation fusion technique with spectral adjustment for IKONOS imagery” IEEE Geoscience and Remote Sensing Letters. 1(4): 309–312. Wald, L. (2000). “Quality of high resolution synthesized images: is there a simple criterion?” Proceedings of the International Conference on Fusion of Earth Data. SEE Gr éca Sophia Antipolis, France pp. 99–105. Wald, L., Ranchin, T. & Mangolini, M. (1997). “Fusion of satellite images of different spatial resolution: assessing the quality of resulting images” Photogrammetric Engineering and Remote Sensing. 63(6): 691–699. Wang, Z. & Bovik, A. C. (2002). “A universal image quality index” IEEE Signal Processing Letters. 9(3): 81–84. Woodcock, C. E. & Strahler, A. H. (1987). “The factor of scale in remote sensing” Remote Sensing of Environment. 21(3): 311–332. Xydeas, C. S. & Petrovic, V. (2000). “Objective image fusion performance measure” Electronics Letters. 36(4): 308–309. Zhai, G., Zhang, W., Yang, X. & Xu, Y. (2005). “Image quality assessment metrics based on multi-scale edge presentation” Proceedings of the IEEE Workshop on Signal Processing Systems Design and Implementation. IEEE IEEE. pp. 331–336. Zhan, Q., Molenaar, M., Tempfli, K. & Shi, W. (2005). “Quality assessment for geo-spatial objects derived from remotely sensed data” International Journal of Remote Sensing. 26(14): 2953–2974. Zhang, W. J. & Kang, J. Y. (2006). “QuickBird panchromatic and multi-spectral image fusion using wavelet packet transform” Intelligent Control and Automation. 344: 976–981. Zhang, Y. (2004). “Understanding image fusion” Photogrammetric Engineering and Remote Sensing. 70(6): 657–661