Image compression is becoming more and more important, as new multispectral and hyperspectral instruments are going to generate very high data rates due to the increased spatial and spectral resolutions. Transmitting all the acquired data to the ground segment is a serious bottleneck, and compression techniques are a feasible solution to this problem. The CCSDS has established a working group (WG) on multispectral and Hyperspectral Data Compression (MHDC), which has the purpose of standardizing compression techniques to be used onboard. The WG has already standardized a lossless compression algorithm for multispectral and hyperspectral images, and has started working on a lossy compression algorithm. The complexity of lossless compression algorithms is typically larger than that of lossy ones, leading to potentially lower throughputs. Therefore, a careful assessment is required in order to identify techniques that are able to sustain very high data rates. The increased complexity can also lead to increased resource occupancy on a hardware device such as an FPGA. Lossy compression introduces information losses in the images, and these losses must be accurately characterized, and their effect on the applications investigated. For these reasons, developing a lossy algorithm requires a more elaborate process. Under an ESA contract primed by Politecnico of Torino, TSD is currently designing an IP core for FPGA and/or ASIC implementation of a lossy compression algorithm that is being proposed for CCSDS standardization. In addition to the IP core, TSD is developing a HW platform based on the Xilinx Virtex-5 XQR5VFX130, the industry's first high performance rad-hard reconfigurable FPGA for processing-intensive for space systems. Advanced results along with details of electronic platform design will be presented in this paper

E. Magli

G. Capuano

M. De Nino

M. Romano

English

MAGLI, ENRICO

PORTO@iris (Publications Open Repository TOrino - Politecnico di Torino)

04 August 2020POLITECNICO DI TORINORepository ISTITUZIONALELossy Multi/Hyperspectral Compression HW Implementation at high data rate / M. De Nino; M. Romano; G. Capuano; E.Magli. - (2014). ((Intervento presentato al convegno International Astronautical Congress.OriginalLossy Multi/Hyperspectral Compression HW Implementation at high data ratePublisher:PublishedDOI:Terms of use:openAccessPublisher copyright(Article begins on next page)This article is made available under terms and conditions as specified in the  corresponding bibliographic description inthe repositoryAvailability:This version is available at: 11583/2592673 since:International Astronautical Federation (IAF)IAC-14,B1,4,10,x25121  LOSSY MULTI/HYPERSPECTRAL COMPRESSION HW IMPLEMENTATION AT HIGH DATA RATE  Maurizio De Nino TSD-Space, Italy, mdenino@tsd-space.it  Giuseppe Capuano TSD-Space, Italy, gcapuano@tsd-space.it Mario Romano TSD-Space, Italy, mromano@tsd-space.it Enrico Magli Politecnico di Torino, Italy, enrico.magli@polito.it IAC-14,B1,4,10,x25121  page  1 of 6  ABSTRACT Image compression is becoming more and more important, as new multispectral and hyperspectral instruments are going to generate very high data rates due to the increased spatial and spectral resolutions. Transmitting all the acquired data to the ground segment is a serious bottleneck, and compression techniques are a feasible solution to this problem. The CCSDS has established a working group (WG) on Multispectral and Hyperspectral Data Compression (MHDC), which has the purpose of standardizing compression techniques to be used onboard. The WG has already standardized a lossless compression algorithm for multispectral and hyperspectral images, and has started working on a lossy compression algorithm. The complexity of lossless compression algorithms is typically larger than that of lossy ones, leading to potentially lower throughputs. Therefore, a careful assessment is required in order to identify techniques that are able to sustain very high data rates. The increased complexity can also lead to increased resource occupancy on a hardware device such as an FPGA. Lossy compression introduces information losses in the images, and these losses must be accurately characterized, and their effect on the applications investigated. For these reasons, developing a lossy algorithm requires a more elaborate process. Under an ESA contract primed by Politecnico of Torino, TSD is currently designing an IP core for FPGA and/or ASIC implementation of a lossy compression algorithm that is being proposed for CCSDS standardization. In addition to the IP core, TSD is developing a HW platform based on the Xilinx Virtex-5 XQR5VFX130, the industry's first high performance rad-hard reconfigurable FPGA for processing-intensive for space systems. Advanced results along with details of electronic platform design will be presented in this paper.  1. INTRODUCTION Compression of hyperspectral images has represented an important and active research topic for a long time. All spectral imagers such as those of the multispectral, hyperspectral and ultraspectral type generate extremely large amounts of data, making image compression mandatory in order to reduce the data volume before the transmission to the ground segment.  More recently, a new class of prediction-based approaches have been proposed for lossless [5] and lossy compression. The CCSDS Working Group (WG) has worked towards the definition of a new image compression standard for lossless compression of multispectral and hyperspectral images. This new lossless compression standard was issued as CCSDS 123 recommendation. Subsequently, the WG started a discussion about the lossy compression standard for multispectral and hyperspectral images. Two techniques have been proposed as candidates for this standard: one technique is based on the pair wise orthogonal transform [6]; the second candidate is an improved version of [7]. The second algorithm will be very competitive in terms of performance and complexity. In particular, while transform coding is a good approach for compression at the ground station, it is not a good fit to onboard compression, as it would pose large memory and computational requirements, leading to low throughput. On the other hand, the predictive lossy compression paradigm, on which [7] is based, is more suitable for onboard space applications.  2. COMPRESSION ALGORITHM In this paper, is presented an hardware implementation of a compression algorithm for multispectral and hyperspectral images based on the IAC-14,B1,4,10,x25121 predictive lossy compression paradigm. predictive lossy compression is able to achieve compression performance as good as the transformbased approach, at a fraction of its complexityproviding a very good fit to onboard image compression. Predictive lossy compression can be seen as an evolution of near-lossless compression.Near-lossless compression techniques are easily derived from lossless ones, by simplyuniform quantizer with feedback loop to the prediction residuals. Since the quantization error introduced on the prediction residual of one sample is identical to the reconstruction error that will be observed on that sample, these techniques allow to control errors in the compressed images in a very accurate and flexible way. The implemented algorithm can be considered as an extension of CCSDS 123 [8] with a quantization stage to achieve lossy compression entropy coder (range encoder) (See Figure 1)Prediction ResidualQuantization StageInputImage Figure 1: Logic block diagram of compressor3. FLIGHT HW PLATFORMThe test environment is based on a flight hardware to validate not only the algorithm but also its onboard implementation. The flight hardware is the (High performance Processing unit for Hyperspecral data Compression), belonging to a family of High performance Processing unit (HPxx) TSD for real time image data processing7).  The platform includes the following modules:• Image Processing Module (IPM) • Power Conditioning & Distribution Module (PCDM)  The - and   applying a and a modified . RangeEncoderCompressed Data  HPHC developed by  (See Figure    Figure 2: Basic block diagram of HPThe IPM is composed of four sections as shown figure 3:   • Image Processing A• Image Processing B• Data Handling A  • Data Handling B  Figure 3: Detailed scheme of IPMThe two Image Processing sections and the two Data Handling sections are identical and they can be configured: • in cold redundancy to provide high reliability• in master-slave mode to run in parallel so to improve the processing capabilitiesThe Data Handling section implements the control & communication functionalities of the unit, while the Image Processing section is dedicated to the page  2 of 6 HC in        IAC-14,B1,4,10,x25121 compression. Figure 4: IPM Mother board. Note on both sides the anti-fuse FPGAs As shown in the following Figure 3the Image Processor sections are redounded and connected in cross-strapped mode to both the Data Handling sections. Point-to point links are also available to  interconnect the Image Processing A section with the Image Processing B and the Data Handling A with the Data Handling B, thus allowing the sections to run in parallel.  Figure 5: IPM Daughter board. On both sides can be seen the two Virtex5 FPGAs  Each section of image processing adopts the Xilinx Virtex-5 XC5VFX130T (XQR5VFX130 for flight Model), the industry's first high performance radreconfigurable FPGA. Each FPGA is provided with 5Gbit SDRAMimage data inputs (1.575 Gbits/s each).  , the outputs of  -hard  and two   Figure 6: PCDM Figure 7: HPHC engineering model assembled in its cabinet 4. MHDC SOFT IPThe core of Multispectral and Hyperspectral Data Compressor is implemented only on Processor Image Unit A since a single Virtex5 chip has hardware resources that are more than sufficient for our purposes.  Figure 8 describes the basic block diagram of compressor. Figure 8: Basic block diagram of compressorpage  3 of 6   enough  IAC-14,B1,4,10,x25121 Once a Cube Of Images (COI) is receivedprocessed by the Grabber that transmits a pair of pixels at a time to the Prediction Unipixels belong to two adjacent rows column. For this reason, considering size of 4096 columns x 4096 bandsmaximum amount of memory necessary is 32Mbytes since it is necessary to store the previous horizontal plane of the COI. Figure 9 shows a generic COI; the current horizontal plane is received pixel by pixel and at same time the grabber stores these pixels on SDRAM and provides the pixels of previous horizontal plane by reading them from SDRAM. At the end of plane the cycle continues to the next plane until the end of COI. When the first plane is transmittedplane doesn't exists yet, therefore the pixel previous plane of the pair to transmit has its value forced to 0x0000.  Figure 9: Anatomy of a COI according to The PU calculates, for each pair of pixels in input, the predicted value of pixel belonging horizontal plane. The predicted value is subtractedthe real value and this residual value, after the quantization, thanks to its lower information content, can be compressed with higher efficiency. Figure 10 shows how the predicted value is calculated [2]. We tell Sz,y,x the current pixel, the related predicted value is calculated using the values of pixels of the five previous bands having the same x,y coordinates (Sz-1,y,x , Sz-2,y,x...... pixel of previous row Sz,y-1,x.  , it is t (PU). The two at the same the maximum  @16bits, the for grabbing , the previous of the BIL format to the current  to  Sz-5,y,x) and the The PU makes use of a SDRAM to store the weights vectors that are updated continuouslynew COI has to be processedweights vectors are written into SDRAM by means of the Prediction Unit Initializator.The stage after the prediction is the quantization. When the quantization is enabled, the compression mode is lossy. The compressor can operate in lossless mode by disabling the Quantizer.Figure 10: Pixels taken in accpredicted value of the currenAll the residuals (with or without quantization) encoded by the Range Encoder, an entropy ethat remove the numerical redundanciesof statistics. Figure 11 illustrates the simplified block diagram of the range encoderindependent statistic models (SM) stores and updates the following statistical data:• The cumulative frequency for each symbol• The frequency for each symbol• Total frequency defined as the sum of overall frequency of Different data structures can be usedcumulative frequencies, each advantages and drawbacks. was adopted the Cumulative Frequency Matrix (CFM) [1] which requires a limited HW resources page  4 of 6 . Every time a ,  the starting values of  - ount to calculate the t pixel are ncoder  making use . Four different are used; each SM    symbols  to organize one with its own In our implementation it IAC-14,B1,4,10,x25121  page  5 of 6 and it is very fast. CFM is implemented by using a built-in RAM on FPGA. The total frequency of each SM is managed making use of a counter. The encoding process consist of following phases: • identification of the symbols to encode starting from the current residual value  • extraction of statistical data for each identified symbol  • processing of statistical by using a complex arithmetic circuit • update the Statistical Models • rescaling the  values of cumulative and total frequencies. The rescaling is necessary to avoid overflows of SMs elements. Figure 11: Basic block diagram of Range Encoder The biggest challenge was to make the encode stage very fast, in particular there is an arithmetic divisor that works with a clock period of 9.5 ns, but since the value of divisor depends on the result of the division of the previous calculation, it is not possible to take full advantage of the divisor pipeline, whereby it needs to wait four clock cycles to complete calculation cycle.  Output bytes from Encode Symbol block are bufferized using a FIFO of 16Kbytes for each SM. When the FIFO goes full, it is emptied and all data  are packetized and transmitted via channel link (see figure 8). Figure 12 illustrates the structure of each packet. The header consists of a Statistical Model Identifier Code and a Packet Data Size. The data rate reached from the compressor system described in this paper is 20Mpix/s with a pixel depth of 16bits. The data rate is the same both in lossless and in lossy mode.  Figure 12. Composition of single data packet  5. TEST ENVIRONMENT Figure 13 shows the test environment (TE) used by TSD to validate the implementation of the compressor. Figure 13: Test Environment  Cubes of images (COI) in BIL format are transmitted from the workstation (WS) to the Simulator via Ethernet Link. The simulator transmits sequentially the received image data to the HPHC via Channel Link. The HPHC receives the images, performs the real time compression and re-transmits the data through the Channel Link to the Simulator. Finally the Simulator transfers the data via Ethernet to the WS where it is stored and compared against the expected data. Several parameters necessary to process the COI as the number of rows, columns and bands, as well as the parameters to setup the compressor are transmitted via CAN Bus from WS to the Simulator and HPHC. The validation has been performed by means of a comparison between the data compressed transmitted by HPHC and the output data generated by the reference SW model written in C language. Thirteen IAC-14,B1,4,10,x25121  page  6 of 6 different COI have been used to test and to validate the implementation of the compression algorithm. 5.1. IMAGER AND MASS MEMORY SIMULATOR The Imager and Mass Memory Simulator is an equipment designed and manufactured by TSD to implement an Hyperspectral data generator and a real time storage of the compressed data, so to provide a real time closed loop test environment. Figure 14: Basic block diagram of Simulator  The Simulator  is based on a processor IBM PPC750CL PowerPC running at 600 MHz and two Virtex-4 FPGAs for image data handling at high data rate and transmission/reception over Channel Link. Two FPGAs are provided with 6 SDRAMs (64 Mbytes each) for temporary buffering of the video data streams (both uncompressed and compressed). The sustained data transfer over Ethernet between  the Simulator and the WS is 40 Mbytes/s in both directions. 6. CONCLUSIONS The proposed HW implementation for onboard compression of multispectral and hyperspectral images achieves good performances in terms of maximum throughput, good flexibility being able to process any COI with a maximum size of 4096 columns and 4096 bands. The number of rows is unlimited. The implemented Soft IP Core requires a limited amount of HW resources of a FPGA Virtex-5 XC5VFX130T allowing a possible future enhancement to integrate additional functionalities. Currently, it is under development an enhanced version of the compressor with a Rate Control block implementing an automatic control of the bit rate.  7. REFERENCES [1] Jyotika Doshi and Savita Gandhi, “Implementing a Novel Data Structure for Maintaining Cumulative Frequency of Symbols”, International Journal of Computer Applications (0975 – 8887) - Vol. 41 - No.15, March 2012 [2] CCSDS, "Lossless Multispectral & Hyperspectral Image Compression", CCSDS 123.0-B-1, May 2012 [3] D. Titomanlio et al., "MASER 12 Digital Video System", 20th ESA Symposium on European Rockets and Balloon Related Research. ESA SP-700 Proceedings (2011), Noordwijk, The Netherlands  [4] G. Capuano et al. "High Data Rate Image Compression HW Platforms", 64th International Astronautical Congress 2013 [5] A. Abrardo et al, ““Low-complexity approaches for lossless and near-lossless hyperspectral image compression,” Satellite Data Compression, B. Huang (Editor), Springer, Sept. 2011. [6] I. Blanes, J. Serra-Sagristà, “Pairwise orthogonal transform for spectral image coding,” IEEE Transactions on Geoscience and Remote sensing, v. 49, n. 3, pp. 961- 972, March 2011. [7] A. Abrardo et al., “Low-complexity predictive lossy compression of hyperspectral and ultraspectral images,” Proc. of IEEE ICASSP, 2011. [8] CCSDS 123.0-B-1, "Lossless Multispectral & Hyperspectral Image Compression", May 2012 

Lossy Multi/Hyperspectral Compression HW Implementation at high data rate

De Nino, M.

Romano, M.

Capuano, G.

Magli, Enrico

PORTO Publications Open Repository TOrino

Politecnico di TorinoPorto Institutional Repository[Proceeding] Lossy Multi/Hyperspectral Compression HW Implementation athigh data rateOriginal Citation:M. De Nino; M. Romano; G. Capuano; E. Magli (2014). Lossy Multi/Hyperspectral CompressionHW Implementation at high data rate. In: International Astronautical Congress.Availability:This version is available at : http://porto.polito.it/2592673/ since: March 2015Publisher:International Astronautical Federation (IAF)Terms of use:This article is made available under terms and conditions applicable to Open Access Policy Article("Public - All rights reserved") , as described at http://porto.polito.it/terms_and_conditions.htmlPorto, the institutional repository of the Politecnico di Torino, is provided by the University Libraryand the IT-Services. The aim is to enable open access to all the world. Please share with us howthis access benefits you. Your story matters.(Article begins on next page)IAC-14,B1,4,10,x25121  LOSSY MULTI/HYPERSPECTRAL COMPRESSION HW IMPLEMENTATION AT HIGH DATA RATE  Maurizio De Nino TSD-Space, Italy, mdenino@tsd-space.it  Giuseppe Capuano TSD-Space, Italy, gcapuano@tsd-space.it Mario Romano TSD-Space, Italy, mromano@tsd-space.it Enrico Magli Politecnico di Torino, Italy, enrico.magli@polito.it IAC-14,B1,4,10,x25121  page  1 of 6  ABSTRACT Image compression is becoming more and more important, as new multispectral and hyperspectral instruments are going to generate very high data rates due to the increased spatial and spectral resolutions. Transmitting all the acquired data to the ground segment is a serious bottleneck, and compression techniques are a feasible solution to this problem. The CCSDS has established a working group (WG) on Multispectral and Hyperspectral Data Compression (MHDC), which has the purpose of standardizing compression techniques to be used onboard. The WG has already standardized a lossless compression algorithm for multispectral and hyperspectral images, and has started working on a lossy compression algorithm. The complexity of lossless compression algorithms is typically larger than that of lossy ones, leading to potentially lower throughputs. Therefore, a careful assessment is required in order to identify techniques that are able to sustain very high data rates. The increased complexity can also lead to increased resource occupancy on a hardware device such as an FPGA. Lossy compression introduces information losses in the images, and these losses must be accurately characterized, and their effect on the applications investigated. For these reasons, developing a lossy algorithm requires a more elaborate process. Under an ESA contract primed by Politecnico of Torino, TSD is currently designing an IP core for FPGA and/or ASIC implementation of a lossy compression algorithm that is being proposed for CCSDS standardization. In addition to the IP core, TSD is developing a HW platform based on the Xilinx Virtex-5 XQR5VFX130, the industry's first high performance rad-hard reconfigurable FPGA for processing-intensive for space systems. Advanced results along with details of electronic platform design will be presented in this paper.  1. INTRODUCTION Compression of hyperspectral images has represented an important and active research topic for a long time. All spectral imagers such as those of the multispectral, hyperspectral and ultraspectral type generate extremely large amounts of data, making image compression mandatory in order to reduce the data volume before the transmission to the ground segment.  More recently, a new class of prediction-based approaches have been proposed for lossless [5] and lossy compression. The CCSDS Working Group (WG) has worked towards the definition of a new image compression standard for lossless compression of multispectral and hyperspectral images. This new lossless compression standard was issued as CCSDS 123 recommendation. Subsequently, the WG started a discussion about the lossy compression standard for multispectral and hyperspectral images. Two techniques have been proposed as candidates for this standard: one technique is based on the pair wise orthogonal transform [6]; the second candidate is an improved version of [7]. The second algorithm will be very competitive in terms of performance and complexity. In particular, while transform coding is a good approach for compression at the ground station, it is not a good fit to onboard compression, as it would pose large memory and computational requirements, leading to low throughput. On the other hand, the predictive lossy compression paradigm, on which [7] is based, is more suitable for onboard space applications.  2. COMPRESSION ALGORITHM In this paper, is presented an hardware implementation of a compression algorithm for multispectral and hyperspectral images based on the IAC-14,B1,4,10,x25121 predictive lossy compression paradigm. predictive lossy compression is able to achieve compression performance as good as the transformbased approach, at a fraction of its complexityproviding a very good fit to onboard image compression. Predictive lossy compression can be seen as an evolution of near-lossless compression.Near-lossless compression techniques are easily derived from lossless ones, by simplyuniform quantizer with feedback loop to the prediction residuals. Since the quantization error introduced on the prediction residual of one sample is identical to the reconstruction error that will be observed on that sample, these techniques allow to control errors in the compressed images in a very accurate and flexible way. The implemented algorithm can be considered as an extension of CCSDS 123 [8] with a quantization stage to achieve lossy compression entropy coder (range encoder) (See Figure 1)Prediction ResidualQuantization StageInputImage Figure 1: Logic block diagram of compressor3. FLIGHT HW PLATFORMThe test environment is based on a flight hardware to validate not only the algorithm but also its onboard implementation. The flight hardware is the (High performance Processing unit for Hyperspecral data Compression), belonging to a family of High performance Processing unit (HPxx) TSD for real time image data processing7).  The platform includes the following modules:• Image Processing Module (IPM) • Power Conditioning & Distribution Module (PCDM)  The - and   applying a and a modified . RangeEncoderCompressed Data  HPHC developed by  (See Figure    Figure 2: Basic block diagram of HPThe IPM is composed of four sections as shown figure 3:   • Image Processing A• Image Processing B• Data Handling A  • Data Handling B  Figure 3: Detailed scheme of IPMThe two Image Processing sections and the two Data Handling sections are identical and they can be configured: • in cold redundancy to provide high reliability• in master-slave mode to run in parallel so to improve the processing capabilitiesThe Data Handling section implements the control & communication functionalities of the unit, while the Image Processing section is dedicated to the page  2 of 6 HC in        IAC-14,B1,4,10,x25121 compression. Figure 4: IPM Mother board. Note on both sides the anti-fuse FPGAs As shown in the following Figure 3the Image Processor sections are redounded and connected in cross-strapped mode to both the Data Handling sections. Point-to point links are also available to  interconnect the Image Processing A section with the Image Processing B and the Data Handling A with the Data Handling B, thus allowing the sections to run in parallel.  Figure 5: IPM Daughter board. On both sides can be seen the two Virtex5 FPGAs  Each section of image processing adopts the Xilinx Virtex-5 XC5VFX130T (XQR5VFX130 for flight Model), the industry's first high performance radreconfigurable FPGA. Each FPGA is provided with 5Gbit SDRAMimage data inputs (1.575 Gbits/s each).  , the outputs of  -hard  and two   Figure 6: PCDM Figure 7: HPHC engineering model assembled in its cabinet 4. MHDC SOFT IPThe core of Multispectral and Hyperspectral Data Compressor is implemented only on Processor Image Unit A since a single Virtex5 chip has hardware resources that are more than sufficient for our purposes.  Figure 8 describes the basic block diagram of compressor. Figure 8: Basic block diagram of compressorpage  3 of 6   enough  IAC-14,B1,4,10,x25121 Once a Cube Of Images (COI) is receivedprocessed by the Grabber that transmits a pair of pixels at a time to the Prediction Unipixels belong to two adjacent rows column. For this reason, considering size of 4096 columns x 4096 bandsmaximum amount of memory necessary is 32Mbytes since it is necessary to store the previous horizontal plane of the COI. Figure 9 shows a generic COI; the current horizontal plane is received pixel by pixel and at same time the grabber stores these pixels on SDRAM and provides the pixels of previous horizontal plane by reading them from SDRAM. At the end of plane the cycle continues to the next plane until the end of COI. When the first plane is transmittedplane doesn't exists yet, therefore the pixel previous plane of the pair to transmit has its value forced to 0x0000.  Figure 9: Anatomy of a COI according to The PU calculates, for each pair of pixels in input, the predicted value of pixel belonging horizontal plane. The predicted value is subtractedthe real value and this residual value, after the quantization, thanks to its lower information content, can be compressed with higher efficiency. Figure 10 shows how the predicted value is calculated [2]. We tell Sz,y,x the current pixel, the related predicted value is calculated using the values of pixels of the five previous bands having the same x,y coordinates (Sz-1,y,x , Sz-2,y,x...... pixel of previous row Sz,y-1,x.  , it is t (PU). The two at the same the maximum  @16bits, the for grabbing , the previous of the BIL format to the current  to  Sz-5,y,x) and the The PU makes use of a SDRAM to store the weights vectors that are updated continuouslynew COI has to be processedweights vectors are written into SDRAM by means of the Prediction Unit Initializator.The stage after the prediction is the quantization. When the quantization is enabled, the compression mode is lossy. The compressor can operate in lossless mode by disabling the Quantizer.Figure 10: Pixels taken in accpredicted value of the currenAll the residuals (with or without quantization) encoded by the Range Encoder, an entropy ethat remove the numerical redundanciesof statistics. Figure 11 illustrates the simplified block diagram of the range encoderindependent statistic models (SM) stores and updates the following statistical data:• The cumulative frequency for each symbol• The frequency for each symbol• Total frequency defined as the sum of overall frequency of Different data structures can be usedcumulative frequencies, each advantages and drawbacks. was adopted the Cumulative Frequency Matrix (CFM) [1] which requires a limited HW resources page  4 of 6 . Every time a ,  the starting values of  - ount to calculate the t pixel are ncoder  making use . Four different are used; each SM    symbols  to organize one with its own In our implementation it IAC-14,B1,4,10,x25121  page  5 of 6 and it is very fast. CFM is implemented by using a built-in RAM on FPGA. The total frequency of each SM is managed making use of a counter. The encoding process consist of following phases: • identification of the symbols to encode starting from the current residual value  • extraction of statistical data for each identified symbol  • processing of statistical by using a complex arithmetic circuit • update the Statistical Models • rescaling the  values of cumulative and total frequencies. The rescaling is necessary to avoid overflows of SMs elements. Figure 11: Basic block diagram of Range Encoder The biggest challenge was to make the encode stage very fast, in particular there is an arithmetic divisor that works with a clock period of 9.5 ns, but since the value of divisor depends on the result of the division of the previous calculation, it is not possible to take full advantage of the divisor pipeline, whereby it needs to wait four clock cycles to complete calculation cycle.  Output bytes from Encode Symbol block are bufferized using a FIFO of 16Kbytes for each SM. When the FIFO goes full, it is emptied and all data  are packetized and transmitted via channel link (see figure 8). Figure 12 illustrates the structure of each packet. The header consists of a Statistical Model Identifier Code and a Packet Data Size. The data rate reached from the compressor system described in this paper is 20Mpix/s with a pixel depth of 16bits. The data rate is the same both in lossless and in lossy mode.  Figure 12. Composition of single data packet  5. TEST ENVIRONMENT Figure 13 shows the test environment (TE) used by TSD to validate the implementation of the compressor. Figure 13: Test Environment  Cubes of images (COI) in BIL format are transmitted from the workstation (WS) to the Simulator via Ethernet Link. The simulator transmits sequentially the received image data to the HPHC via Channel Link. The HPHC receives the images, performs the real time compression and re-transmits the data through the Channel Link to the Simulator. Finally the Simulator transfers the data via Ethernet to the WS where it is stored and compared against the expected data. Several parameters necessary to process the COI as the number of rows, columns and bands, as well as the parameters to setup the compressor are transmitted via CAN Bus from WS to the Simulator and HPHC. The validation has been performed by means of a comparison between the data compressed transmitted by HPHC and the output data generated by the reference SW model written in C language. Thirteen IAC-14,B1,4,10,x25121  page  6 of 6 different COI have been used to test and to validate the implementation of the compression algorithm. 5.1. IMAGER AND MASS MEMORY SIMULATOR The Imager and Mass Memory Simulator is an equipment designed and manufactured by TSD to implement an Hyperspectral data generator and a real time storage of the compressed data, so to provide a real time closed loop test environment. Figure 14: Basic block diagram of Simulator  The Simulator  is based on a processor IBM PPC750CL PowerPC running at 600 MHz and two Virtex-4 FPGAs for image data handling at high data rate and transmission/reception over Channel Link. Two FPGAs are provided with 6 SDRAMs (64 Mbytes each) for temporary buffering of the video data streams (both uncompressed and compressed). The sustained data transfer over Ethernet between  the Simulator and the WS is 40 Mbytes/s in both directions. 6. CONCLUSIONS The proposed HW implementation for onboard compression of multispectral and hyperspectral images achieves good performances in terms of maximum throughput, good flexibility being able to process any COI with a maximum size of 4096 columns and 4096 bands. The number of rows is unlimited. The implemented Soft IP Core requires a limited amount of HW resources of a FPGA Virtex-5 XC5VFX130T allowing a possible future enhancement to integrate additional functionalities. Currently, it is under development an enhanced version of the compressor with a Rate Control block implementing an automatic control of the bit rate.  7. REFERENCES [1] Jyotika Doshi and Savita Gandhi, “Implementing a Novel Data Structure for Maintaining Cumulative Frequency of Symbols”, International Journal of Computer Applications (0975 – 8887) - Vol. 41 - No.15, March 2012 [2] CCSDS, "Lossless Multispectral & Hyperspectral Image Compression", CCSDS 123.0-B-1, May 2012 [3] D. Titomanlio et al., "MASER 12 Digital Video System", 20th ESA Symposium on European Rockets and Balloon Related Research. ESA SP-700 Proceedings (2011), Noordwijk, The Netherlands  [4] G. Capuano et al. "High Data Rate Image Compression HW Platforms", 64th International Astronautical Congress 2013 [5] A. Abrardo et al, ““Low-complexity approaches for lossless and near-lossless hyperspectral image compression,” Satellite Data Compression, B. Huang (Editor), Springer, Sept. 2011. [6] I. Blanes, J. Serra-Sagristà, “Pairwise orthogonal transform for spectral image coding,” IEEE Transactions on Geoscience and Remote sensing, v. 49, n. 3, pp. 961- 972, March 2011. [7] A. Abrardo et al., “Low-complexity predictive lossy compression of hyperspectral and ultraspectral images,” Proc. of IEEE ICASSP, 2011. [8] CCSDS 123.0-B-1, "Lossless Multispectral & Hyperspectral Image Compression", May 2012 

https://iris.polito.it/retrieve/handle/11583/2592673/64609/IAC-14%2cB1%2c4%2cx25121.pdf

Lossy Multi/Hyperspectral Compression HW Implementation at high data rate

Abstract

Similar works

Full text

Available Versions

PORTO@iris (Publications Open Repository TOrino - Politecnico di Torino)

PORTO Publications Open Repository TOrino