Implementing the New CCSDS Housekeeping Data Compression Standard 124.0-B-1 (Based on POCKET+) on OPS-SAT-1

The number of telemetry parameters available in a typical spacecraft is constantly increasing. At the same time, the bandwidth available to download all that information is rather static. Operators must therefore make hard choices between which parameters to downlink or not, in which different situations, and at which sampling rates. This tradeoff is more problematic for missions with higher communication latency beyond LEO. Since 2009, The European Space Agency’s European Space Operations Center (ESA/ESOC) has been promoting the compression of housekeeping telemetry as a solution to this problem. Most spacecraft housekeeping telemetry parameters compress extremely well if they are pre-processed correctly. Unfortunately, most spacecraft record telemetry packets in flat packet stores so accessing different packets within them is too CPU and memory intensive for flight computers. Using traditional compression schemes such as zip or tar are not compatible with the traditional “fire and forget” mode of operation i.e., occasional packet losses are expected. This would render entire compressed files unusable. ESOC invented an algorithm called POCKET+ to solve this problem. It is implemented using very low-level processor instructions such as OR, XOR, AND, etc. This means that it can run with low CPU usage and, more importantly, with a short execution time. It is designed to run fast enough to compress a stream of incoming packets as they are generated by the on-board packetiser. The output is a smaller stream of packets. The compressed packets can be handled by the on-board system in an identical fashion to the original larger uncompressed packets. Robustness with respect to the occasional packet loss is built into the protocol and does not require a back channel. In 2018, POCKET+ was proposed to the CCSDS data compression working group and after extensive research by other agencies the core idea has been Evans 2 36th Annual Small Satellite Conference incorporated into a proposed new standard for “Robust Compression of Fixed Length Housekeeping Data.” The second supporter for the mission is CNES, supported technically by the University of Barcelona (UAB). Both CNES and UAB have suggested changes that make POCKET+ even more powerful. POCKET+ is already flying on OPSSAT, a 3U CubeSat launched by the European Space Agency on December 18th, 2019. The mission has updated the Onboard Software (OBSW) and ground control software to be compliant with the latest POCKET+ standard. The standard is set to be available for an ESA review. This paper describes the latest algorithm and how it is implemented on OPS-SAT, including how the same core software has been successfully deployed in two completely different scenarios/environments. One compresses files offline and then uses a transport protocol with a completeness guarantee; the other compresses a packet stream in real-time and uses the classic transport protocol where completeness is not guaranteed. The results show that compression ratios between eight and ten are usual for the OPSSAT mission. Improvements made during the development of the planned CCSDS standard for “Robust Compression of Fixed Length Housekeeping Data” are also presented

Parallel data compression

Data compression schemes remove data redundancy in communicated and stored data and increase the effective capacities of communication and storage devices. Parallel algorithms and implementations for textual data compression are surveyed. Related concepts from parallel computation and information theory are briefly discussed. Static and dynamic methods for codeword construction and transmission on various models of parallel computation are described. Included are parallel methods which boost system speed by coding data concurrently, and approaches which employ multiple compression techniques to improve compression ratios. Theoretical and empirical comparisons are reported and areas for future research are suggested

Real-time and distributed applications for dictionary-based data compression

The greedy approach to dictionary-based static text compression can be executed by a finite state machine. When it is applied in parallel to different blocks of data independently, there is no lack of robustness even on standard large scale distributed systems with input files of arbitrary size. Beyond standard large scale, a negative effect on the compression effectiveness is caused by the very small size of the data blocks. A robust approach for extreme distributed systems is presented in this paper, where this problem is fixed by overlapping adjacent blocks and preprocessing the neighborhoods of the boundaries. Moreover, we introduce the notion of pseudo-prefix dictionary, which allows optimal compression by means of a real-time semi-greedy procedure and a slight improvement on the compression ratio obtained by the distributed implementations

Extended Bit-Plane Compression for Convolutional Neural Network Accelerators

After the tremendous success of convolutional neural networks in image classification, object detection, speech recognition, etc., there is now rising demand for deployment of these compute-intensive ML models on tightly power constrained embedded and mobile systems at low cost as well as for pushing the throughput in data centers. This has triggered a wave of research towards specialized hardware accelerators. Their performance is often constrained by I/O bandwidth and the energy consumption is dominated by I/O transfers to off-chip memory. We introduce and evaluate a novel, hardware-friendly compression scheme for the feature maps present within convolutional neural networks. We show that an average compression ratio of 4.4x relative to uncompressed data and a gain of 60% over existing method can be achieved for ResNet-34 with a compression block requiring <300 bit of sequential cells and minimal combinational logic

Estimating the Algorithmic Complexity of Stock Markets

Randomness and regularities in Finance are usually treated in probabilistic terms. In this paper, we develop a completely different approach in using a non-probabilistic framework based on the algorithmic information theory initially developed by Kolmogorov (1965). We present some elements of this theory and show why it is particularly relevant to Finance, and potentially to other sub-fields of Economics as well. We develop a generic method to estimate the Kolmogorov complexity of numeric series. This approach is based on an iterative "regularity erasing procedure" implemented to use lossless compression algorithms on financial data. Examples are provided with both simulated and real-world financial time series. The contributions of this article are twofold. The first one is methodological : we show that some structural regularities, invisible with classical statistical tests, can be detected by this algorithmic method. The second one consists in illustrations on the daily Dow-Jones Index suggesting that beyond several well-known regularities, hidden structure may in this index remain to be identified