Compression of spectral meteorological imagery

Data compression is essential to current low-earth-orbit spectral sensors with global coverage, e.g., meteorological sensors. Such sensors routinely produce in excess of 30 Gb of data per orbit (over 4 Mb/s for about 110 min) while typically limited to less than 10 Gb of downlink capacity per orbit (15 minutes at 10 Mb/s). Astro-Space Division develops spaceborne compression systems for compression ratios from as little as three to as much as twenty-to-one for high-fidelity reconstructions. Current hardware production and development at Astro-Space Division focuses on discrete cosine transform (DCT) systems implemented with the GE PFFT chip, a 32x32 2D-DCT engine. Spectral relations in the data are exploited through block mean extraction followed by orthonormal transformation. The transformation produces blocks with spatial correlation that are suitable for further compression with any block-oriented spatial compression system, e.g., Astro-Space Division's Laplacian modeler and analytic encoder of DCT coefficients

The 1995 Science Information Management and Data Compression Workshop

This document is the proceedings from the 'Science Information Management and Data Compression Workshop,' which was held on October 26-27, 1995, at the NASA Goddard Space Flight Center, Greenbelt, Maryland. The Workshop explored promising computational approaches for handling the collection, ingestion, archival, and retrieval of large quantities of data in future Earth and space science missions. It consisted of fourteen presentations covering a range of information management and data compression approaches that are being or have been integrated into actual or prototypical Earth or space science data information systems, or that hold promise for such an application. The Workshop was organized by James C. Tilton and Robert F. Cromp of the NASA Goddard Space Flight Center

Green compressive sampling reconstruction in IoT networks

In this paper, we address the problem of green Compressed Sensing (CS) reconstruction within Internet of Things (IoT) networks, both in terms of computing architecture and reconstruction algorithms. The approach is novel since, unlike most of the literature dealing with energy efficient gathering of the CS measurements, we focus on the energy efficiency of the signal reconstruction stage given the CS measurements. As a first novel contribution, we present an analysis of the energy consumption within the IoT network under two computing architectures. In the first one, reconstruction takes place within the IoT network and the reconstructed data are encoded and transmitted out of the IoT network; in the second one, all the CS measurements are forwarded to off-network devices for reconstruction and storage, i.e., reconstruction is off-loaded. Our analysis shows that the two architectures significantly differ in terms of consumed energy, and it outlines a theoretically motivated criterion to select a green CS reconstruction computing architecture. Specifically, we present a suitable decision function to determine which architecture outperforms the other in terms of energy efficiency. The presented decision function depends on a few IoT network features, such as the network size, the sink connectivity, and other systems’ parameters. As a second novel contribution, we show how to overcome classical performance comparison of different CS reconstruction algorithms usually carried out w.r.t. the achieved accuracy. Specifically, we consider the consumed energy and analyze the energy vs. accuracy trade-off. The herein presented approach, jointly considering signal processing and IoT network issues, is a relevant contribution for designing green compressive sampling architectures in IoT networks

The Space and Earth Science Data Compression Workshop

This document is the proceedings from a Space and Earth Science Data Compression Workshop, which was held on March 27, 1992, at the Snowbird Conference Center in Snowbird, Utah. This workshop was held in conjunction with the 1992 Data Compression Conference (DCC '92), which was held at the same location, March 24-26, 1992. The workshop explored opportunities for data compression to enhance the collection and analysis of space and Earth science data. The workshop consisted of eleven papers presented in four sessions. These papers describe research that is integrated into, or has the potential of being integrated into, a particular space and/or Earth science data information system. Presenters were encouraged to take into account the scientists's data requirements, and the constraints imposed by the data collection, transmission, distribution, and archival system

Utilising Reduced File Representations to Facilitate Fast Contraband Detection

Digital forensics practitioners can be tasked with analysing digital data, in all its forms, for legal proceedings. In law enforcement, this largely involves searching for contraband media, such as illegal images and videos, on a wide array of electronic devices. Unfortunately, law enforcement agencies are often under-resourced and under-staffed, while the volume of digital evidence, and number of investigations, continues to rise each year, contributing to large investigative backlogs.A primary bottleneck in forensic processing can be the speed at which data is acquired from a disk or network, which can be mitigated with data reduction techniques. The data reduction approach in this thesis uses reduced representations for individual images which can be used in lieu of cryptographic hashes for the automatic detection of illegal media. These approaches can facilitate reduced forensic processing times, faster investigation turnaround, and a reduction in the investigative backlog.Reduced file representations are achieved in two ways. The first approach is to generate signatures from partial files, where highly discriminative features are analysed, while reading as little of the file as possible. Such signatures can be generated using either header features of a particular file format, or by reading logical data blocks. This works best when reading from the end of the file. These sub-file signatures are particularly effective on solid state drives and networked drives, reducing processing times by up to 70× compared to full file cryptographic hashing. Overall the thesis shows that these signatures are highly discriminative, or unique, at the million image scale, and are thus suitable for the forensic context. This approach is effectively a starting point for developing forensics techniques which leverage the performance characteristics of non-mechanical media, allowing for evidence on flash based devices to be processed more efficiently.The second approach makes use of thumbnails, particularly those stored in the Windows thumbnail cache database. A method was developed which allows for image previews for an entire computer to be parsed in less than 20 seconds using cryptographic hashes, effecting rapid triage. The use of perceptual hashing allows for variations between operating systems to be accounted for, while also allowing for small image modifications to be captured in an analysis. This approach is not computationally expensive but has the potential to flag illegal media in seconds, rather than an hour in traditional triage, making a good starting point for investigations of illegal media