Data Compression in the Petascale Astronomy Era: a GERLUMPH case study

As the volume of data grows, astronomers are increasingly faced with choices on what data to keep -- and what to throw away. Recent work evaluating the JPEG2000 (ISO/IEC 15444) standards as a future data format standard in astronomy has shown promising results on observational data. However, there is still a need to evaluate its potential on other type of astronomical data, such as from numerical simulations. GERLUMPH (the GPU-Enabled High Resolution cosmological MicroLensing parameter survey) represents an example of a data intensive project in theoretical astrophysics. In the next phase of processing, the ~27 terabyte GERLUMPH dataset is set to grow by a factor of 100 -- well beyond the current storage capabilities of the supercomputing facility on which it resides. In order to minimise bandwidth usage, file transfer time, and storage space, this work evaluates several data compression techniques. Specifically, we investigate off-the-shelf and custom lossless compression algorithms as well as the lossy JPEG2000 compression format. Results of lossless compression algorithms on GERLUMPH data products show small compression ratios (1.35:1 to 4.69:1 of input file size) varying with the nature of the input data. Our results suggest that JPEG2000 could be suitable for other numerical datasets stored as gridded data or volumetric data. When approaching lossy data compression, one should keep in mind the intended purposes of the data to be compressed, and evaluate the effect of the loss on future analysis. In our case study, lossy compression and a high compression ratio do not significantly compromise the intended use of the data for constraining quasar source profiles from cosmological microlensing.Comment: 15 pages, 9 figures, 5 tables. Published in the Special Issue of Astronomy & Computing on The future of astronomical data format

NASA Tech Briefs, December 2009

Topics include: A Deep Space Network Portable Radio Science Receiver; Detecting Phase Boundaries in Hard-Sphere Suspensions; Low-Complexity Lossless and Near-Lossless Data Compression Technique for Multispectral Imagery; Very-Long-Distance Remote Hearing and Vibrometry; Using GPS to Detect Imminent Tsunamis; Stream Flow Prediction by Remote Sensing and Genetic Programming; Pilotless Frame Synchronization Using LDPC Code Constraints; Radiometer on a Chip; Measuring Luminescence Lifetime With Help of a DSP; Modulation Based on Probability Density Functions; Ku Telemetry Modulator for Suborbital Vehicles; Photonic Links for High-Performance Arraying of Antennas; Reconfigurable, Bi-Directional Flexfet Level Shifter for Low-Power, Rad-Hard Integration; Hardware-Efficient Monitoring of I/O Signals; Video System for Viewing From a Remote or Windowless Cockpit; Spacesuit Data Display and Management System; IEEE 1394 Hub With Fault Containment; Compact, Miniature MMIC Receiver Modules for an MMIC Array Spectrograph; Waveguide Transition for Submillimeter-Wave MMICs; Magnetic-Field-Tunable Superconducting Rectifier; Bonded Invar Clip Removal Using Foil Heaters; Fabricating Radial Groove Gratings Using Projection Photolithography; Gratings Fabricated on Flat Surfaces and Reproduced on Non-Flat Substrates; Method for Measuring the Volume-Scattering Function of Water; Method of Heating a Foam-Based Catalyst Bed; Small Deflection Energy Analyzer for Energy and Angular Distributions; Polymeric Bladder for Storing Liquid Oxygen; Pyrotechnic Simulator/Stray-Voltage Detector; Inventions Utilizing Microfluidics and Colloidal Particles; RuO2 Thermometer for Ultra-Low Temperatures; Ultra-Compact, High-Resolution LADAR System for 3D Imaging; Dual-Channel Multi-Purpose Telescope; Objective Lens Optimized for Wavefront Delivery, Pupil Imaging, and Pupil Ghosting; CMOS Camera Array With Onboard Memory; Quickly Approximating the Distance Between Two Objects; Processing Images of Craters for Spacecraft Navigation; Adaptive Morphological Feature-Based Object Classifier for a Color Imaging System; Rover Slip Validation and Prediction Algorithm; Safety and Quality Training Simulator; Supply-Chain Optimization Template; Algorithm for Computing Particle/Surface Interactions; Cryogenic Pupil Alignment Test Architecture for Aberrated Pupil Images; and Thermal Transport Model for Heat Sink Design

Hierarchical Autoencoder-based Lossy Compression for Large-scale High-resolution Scientific Data

Lossy compression has become an important technique to reduce data size in many domains. This type of compression is especially valuable for large-scale scientific data, whose size ranges up to several petabytes. Although Autoencoder-based models have been successfully leveraged to compress images and videos, such neural networks have not widely gained attention in the scientific data domain. Our work presents a neural network that not only significantly compresses large-scale scientific data but also maintains high reconstruction quality. The proposed model is tested with scientific benchmark data available publicly and applied to a large-scale high-resolution climate modeling data set. Our model achieves a compression ratio of 140 on several benchmark data sets without compromising the reconstruction quality. Simulation data from the High-Resolution Community Earth System Model (CESM) Version 1.3 over 500 years are also being compressed with a compression ratio of 200 while the reconstruction error is negligible for scientific analysis.Comment: 15 pages, 15 figure

NASA Space Engineering Research Center for VLSI System Design

This annual report outlines the activities of the past year at the NASA SERC on VLSI Design. Highlights for this year include the following: a significant breakthrough was achieved in utilizing commercial IC foundries for producing flight electronics; the first two flight qualified chips were designed, fabricated, and tested and are now being delivered into NASA flight systems; and a new technology transfer mechanism has been established to transfer VLSI advances into NASA and commercial systems

Estudi bibliomètric any 2016. EETAC

El present document recull les publicacions indexades a la base de dades Scopus durant el període comprès entre el mesos de gener a desembre de l’any 2016, escrits per autors pertanyents a l’EETAC. Es presenten les dades recollides segons la font on s’ha publicat, els autors que han publicat, i el tipus de document publicat. S’hi inclou un annex amb la llista de totes les referències bibliogràfiques publicades.Postprint (author's final draft

Scientific and Technical Publishing at Goddard Space Flight Center in Fiscal Year 1994

This publication is a compilation of scientific and technical material that was researched, written, prepared, and disseminated by the Center's scientists and engineers during FY94. It is presented in numerical order of the GSFC author's sponsoring technical directorate; i.e., Code 300 is the Office of Flight Assurance, Code 400 is the Flight Projects Directorate, Code 500 is the Mission Operations and Data Systems Directorate, Code 600 is the Space Sciences Directorate, Code 700 is the Engineering Directorate, Code 800 is the Suborbital Projects and Operations Directorate, and Code 900 is the Earth Sciences Directorate. The publication database contains publication or presentation title, author(s), document type, sponsor, and organizational code. This is the second annual compilation for the Center

Data Compression for Helioseismology

Die effiziente Kompression von Daten wird eine wichtige Rolle für mehrere bevorste- hende und geplante Weltraummissionen spielen, die Helioseismologie betreiben werden, wie beispielsweise Solar Orbiter. Solar Orbiter ist die nächste Mission, die Helioseismologie beinhaltet, und soll im Oktober 2018 gestartet werden. Das Hauptmerkmal von Solar Orbiter ist der Orbit. Die Umlaufbahn des Satelliten wird zur Ekliptik geneigt sein, sodass der Satellit einen solaren Breitengrad von bis zu 33 Grad erreichen wird. Dies wird erstmals ermöglichen, die Pole der Sonne mit Hilfe von lokaler Helioseismologie zu studieren. Zusätzlich dazu können kombinierte Beobachtungen von Solar Orbiter und einem anderen Instrument dazu benutzt werden, die tiefen Schichten der Sonne mittels stereoskopischer Helioseismologie zu erforschen. Die Aufnahmen der Dopplergeschwindigkeit und der Kontinuumsintensität, die für Helioseismologie benötigt werden, werden vom Polarimetric and Helioseismic Imager (PHI) geliefert werden. Große Hindernisse für Helioseismologie mit Solar Orbiter sind die niedrige Datenüber- tragungsrate und die (wahrscheinlich) kurzen Beobachtungszeiten. Außerdem erfordert die Untersuchung der Pole der Sonne Beobachtungen in der Nähe des Sonnenrandes, sogar von dem geneigten Orbit von Solar Orbiter aus. Dies kann zu systematischen Fehlern führen. In dieser Doktorarbeit gebe ich eine erste Einschätzung ab, wie stark Helioseismologie von verlustbehafteter Datenkompression beeinflusst wird. Mein Schwerpunkt liegt dabei auf der Solar Orbiter Mission, die von mir erzielten Ergebnisse sind aber auch auf andere geplante Missionen übertragbar. Zunächst habe ich mit Hilfe synthetischer Daten die Eignung des PHI Instruments für Helioseismologie getestet. Diese basieren auf Simulationen der Konvektion nahe der Sonnenoberfläche und einem Modell von PHI. Ich habe eine sechs Stunden lange Zeitreihe synthetischer Daten erstellt, die die gleichen Eigenschaften wie die von PHI erwarteten Daten haben. Hierbei habe ich mich auf den Einfluss der Punktspreizfunktion, der Vibrationen des Satelliten und des Photonenrauschen konzentriert. Die von diesen Daten abgeleitete spektrale Leistungsdichte der solaren Oszillationen legt nahe, dass PHI für Helioseismologie geeignet sein wird. Aufgrund der niedrigen Datenübertragungsrate von Solar Orbiter müssen die von PHI für die Helioseismologie gewonnenen Daten stark komprimiert werden. Ich habe den Einfluss von Kompression mit Hilfe von Daten getestet, die vom Helioseismic and Magnetic Imager (HMI) stammen. HMI ist ein Instrument an Bord des Solar Dynam- ics Observatory Satelliten (SDO), der 2010 gestartet worden ist. HMI erstellt mit hoher zeitlicher Abfolge Karten der Kontinuumsintensität, der Dopplergeschwindigkeit und des kompletten Magnetfeldvektors für die komplette von der Erde aus sichtbare Hemispäre der Sonne. Mit Hilfe mit von HMI aufgenommenen Karten der Dopplergeschwindigkeit konnte ich zeigen, dass das Signal-zu-Rausch Verhältnis von Supergranulation in der Zeit-Entfernungs Helioseismologie nicht stark von Datenkompression beeinflusst wird. Außerdem habe ich nachgewiesen, dass die Genauigkeit und Präzision von Messungen der Sonnenrotation mittels Local Correlation Tracking von Granulation durch verlust- behaftete Datenkompression nicht wesentlich verschlechtert werden. Diese Ergebnisse deuten an, dass die niedrige Datenübertragungsrate von Solar Orbiter nicht unbedingt ein großes Hinderniss für Helioseismologie sein muss

NASA Tech Briefs, January 2014

Topics include: Multi-Source Autonomous Response for Targeting and Monitoring of Volcanic Activity; Software Suite to Support In-Flight Characterization of Remote Sensing Systems; Visual Image Sensor Organ Replacement; Ultra-Wideband, Dual-Polarized, Beam-Steering P-Band Array Antenna; Centering a DDR Strobe in the Middle of a Data Packet; Using a Commercial Ethernet PHY Device in a Radiation Environment; Submerged AUV Charging Station; Habitat Demonstration Unit (HDU) Vertical Cylinder Habitat; Origami-Inspired Folding of Thick, Rigid Panels; A Novel Protocol for Decoating and Permeabilizing Bacterial Spores for Epifluorescent Microscopy; Method and Apparatus for Automated Isolation of Nucleic Acids from Small Cell Samples; Enabling Microliquid Chromatography by Microbead Packing of Microchannels; On-Command Force and Torque Impeding Devices (OC-FTID) Using ERF; Deployable Fresnel Rings; Transition-Edge Hot-Electron Microbolometers for Millimeter and Submillimeter Astrophysics; Spacecraft Trajectory Analysis and Mission Planning Simulation (STAMPS) Software; Cross Support Transfer Service (CSTS) Framework Library; Arbitrary Shape Deformation in CFD Design; Range Safety Flight Elevation Limit Calculation Software; Frequency-Modulated, Continuous-Wave Laser Ranging Using Photon-Counting Detectors; Calculation of Operations Efficiency Factors for Mars Surface Missions; GPU Lossless Hyperspectral Data Compression System; Robust, Optimal Subsonic Airfoil Shapes; Protograph-Based Raptor-Like Codes; Fuzzy Neuron: Method and Hardware Realization; Kalman Filter Input Processor for Boresight Calibration; Organizing Compression of Hyperspectral Imagery to Allow Efficient Parallel Decompression; and Temperature Dependences of Mechanisms Responsible for the Water-Vapor Continuum Absorption