Astronomical Data Analysis and Sparsity: from Wavelets to Compressed Sensing

Wavelets have been used extensively for several years now in astronomy for many purposes, ranging from data filtering and deconvolution, to star and galaxy detection or cosmic ray removal. More recent sparse representations such ridgelets or curvelets have also been proposed for the detection of anisotropic features such cosmic strings in the cosmic microwave background. We review in this paper a range of methods based on sparsity that have been proposed for astronomical data analysis. We also discuss what is the impact of Compressed Sensing, the new sampling theory, in astronomy for collecting the data, transferring them to the earth or reconstructing an image from incomplete measurements.Comment: Submitted. Full paper will figures available at http://jstarck.free.fr/IEEE09_SparseAstro.pd

Photon counting compressive depth mapping

We demonstrate a compressed sensing, photon counting lidar system based on the single-pixel camera. Our technique recovers both depth and intensity maps from a single under-sampled set of incoherent, linear projections of a scene of interest at ultra-low light levels around 0.5 picowatts. Only two-dimensional reconstructions are required to image a three-dimensional scene. We demonstrate intensity imaging and depth mapping at 256 x 256 pixel transverse resolution with acquisition times as short as 3 seconds. We also show novelty filtering, reconstructing only the difference between two instances of a scene. Finally, we acquire 32 x 32 pixel real-time video for three-dimensional object tracking at 14 frames-per-second.Comment: 16 pages, 8 figure

Challenges in Partially-Automated Roadway Feature Mapping Using Mobile Laser Scanning and Vehicle Trajectory Data

Connected vehicle and driver's assistance applications are greatly facilitated by Enhanced Digital Maps (EDMs) that represent roadway features (e.g., lane edges or centerlines, stop bars). Due to the large number of signalized intersections and miles of roadway, manual development of EDMs on a global basis is not feasible. Mobile Terrestrial Laser Scanning (MTLS) is the preferred data acquisition method to provide data for automated EDM development. Such systems provide an MTLS trajectory and a point cloud for the roadway environment. The challenge is to automatically convert these data into an EDM. This article presents a new processing and feature extraction method, experimental demonstration providing SAE-J2735 map messages for eleven example intersections, and a discussion of the results that points out remaining challenges and suggests directions for future research.Comment: 6 pages, 5 figure

The Footprint Database and Web Services of the Herschel Space Observatory

Data from the Herschel Space Observatory is freely available to the public but no uniformly processed catalogue of the observations has been published so far. To date, the Herschel Science Archive does not contain the exact sky coverage (footprint) of individual observations and supports search for measurements based on bounding circles only. Drawing on previous experience in implementing footprint databases, we built the Herschel Footprint Database and Web Services for the Herschel Space Observatory to provide efficient search capabilities for typical astronomical queries. The database was designed with the following main goals in mind: (a) provide a unified data model for meta-data of all instruments and observational modes, (b) quickly find observations covering a selected object and its neighbourhood, (c) quickly find every observation in a larger area of the sky, (d) allow for finding solar system objects crossing observation fields. As a first step, we developed a unified data model of observations of all three Herschel instruments for all pointing and instrument modes. Then, using telescope pointing information and observational meta-data, we compiled a database of footprints. As opposed to methods using pixellation of the sphere, we represent sky coverage in an exact geometric form allowing for precise area calculations. For easier handling of Herschel observation footprints with rather complex shapes, two algorithms were implemented to reduce the outline. Furthermore, a new visualisation tool to plot footprints with various spherical projections was developed. Indexing of the footprints using Hierarchical Triangular Mesh makes it possible to quickly find observations based on sky coverage, time and meta-data. The database is accessible via a web site (http://herschel.vo.elte.hu) and also as a set of REST web service functions.Comment: Accepted for publication in Experimental Astronom

Optimizing ISOCAM data processing using spatial redundancy

We present new data processing techniques that allow to correct the main instrumental effects that degrade the images obtained by ISOCAM, the camera on board the Infrared Space Observatory (ISO). Our techniques take advantage of the fact that a position on the sky has been observed by several pixels at different times. We use this information (1) to correct the long term variation of the detector response, (2) to correct memory effects after glitches and point sources, and (3) to refine the deglitching process. Our new method allows the detection of faint extended emission with contrast smaller than 1% of the zodiacal background. The data reduction corrects instrumental effects to the point where the noise in the final map is dominated by the readout and the photon noises. All raster ISOCAM observations can benefit from the data processing described here. These techniques could also be applied to other raster type observations (e.g. ISOPHOT or IRAC on SIRTF).Comment: 13 pages, 10 figures, to be published in Astronomy and Astrophysics Supplement Serie

Using treemaps for variable selection in spatio-temporal visualisation

We demonstrate and reflect upon the use of enhanced treemaps that incorporate spatial and temporal ordering for exploring a large multivariate spatio-temporal data set. The resulting data-dense views summarise and simultaneously present hundreds of space-, time-, and variable-constrained subsets of a large multivariate data set in a structure that facilitates their meaningful comparison and supports visual analysis. Interactive techniques allow localised patterns to be explored and subsets of interest selected and compared with the spatial aggregate. Spatial variation is considered through interactive raster maps and high-resolution local road maps. The techniques are developed in the context of 42.2 million records of vehicular activity in a 98 km(2) area of central London and informally evaluated through a design used in the exploratory visualisation of this data set. The main advantages of our technique are the means to simultaneously display hundreds of summaries of the data and to interactively browse hundreds of variable combinations with ordering and symbolism that are consistent and appropriate for space- and time- based variables. These capabilities are difficult to achieve in the case of spatio-temporal data with categorical attributes using existing geovisualisation methods. We acknowledge limitations in the treemap representation but enhance the cognitive plausibility of this popular layout through our two-dimensional ordering algorithm and interactions. Patterns that are expected (e.g. more traffic in central London), interesting (e.g. the spatial and temporal distribution of particular vehicle types) and anomalous (e.g. low speeds on particular road sections) are detected at various scales and locations using the approach. In many cases, anomalies identify biases that may have implications for future use of the data set for analyses and applications. Ordered treemaps appear to have potential as interactive interfaces for variable selection in spatio-temporal visualisation. Information Visualization (2008) 7, 210-224. doi: 10.1057/palgrave.ivs.950018

The Bolocam Galactic Plane Survey: Survey Description and Data Reduction

We present the Bolocam Galactic Plane Survey (BGPS), a 1.1 mm continuum survey at 33" effective resolution of 170 square degrees of the Galactic Plane visible from the northern hemisphere. The survey is contiguous over the range -10.5 < l < 90.5, |b| < 0.5 and encompasses 133 square degrees, including some extended regions |b| < 1.5. In addition to the contiguous region, four targeted regions in the outer Galaxy were observed: IC1396, a region towards the Perseus Arm, W3/4/5, and Gem OB1. The BGPS has detected approximately 8400 clumps over the entire area to a limiting non-uniform 1-sigma noise level in the range 11 to 53 mJy/beam in the inner Galaxy. The BGPS source catalog is presented in a companion paper (Rosolowsky et al. 2010). This paper details the survey observations and data reduction methods for the images. We discuss in detail the determination of astrometric and flux density calibration uncertainties and compare our results to the literature. Data processing algorithms that separate astronomical signals from time-variable atmospheric fluctuations in the data time-stream are presented. These algorithms reproduce the structure of the astronomical sky over a limited range of angular scales and produce artifacts in the vicinity of bright sources. Based on simulations, we find that extended emission on scales larger than about 5.9' is nearly completely attenuated (> 90%) and the linear scale at which the attenuation reaches 50% is 3.8'. Comparison with other millimeter-wave data sets implies a possible systematic offset in flux calibration, for which no cause has been discovered. This presentation serves as a companion and guide to the public data release through NASA's Infrared Processing and Analysis Center (IPAC) Infrared Science Archive (IRSA). New data releases will be provided through IPAC IRSA with any future improvements in the reduction.Comment: Accepted for publication in Astrophysical Journal Supplemen