21,798 research outputs found
Data Provenance and Management in Radio Astronomy: A Stream Computing Approach
New approaches for data provenance and data management (DPDM) are required
for mega science projects like the Square Kilometer Array, characterized by
extremely large data volume and intense data rates, therefore demanding
innovative and highly efficient computational paradigms. In this context, we
explore a stream-computing approach with the emphasis on the use of
accelerators. In particular, we make use of a new generation of high
performance stream-based parallelization middleware known as InfoSphere
Streams. Its viability for managing and ensuring interoperability and integrity
of signal processing data pipelines is demonstrated in radio astronomy. IBM
InfoSphere Streams embraces the stream-computing paradigm. It is a shift from
conventional data mining techniques (involving analysis of existing data from
databases) towards real-time analytic processing. We discuss using InfoSphere
Streams for effective DPDM in radio astronomy and propose a way in which
InfoSphere Streams can be utilized for large antennae arrays. We present a
case-study: the InfoSphere Streams implementation of an autocorrelating
spectrometer, and using this example we discuss the advantages of the
stream-computing approach and the utilization of hardware accelerators
Distributed image reconstruction for very large arrays in radio astronomy
Current and future radio interferometric arrays such as LOFAR and SKA are
characterized by a paradox. Their large number of receptors (up to millions)
allow theoretically unprecedented high imaging resolution. In the same time,
the ultra massive amounts of samples makes the data transfer and computational
loads (correlation and calibration) order of magnitudes too high to allow any
currently existing image reconstruction algorithm to achieve, or even approach,
the theoretical resolution. We investigate here decentralized and distributed
image reconstruction strategies which select, transfer and process only a
fraction of the total data. The loss in MSE incurred by the proposed approach
is evaluated theoretically and numerically on simple test cases.Comment: Sensor Array and Multichannel Signal Processing Workshop (SAM), 2014
IEEE 8th, Jun 2014, Coruna, Spain. 201
- …