Distributed and parallel sparse convex optimization for radio interferometry with PURIFY

Next generation radio interferometric telescopes are entering an era of big data with extremely large data sets. While these telescopes can observe the sky in higher sensitivity and resolution than before, computational challenges in image reconstruction need to be overcome to realize the potential of forthcoming telescopes. New methods in sparse image reconstruction and convex optimization techniques (cf. compressive sensing) have shown to produce higher fidelity reconstructions of simulations and real observations than traditional methods. This article presents distributed and parallel algorithms and implementations to perform sparse image reconstruction, with significant practical considerations that are important for implementing these algorithms for Big Data. We benchmark the algorithms presented, showing that they are considerably faster than their serial equivalents. We then pre-sample gridding kernels to scale the distributed algorithms to larger data sizes, showing application times for 1 Gb to 2.4 Tb data sets over 25 to 100 nodes for up to 50 billion visibilities, and find that the run-times for the distributed algorithms range from 100 milliseconds to 3 minutes per iteration. This work presents an important step in working towards computationally scalable and efficient algorithms and implementations that are needed to image observations of both extended and compact sources from next generation radio interferometers such as the SKA. The algorithms are implemented in the latest versions of the SOPT (https://github.com/astro-informatics/sopt) and PURIFY (https://github.com/astro-informatics/purify) software packages {(Versions 3.1.0)}, which have been released alongside of this article.Comment: 25 pages, 5 figure

Bootstrap resampling as a tool for radio-interferometric imaging fidelity assessment

We report on a numerical evaluation of the statistical bootstrap as a technique for radio-interferometric imaging fidelity assessment. The development of a fidelity assessment technique is an important scientific prerequisite for automated pipeline reduction of data from modern radio interferometers. We evaluate the statistical performance of two bootstrap methods, the model-based bootstrap and subsample bootstrap, against a Monte Carlo parametric simulation, using interferometric polarization calibration and imaging as the representative problem under study. We find both statistical resampling techniques to be viable approaches to radio-interferometric imaging fidelity assessment which merit further investigation. We also report on the development and implementation of a new self-calibration algorithm for radio-interferometric polarimetry which makes no approximations for the polarization source model.Comment: Accepted by AJ; 41 pages, 13 figure

Radio Astronomy Image Reconstruction in the Big Data Era

Next generation radio interferometric telescopes pave the way for the future of radio astronomy with extremely wide-fields of view and precision polarimetry not possible at other optical wavelengths, with the required cost of image reconstruction. These instruments will be used to map large scale Galactic and extra-galactic structures at higher resolution and fidelity than ever before. However, radio astronomy has entered the era of big data, limiting the expected sensitivity and fidelity of the instruments due to the large amounts of data. New image reconstruction methods are critical to meet the data requirements needed to obtain new scientific discoveries in radio astronomy. To meet this need, this work takes traditional radio astronomical imaging and introduces new of state-of-the-art image reconstruction frameworks of sparse image reconstruction algorithms. The software package PURIFY, developed in this work, uses convex optimization algorithms (i.e. alternating direction method of multipliers) to solve for the reconstructed image. We design, implement, and apply distributed radio interferometric image reconstruction methods for the message passing interface (MPI), showing that PURIFY scales to big data image reconstruction on computing clusters. We design a distributed wide-field imaging algorithm for non-coplanar arrays, while providing new theoretical insights for wide-field imaging. It is shown that PURIFY’s methods provide higher dynamic range than traditional image reconstruction methods, providing a more accurate and detailed sky model for real observations. This sets the stage for state-of-the-art image reconstruction methods to be distributed and applied to next generation interferometric telescopes, where they can be used to meet big data challenges and to make new scientific discoveries in radio astronomy and astrophysics

Advanced sparse optimization algorithms for interferometric imaging inverse problems in astronomy

In the quest to produce images of the sky at unprecedented resolution with high sensitivity, new generation of astronomical interferometers have been designed. To meet the sensing capabilities of these instruments, techniques aiming to recover the sought images from the incompletely sampled Fourier domain measurements need to be reinvented. This goes hand-in-hand with the necessity to calibrate the measurement modulating unknown effects, which adversely affect the image quality, limiting its dynamic range. The contribution of this thesis consists in the development of advanced optimization techniques tailored to address these issues, ranging from radio interferometry (RI) to optical interferometry (OI). In the context of RI, we propose a novel convex optimization approach for full polarization imaging relying on sparsity-promoting regularizations. Unlike standard RI imaging algorithms, our method jointly solves for the Stokes images by enforcing the polarization constraint, which imposes a physical dependency between the images. These priors are shown to enhance the imaging quality via various performed numerical studies. The proposed imaging approach also benefits from its scalability to handle the huge amounts of data expected from the new instruments. When it comes to deal with the critical and challenging issues of the direction-dependent effects calibration, we further propose a non-convex optimization technique that unifies calibration and imaging steps in a global framework, in which we adapt the earlier developed imaging method for the imaging step. In contrast to existing RI calibration modalities, our method benefits from well-established convergence guarantees even in the non-convex setting considered in this work and its efficiency is demonstrated through several numerical experiments. Last but not least, inspired by the performance of these methodologies and drawing ideas from them, we aim to solve image recovery problem in OI that poses its own set of challenges primarily due to the partial loss of phase information. To this end, we propose a sparsity regularized non-convex optimization algorithm that is equipped with convergence guarantees and is adaptable to both monochromatic and hyperspectral OI imaging. We validate it by presenting the simulation results

The Precision Array for Probing the Epoch of Reionization: 8 Station Results

We are developing the Precision Array for Probing the Epoch of Reionization (PAPER) to detect 21cm emission from the early Universe, when the first stars and galaxies were forming. We describe the overall experiment strategy and architecture and summarize two PAPER deployments: a 4-antenna array in the low-RFI environment of Western Australia and an 8-antenna array at our prototyping site in Green Bank, WV. From these activities we report on system performance, including primary beam model verification, dependence of system gain on ambient temperature, measurements of receiver and overall system temperatures, and characterization of the RFI environment at each deployment site. We present an all-sky map synthesized between 139 MHz and 174 MHz using data from both arrays that reaches down to 80 mJy (4.9 K, for a beam size of 2.15e-5 steradians at 154 MHz), with a 10 mJy (620 mK) thermal noise level that indicates what would be achievable with better foreground subtraction. We calculate angular power spectra ($C_\ell$) in a cold patch and determine them to be dominated by point sources, but with contributions from galactic synchrotron emission at lower radio frequencies and angular wavemodes. Although the cosmic variance of foregrounds dominates errors in these power spectra, we measure a thermal noise level of 310 mK at $\ell=100$ for a 1.46-MHz band centered at 164.5 MHz. This sensitivity level is approximately three orders of magnitude in temperature above the level of the fluctuations in 21cm emission associated with reionization.Comment: 13 pages, 14 figures, submitted to AJ. Revision 2 corrects a scaling error in the x axis of Fig. 12 that lowers the calculated power spectrum temperatur

Deciphering Radio Emission from Solar Coronal Mass Ejections using High-fidelity Spectropolarimetric Radio Imaging

Coronal mass ejections (CMEs) are large-scale expulsions of plasma and magnetic fields from the Sun into the heliosphere and are the most important driver of space weather. The geo-effectiveness of a CME is primarily determined by its magnetic field strength and topology. Measurement of CME magnetic fields, both in the corona and heliosphere, is essential for improving space weather forecasting. Observations at radio wavelengths can provide several remote measurement tools for estimating both strength and topology of the CME magnetic fields. Among them, gyrosynchrotron (GS) emission produced by mildly-relativistic electrons trapped in CME magnetic fields is one of the promising methods to estimate magnetic field strength of CMEs at lower and middle coronal heights. However, GS emissions from some parts of the CME are much fainter than the quiet Sun emission and require high dynamic range (DR) imaging for their detection. This thesis presents a state-of-the-art calibration and imaging algorithm capable of routinely producing high DR spectropolarimetric snapshot solar radio images using data from a new technology radio telescope, the Murchison Widefield Array. This allows us to detect much fainter GS emissions from CME plasma at much higher coronal heights. For the first time, robust circular polarization measurements have been jointly used with total intensity measurements to constrain the GS model parameters, which has significantly improved the robustness of the estimated GS model parameters. A piece of observational evidence is also found that routinely used homogeneous and isotropic GS models may not always be sufficient to model the observations. In the future, with upcoming sensitive telescopes and physics-based forward models, it should be possible to relax some of these assumptions and make this method more robust for estimating CME plasma parameters at coronal heights.Comment: 297 pages, 100 figures, 9 tables. Submitted at Tata Institute of Fundamental Research, Mumbai, India, Ph.D Thesi