The Data Big Bang and the Expanding Digital Universe: High-Dimensional, Complex and Massive Data Sets in an Inflationary Epoch

Recent and forthcoming advances in instrumentation, and giant new surveys, are creating astronomical data sets that are not amenable to the methods of analysis familiar to astronomers. Traditional methods are often inadequate not merely because of the size in bytes of the data sets, but also because of the complexity of modern data sets. Mathematical limitations of familiar algorithms and techniques in dealing with such data sets create a critical need for new paradigms for the representation, analysis and scientific visualization (as opposed to illustrative visualization) of heterogeneous, multiresolution data across application domains. Some of the problems presented by the new data sets have been addressed by other disciplines such as applied mathematics, statistics and machine learning and have been utilized by other sciences such as space-based geosciences. Unfortunately, valuable results pertaining to these problems are mostly to be found only in publications outside of astronomy. Here we offer brief overviews of a number of concepts, techniques and developments, some "old" and some new. These are generally unknown to most of the astronomical community, but are vital to the analysis and visualization of complex datasets and images. In order for astronomers to take advantage of the richness and complexity of the new era of data, and to be able to identify, adopt, and apply new solutions, the astronomical community needs a certain degree of awareness and understanding of the new concepts. One of the goals of this paper is to help bridge the gap between applied mathematics, artificial intelligence and computer science on the one side and astronomy on the other.Comment: 24 pages, 8 Figures, 1 Table. Accepted for publication: "Advances in Astronomy, special issue "Robotic Astronomy

Problems in computational helioseismology

We discuss current advances in forward and inverse modeling for local helioseismology. We report theoretical uniqueness results, in particular the Novikov-Agaltsov reconstruction algorithm, which is relevant to solving the non-linear inverse problem of time-distance helioseismology (finite amplitude pertubations to the medium). Numerical experiments were conducted to determine the number of frequencies required to reconstruct density and sound speed in the solar interior.Comment: Oberwolfach Report, Computational Inverse Problems for Partial Differential Equations, 14 May - 20 May 2017. https://www.mfo.de/occasion/1720/www_vie

Phase Retrieval for Sparse Signals: Uniqueness Conditions

In a variety of fields, in particular those involving imaging and optics, we often measure signals whose phase is missing or has been irremediably distorted. Phase retrieval attempts the recovery of the phase information of a signal from the magnitude of its Fourier transform to enable the reconstruction of the original signal. A fundamental question then is: "Under which conditions can we uniquely recover the signal of interest from its measured magnitudes?" In this paper, we assume the measured signal to be sparse. This is a natural assumption in many applications, such as X-ray crystallography, speckle imaging and blind channel estimation. In this work, we derive a sufficient condition for the uniqueness of the solution of the phase retrieval (PR) problem for both discrete and continuous domains, and for one and multi-dimensional domains. More precisely, we show that there is a strong connection between PR and the turnpike problem, a classic combinatorial problem. We also prove that the existence of collisions in the autocorrelation function of the signal may preclude the uniqueness of the solution of PR. Then, assuming the absence of collisions, we prove that the solution is almost surely unique on 1-dimensional domains. Finally, we extend this result to multi-dimensional signals by solving a set of 1-dimensional problems. We show that the solution of the multi-dimensional problem is unique when the autocorrelation function has no collisions, significantly improving upon a previously known result.Comment: submitted to IEEE TI

A Monte Carlo Template based analysis for Air-Cherenkov Arrays

We present a high-performance event reconstruction algorithm: an Image Pixel-wise fit for Atmospheric Cherenkov Telescopes (ImPACT). The reconstruction algorithm is based around the likelihood fitting of camera pixel amplitudes to an expected image template. A maximum likelihood fit is performed to find the best-fit shower parameters. A related reconstruction algorithm has already been shown to provide significant improvements over traditional reconstruction for both the CAT and H.E.S.S. experiments. We demonstrate a significant improvement to the template generation step of the procedure, by the use of a full Monte Carlo air shower simulation in combination with a ray-tracing optics simulation to more accurately model the expected camera images. This reconstruction step is combined with an MVA-based background rejection. Examples are shown of the performance of the ImPACT analysis on both simulated and measured (from a strong VHE source) gamma-ray data from the H.E.S.S. array, demonstrating an improvement in sensitivity of more than a factor two in observation time over traditional image moments-fitting methods, with comparable performance to previous likelihood fitting analyses. ImPACT is a particularly promising approach for future large arrays such as the Cherenkov Telescope Array (CTA) due to its improved high-energy performance and suitability for arrays of mixed telescope types.Comment: 13 pages, 10 figure

High-Dimensional Data Reduction, Image Inpainting and their Astronomical Applications

Technological advances are revolutionizing multispectral astrophysics as well as the detection and study of transient sources. This new era of multitemporal and multispectral data sets demands new ways of data representation, processing and management thus making data dimension reduction instrumental in efficient data organization, retrieval, analysis and information visualization. Other astrophysical applications of data dimension reduction which require new paradigms of data analysis include knowledge discovery, cluster analysis, feature extraction and object classification, de-correlating data elements, discovering meaningful patterns and finding essential representation of correlated variables that form a manifold (e.g. the manifold of galaxies), tagging astronomical images, multiscale analysis synchronized across all available wavelengths, denoising, etc. The second part of this paper is dedicated to a new, active area of image processing: image inpainting that consists of automated methods for filling in missing or damaged regions in images. Inpainting has multiple astronomical applications including restoring images corrupted by instrument artifacts, removing undesirable objects like bright stars and their halos, sky estimating, and pre-processing for the Fourier or wavelet transforms. Applications of high-dimensional data reduction and mitigation of instrument artifacts are demonstrated on images taken by the Spitzer Space Telescope

Phaseless VLBI mapping of compact extragalactic radio sources

The problem of phaseless aperture synthesis is of current interest in phase-unstable VLBI with a small number of elements when either the use of closure phases is not possible (a two-element interferometer) or their quality and number are not enough for acceptable image reconstruction by standard adaptive calibration methods. Therefore, we discuss the problem of unique image reconstruction only from the spectrum magnitude of a source. We suggest an efficient method for phaseless VLBI mapping of compact extragalactic radio sources. This method is based on the reconstruction of the spectrum magnitude for a source on the entire UV plane from the measured visibility magnitude on a limited set of points and the reconstruction of the sought-for image of the source by Fienup's method from the spectrum magnitude reconstructed at the first stage. We present the results of our mapping of the extragalactic radio source 2200 +420 using astrometric and geodetic observations on a global VLBI array. Particular attention is given to studying the capabilities of a two-element interferometer in connection with the putting into operation of a Russian-made radio interferometer based on Quasar RT-32 radio telescopes.Comment: 21 pages, 6 figure

On Relaxed Averaged Alternating Reflections (RAAR) Algorithm for Phase Retrieval from Structured Illuminations

In this paper, as opposed to the random phase masks, the structured illuminations with a pixel-dependent deterministic phase shift are considered to derandomize the model setup. The RAAR algorithm is modified to adapt to two or more diffraction patterns, and the modified RAAR algorithm operates in Fourier domain rather than space domain. The local convergence of the RAAR algorithm is proved by some eigenvalue analysis. Numerical simulations is presented to demonstrate the effectiveness and stability of the algorithm compared to the HIO (Hybrid Input-Output) method. The numerical performances show the global convergence of the RAAR in our tests.Comment: 17 pages, 26 figures, submitting to Inverse Problem

The Atacama Cosmology Telescope: CO(J = 3 - 2) mapping and lens modeling of an ACT-selected dusty star-forming galaxy

We report Northern Extended Millimeter Array (NOEMA) CO($J = 3 - 2$) observations of the dusty star-forming galaxy ACT-S\,J020941+001557 at $z = 2.5528$, which was detected as an unresolved source in the Atacama Cosmology Telescope (ACT) equatorial survey. Our spatially resolved spectral line data support the derivation of a gravitational lens model from 37 independent velocity channel maps using a pixel-based algorithm, from which we infer a velocity-dependent magnification factor $\mu \approx 7-22$ with a luminosity-weighted mean \left\approx 13. The resulting source-plane reconstruction is consistent with a rotating disk, although other scenarios cannot be ruled out by our data. After correction for lensing, we derive a line luminosity $L^{\prime}_{\rm CO(3-2)}= (5.53\pm 0.69) \times 10^{10}\,{\rm \,K\,km\,s^{-1}\,pc^{2}}$, a cold gas mass $M_{{\rm gas}}= (3.86 \pm 0.33) \times 10^{10}\,M_{\odot}$, a dynamical mass $M_{\rm dyn}\,{\rm sin}^2\,i = 3.9^{+1.8}_{-1.5} \times 10^{10}\,M_{\odot}$, and a gas mass fraction $f_{\rm gas}\,{\rm csc}^2\,i = 1.0^{+0.8}_{-0.4}$. The line brightness temperature ratio of $r_{3,1}\approx 1.6$ relative to a Green Bank Telescope CO($J=1-0$) detection may be elevated by a combination of external heating of molecular clouds, differential lensing, and/or pointing errors.Comment: 8 pages, 5 figures, accepted to Ap