Graph Spectral Clustering of Convolution Artefacts in Radio Interferometric Images

The starting point for deconvolution methods in radioastronomy is an estimate of the sky intensity called a dirty image. These methods rely on the telescope point-spread function so as to remove artefacts which pollute it. In this work, we show that the intensity field is only a partial summary statistic of the matched filtered interferometric data, which we prove is spatially correlated on the celestial sphere. This allows us to define a sky covariance function. This previously unexplored quantity brings us additional information that can be leveraged in the process of removing dirty image artefacts. We demonstrate this using a novel unsupervised learning method. The problem is formulated on a graph: each pixel interpreted as a node, linked by edges weighted according to their spatial correlation. We then use spectral clustering to separate the artefacts in groups, and identify physical sources within them

Using JVLA to remove CMB foregrounds.

Master of Science in Chemistry. University of KwaZulu-Natal, Durban 2016

Accurate image reconstruction in radio interferometry

This thesis is concerned with accurate imaging from radio interferometry data and with subsequent analysis so as to determine source positions and fluxes in the radio sky. The thesis makes proposals and implementations of new algorithms, which substantially improve the accuracy of image products and the results of source extraction. These improvements in accuracy promise to assist scientific research into astronomical objects and phenomena in radio astronomy. The thesis contains six chapters, beginning with an overview of the imaging process in radio interferometry in Chapter 1. Chapter 2 focuses on improving the accuracy of source extraction, by utilising the Bayesian methodology. The proposed Bayesian method has been implemented in a software package called 'BaSC' which uses the Markov Chain Monte Carlo (MCMC) technique. By design, it works with intermediate radio interferometry image products, such as dirty images, rather than with reconstructed images. BaSC achieves greater precision in source location and better resolving power than mainstream source extraction software such as SExtractor, which works with reconstructed images. This finding confirms that reconstructed images are not a true representation of the radio sky, whereas dirty images already contain full information about the observations. This piece of work has been accepted by Monthly Notices of the Royal Astronomical Society (Hague et al. 2018). Chapter 2 is based on this paper, but has been rewritten and expanded. Based on this conclusion, Chapter 3 seeks to optimise the gridding process so as to make accurate dirty images. Since the Fast Fourier transform (FFT) produces dirty images with a much lower computational cost than the Direct Fourier transform (DFT), a new gridding function is needed which minimises the difference between DFT and FFT dirty images. The 'Least-misfit' gridding function is proposed, so as to minimise the image misfit between the DFT and FFT dirty images, and this is implemented and tested. Given an identical support width, it outperforms the main-stream spheroidal function in the image misfit by a factor of at least 100, it also suppresses aliasing in the image plane better. Aliasing is essentially a part of the image misfit, so there is no need to consider it separately. The least-misfit gridding function, with a support width of 7 and an image cropping rate of 0.5, is recommended for application to both the gridding and degridding processes, and makes it realistic to achieve the limit of single precision arithmetic in the image misfit and visibility misfit. With the new gridding function in place, Chapter 4 proposes two novel wide-field imaging algorithms: improved W-Stacking and N-Faceting. The improved W-Stacking method uses a three-dimensional gridding, rather than two-dimensional gridding as in the original W-Stacking method. This renders possible the calculation and application of the correcting function on the $n$ (third) dimension. This improvement greatly increases the accuracy of the FFT dirty image on the celestial sphere, relative to the DFT dirty image. The image misfit is as small as $10^{-8}$ when using the proposed least-misfit gridding function with a support width of 8, and it further reaches the double precision limit when the support width is increased to 14. For comparison, the image misfit levels achieved by the W-Projection algorithm in CASA and the original W-Stacking algorithm in WSCLEAN are $10^{-3}$, several orders of magnitude worse. In addition, since the number of $w$-planes required by the improved W-Stacking method is reduced compared to the original method, cutting a significant amount of FFT computational cost. As for the original W-Stacking method, if less $w$-planes than needed are used, the dirty images and reconstructed images produced will underestimate the fluxes of sources that are located far from the phase centre. The N-Faceting method involves imaging of multiple $n$-planes, followed by a three-dimensional deconvolution process, where a position-independent beam is used. Chapter 5 applies the improved W-Stacking method to two real sets of observational data, comprising one GMRT dataset and one VLA dataset. The dirty images on the celestial sphere and the reconstructed images are shown. The improved W-Stacking method successfully removes non-coplanar effects. For the observation with a larger range of $w$, improved W-Stacking method is recommended to generate a more accurate image with lower computational cost compared to the original W-Stacking method. Finally, Chapter 6 sets out conclusions drawn from the present work

Advanced Computational Methods for Oncological Image Analysis

[Cancer is the second most common cause of death worldwide and encompasses highly variable clinical and biological scenarios. Some of the current clinical challenges are (i) early diagnosis of the disease and (ii) precision medicine, which allows for treatments targeted to specific clinical cases. The ultimate goal is to optimize the clinical workflow by combining accurate diagnosis with the most suitable therapies. Toward this, large-scale machine learning research can define associations among clinical, imaging, and multi-omics studies, making it possible to provide reliable diagnostic and prognostic biomarkers for precision oncology. Such reliable computer-assisted methods (i.e., artificial intelligence) together with clinicians’ unique knowledge can be used to properly handle typical issues in evaluation/quantification procedures (i.e., operator dependence and time-consuming tasks). These technical advances can significantly improve result repeatability in disease diagnosis and guide toward appropriate cancer care. Indeed, the need to apply machine learning and computational intelligence techniques has steadily increased to effectively perform image processing operations—such as segmentation, co-registration, classification, and dimensionality reduction—and multi-omics data integration.

An RFI simulation pipeline to help teach interferometry and machine learning

Thesis (MEng)--Stellenbosch University, 2022.ENGLISH ABSTRACT: An interferometer is a collection of radio antennas that together form one instrument. Machine Learning is the collective term that is used to refer to a set of algorithms that can automatically learn to perform a specific task if it is provided with training examples. Interferometry has become an intricate part of the scientific landscape in South Africa with the advent of MeerKAT. Similarly, utilizing Machine Learning (ML to improve our lives has grown in popularity worldwide. Machine Learning is nowadays used to determine the likes of people, to interpret human utterings, to automatically classify images and the like. As these two fields grow in popularity and importance within the South African context, so does the development of tools that can aid in teaching these fields to undergraduate students. A major problem for radio observatories worldwide is Radio Frequency Interference (RFI. RFI can be detected using ML. A simulator that can simulate interferometric observations that are corrupted by RFI can serve as a testbed for different ML approaches. Moreover, if the simulator is simplistic enough it can even be utilized as a teaching tool. In this thesis such a simulator is developed. This simulator can aid in teaching students how visibilities can be simulated and how RFI can be detected via ML. In effect, one tool that can help teach two relevant undergraduate topics, namely interferometry and ML. In particular, an experiment is proposed which an undergraduate student can repeat to gain a deeper understanding of interferometry and ML. In this experiment, visibilities are simulated, RFI is injected and detected using four different ML techniques, namely Naive Bayes, Logistic Regression, k-means and Gaussian Mixture Models (GMM). The results are then analysed and conclusions are drawn. For the simplistic setup considered here, the ranking of the four algorithms is from best to worst: Naive Bayes, Logistic Regression, GMM and then k-means. In the future, if the simulator is extended somewhat, it can also be used as a testbed for comparing numerous other ML algorithms. The thesis also provides a comprehensive review of all the theory that a student requires to master both interferometry and ML.AFRIKAANSE OPSOMMING: 'n Interferometer is 'n versameling van radio antennas wat saam een instrument vorm. Masjienleer is die kollektiewe term wat grebruik word om te verwys na 'n stel algoritmes wat automaties kan leer hoe om 'n spesifieke funksies te verrig, gegee afrigtingsvoorbeelde. Interferometrie, het 'n belangrike deel van die wetenskaplike landskap in Suid-Afrika geword met die loots van MeerKAT. Soortgelyk, masjienleer se gebruik het wˆereldwyd drasties gegroei. Masjienleer word deesdae gebruik om die voorkeure van mense te bepaal, om die woorde wat mense uiter te herken, om prentjies te klassifiseer en dies meer. Soos wat die twee velde se gewildheid groei, word dit al hoe meer belangrik om toepassings te ontwikkel wat gebruik kan word om te help om die twee velde aan voorgraadse studente te verduidelik. 'n Groot probleem wat radio-sterrewagte in die gesig staar is Radio Frekwensie Inmenging (RFI. RFI kan met behulp van masjienleer geïdentifiseer word. 'n Simulator wat sigbaarheidsmetings kan genereer wat besmet is met RFI kan gebruik word om verkillende masjienleer tegnieke met mekaar te vergelyk. Verder, as 'n simulator eenvoudig genoeg is, kan dit ook gebruik word as 'n onderrigstoepassing. In hierdie tesis word so 'n simulator ontwikkel. Die simulator kan gebruik word om beide, interferometrie en masjienleer, aan studente te verduidelik. Meer spesifiek, 'n eksperiment word voorgestel wat studente sal kan herhaal. In die eksperiment word sigbaarhaeidsmetings gegenereer wat vermeng word met RFI. Vier masjienleer algoritmes word dan gebruik om die RFI te identi seer. Die vier algoritmes is: Naïewe Bayes, Lo gistiese Regressie, Gausiese Mengsel Modelle (GMM) en k-gemdideldes. Die akkuraatheidsrangorde van die vier algoritmes, soos in die studie bevind, is dieselfde as wat hier gegee is. As die simulator uitgebrei word kan dit ook gebruik word om verkeie ander masjienleeralgoritmes met mekaar te vergelyk. Die tesis bevat ook 'n oorsig van al die teorie wat 'n student sou kon help om beide velde te bemeester.Master

Advanced VLBI Imaging

Very Long Baseline Interferometry (VLBI) is an observational technique developed in astronomy for combining multiple radio telescopes into a single virtual instrument with an effective aperture reaching up to many thousand kilometers and enabling measurements at highest angular resolutions. The celebrated examples of applying VLBI to astrophysical studies include detailed, high-resolution images of the innermost parts of relativistic outflows (jets) in active galactic nuclei (AGN) and recent pioneering observations of the shadows of supermassive black holes (SMBH) in the center of our Galaxy and in the galaxy M87. Despite these and many other proven successes of VLBI, analysis and imaging of VLBI data still remain difficult, owing in part to the fact that VLBI imaging inherently constitutes an ill-posed inverse problem. Historically, this problem has been addressed in radio interferometry by the CLEAN algorithm, a matching-pursuit inverse modeling method developed in the early 1970-s and since then established as a de-facto standard approach for imaging VLBI data. In recent years, the constantly increasing demand for improving quality and fidelity of interferometric image reconstruction has resulted in several attempts to employ new approaches, such as forward modeling and Bayesian estimation, for application to VLBI imaging. While the current state-of-the-art forward modeling and Bayesian techniques may outperform CLEAN in terms of accuracy, resolution, robustness, and adaptability, they also tend to require more complex structure and longer computation times, and rely on extensive finetuning of a larger number of non-trivial hyperparameters. This leaves an ample room for further searches for potentially more effective imaging approaches and provides the main motivation for this dissertation and its particular focusing on the need to unify algorithmic frameworks and to study VLBI imaging from the perspective of inverse problems in general. In pursuit of this goal, and based on an extensive qualitative comparison of the existing methods, this dissertation comprises the development, testing, and first implementations of two novel concepts for improved interferometric image reconstruction. The concepts combine the known benefits of current forward modeling techniques, develop more automatic and less supervised algorithms for image reconstruction, and realize them within two different frameworks. The first framework unites multiscale imaging algorithms in the spirit of compressive sensing with a dictionary adapted to the uv-coverage and its defects (DoG-HiT, DoB-CLEAN). We extend this approach to dynamical imaging and polarimetric imaging. The core components of this framework are realized in a multidisciplinary and multipurpose software MrBeam, developed as part of this dissertation. The second framework employs a multiobjective genetic evolutionary algorithm (MOEA/D) for the purpose of achieving fully unsupervised image reconstruction and hyperparameter optimization. These new methods are shown to outperform the existing methods in various metrics such as angular resolution, structural sensitivity, and degree of supervision. We demonstrate the great potential of these new techniques with selected applications to frontline VLBI observations of AGN jets and SMBH. In addition to improving the quality and robustness of image reconstruction, DoG-HiT, DoB-CLEAN and MOEA/D also provide such novel capabilities as dynamic reconstruction of polarimetric images on minute time-scales, or near-real time and unsupervised data analysis (useful in particular for application to large imaging surveys). The techniques and software developed in this dissertation are of interest for a wider range of inverse problems as well. This includes such versatile fields such as Ly-alpha tomography (where we improve estimates of the thermal state of the intergalactic medium), the cosmographic search for dark matter (where we improve forecasted bounds on ultralight dilatons), medical imaging, and solar spectroscopy

Proceedings of Abstracts, School of Physics, Engineering and Computer Science Research Conference 2022

© 2022 The Author(s). This is an open-access work distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. For further details please see https://creativecommons.org/licenses/by/4.0/. Plenary by Prof. Timothy Foat, ‘Indoor dispersion at Dstl and its recent application to COVID-19 transmission’ is © Crown copyright (2022), Dstl. This material is licensed under the terms of the Open Government Licence except where otherwise stated. To view this licence, visit http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected] present proceedings record the abstracts submitted and accepted for presentation at SPECS 2022, the second edition of the School of Physics, Engineering and Computer Science Research Conference that took place online, the 12th April 2022

SKA Science Data Challenge 2: analysis and results

The Square Kilometre Array Observatory (SKAO) will explore the radio sky to new depths in order to conduct transformational science. SKAO data products made available to astronomers will be correspondingly large and complex, requiring the application of advanced analysis techniques to extract key science findings. To this end, SKAO is conducting a series of Science Data Challenges, each designed to familiarise the scientific community with SKAO data and to drive the development of new analysis techniques. We present the results from Science Data Challenge 2 (SDC2), which invited participants to find and characterise 233245 neutral hydrogen (Hi) sources in a simulated data product representing a 2000~h SKA MID spectral line observation from redshifts 0.25 to 0.5. Through the generous support of eight international supercomputing facilities, participants were able to undertake the Challenge using dedicated computational resources. Alongside the main challenge, `reproducibility awards' were made in recognition of those pipelines which demonstrated Open Science best practice. The Challenge saw over 100 participants develop a range of new and existing techniques, with results that highlight the strengths of multidisciplinary and collaborative effort. The winning strategy -- which combined predictions from two independent machine learning techniques to yield a 20 percent improvement in overall performance -- underscores one of the main Challenge outcomes: that of method complementarity. It is likely that the combination of methods in a so-called ensemble approach will be key to exploiting very large astronomical datasets.Comment: Under review by MNRAS; 28 pages, 16 figure