NEARBY Platform: Algorithm for Automated Asteroids Detection in Astronomical Images

In the past two decades an increasing interest in discovering Near Earth Objects has been noted in the astronomical community. Dedicated surveys have been operated for data acquisition and processing, resulting in the present discovery of over 18.000 objects that are closer than 30 million miles of Earth. Nevertheless, recent events have shown that there still are many undiscovered asteroids that can be on collision course to Earth. This article presents an original NEO detection algorithm developed in the NEARBY research object, that has been integrated into an automated MOPS processing pipeline aimed at identifying moving space objects based on the blink method. Proposed solution can be considered an approach of Big Data processing and analysis, implementing visual analytics techniques for rapid human data validation.Comment: IEEE 14th International Conference on Intelligent Computer Communication and Processing (ICCP), Sep 6-8, 2018, Cluj-Napoca, Romani

Science Pipelines for the Square Kilometre Array

The Square Kilometre Array (SKA) will be both the largest radio telescope ever constructed and the largest Big Data project in the known Universe. The first phase of the project will generate on the order of 5 zettabytes of data per year. A critical task for the SKA will be its ability to process data for science, which will need to be conducted by science pipelines. Together with polarization data from the LOFAR Multifrequency Snapshot Sky Survey (MSSS), we have been developing a realistic SKA-like science pipeline that can handle the large data volumes generated by LOFAR at 150 MHz. The pipeline uses task-based parallelism to image, detect sources, and perform Faraday Tomography across the entire LOFAR sky. The project thereby provides a unique opportunity to contribute to the technological development of the SKA telescope, while simultaneously enabling cutting-edge scientific results. In this paper, we provide an update on current efforts to develop a science pipeline that can enable tight constraints on the magnetised large-scale structure of the Universe.Comment: Published in Galaxies, as part of a Special Issue on The Power of Faraday Tomograph

Big Universe, Big Data: Machine Learning and Image Analysis for Astronomy

Astrophysics and cosmology are rich with data. The advent of wide-area digital cameras on large aperture telescopes has led to ever more ambitious surveys of the sky. Data volumes of entire surveys a decade ago can now be acquired in a single night and real-time analysis is often desired. Thus, modern astronomy requires big data know-how, in particular it demands highly efficient machine learning and image analysis algorithms. But scalability is not the only challenge: Astronomy applications touch several current machine learning research questions, such as learning from biased data and dealing with label and measurement noise. We argue that this makes astronomy a great domain for computer science research, as it pushes the boundaries of data analysis. In the following, we will present this exciting application area for data scientists. We will focus on exemplary results, discuss main challenges, and highlight some recent methodological advancements in machine learning and image analysis triggered by astronomical applications

A novel astronomical application for formation flying small satellites

OLFAR, Orbiting Low Frequency Antennas for Radio Astronomy, will be a space mission to observe the universe frequencies below 30 MHz, as it was never done before with an orbiting telescope. Because of the ionospheric scintillations below 30 MHz and the opaqueness of the ionosphere below 15 MHz, a space mission is the only opportunity for this as yet unexplored frequency range in radio astronomy. The frequency band is scientifically very interesting for exploring the early cosmos at high hydrogen redshifts, the so-called dark-ages and the epoch of reionization, the discovery of planetary and solar bursts in other solar systems, for obtaining a tomographic view of space weather, ultra-high energy cosmic rays and for many other astronomical areas of interest. Because of the low observing frequency the aperture size of the instrument must be in the order of 100 km. This requires a distributed space mission which is proposed to be implemented using formation flying of small satellites. The individual satellites are broken down in five major subsystems: the spacecraft bus, the antenna design, the frontend, backend and data transport. One of the largest challenges is the inter-satellite communication. In this paper the concept and design considerations of OLFAR are presented

DALiuGE: A Graph Execution Framework for Harnessing the Astronomical Data Deluge

The Data Activated Liu Graph Engine - DALiuGE - is an execution framework for processing large astronomical datasets at a scale required by the Square Kilometre Array Phase 1 (SKA1). It includes an interface for expressing complex data reduction pipelines consisting of both data sets and algorithmic components and an implementation run-time to execute such pipelines on distributed resources. By mapping the logical view of a pipeline to its physical realisation, DALiuGE separates the concerns of multiple stakeholders, allowing them to collectively optimise large-scale data processing solutions in a coherent manner. The execution in DALiuGE is data-activated, where each individual data item autonomously triggers the processing on itself. Such decentralisation also makes the execution framework very scalable and flexible, supporting pipeline sizes ranging from less than ten tasks running on a laptop to tens of millions of concurrent tasks on the second fastest supercomputer in the world. DALiuGE has been used in production for reducing interferometry data sets from the Karl E. Jansky Very Large Array and the Mingantu Ultrawide Spectral Radioheliograph; and is being developed as the execution framework prototype for the Science Data Processor (SDP) consortium of the Square Kilometre Array (SKA) telescope. This paper presents a technical overview of DALiuGE and discusses case studies from the CHILES and MUSER projects that use DALiuGE to execute production pipelines. In a companion paper, we provide in-depth analysis of DALiuGE's scalability to very large numbers of tasks on two supercomputing facilities.Comment: 31 pages, 12 figures, currently under review by Astronomy and Computin