Machine Learning and Data Analysis in Astroinformatics

Astroinformatics is a new discipline at the cross-road of astronomy, advanced statistics and computer science. With next generation sky surveys, space missions and modern instrumentation astronomy will enter the Petascale regime raising the demand for advanced computer science techniques with hard- and software solutions for data management, analysis, efficient automation and knowledge discovery. This tutorial reviews important developments in astroinformatics over the past years and discusses some relevant research questions and concrete problems. The contribution ends with a short review of the special session papers in these proceedings, as well as perspectives and challenges for the near future

Utilizing Astroinformatics to Maximize the Science Return of the Next Generation Virgo Cluster Survey

The Next Generation Virgo Cluster Survey is a 104 square degree survey of the Virgo Cluster, carried out using the MegaPrime camera of the Canada-France-Hawaii telescope, from semesters 2009A-2012A. The survey will provide coverage of this nearby dense environment in the universe to unprecedented depth, providing profound insights into galaxy formation and evolution, including definitive measurements of the properties of galaxies in a dense environment in the local universe, such as the luminosity function. The limiting magnitude of the survey is g_AB = 25.7 (10 sigma point source), and the 2 sigma surface brightness limit is g_AB ~ 29 mag arcsec^-2. The data volume of the survey (approximately 50 terabytes of images), while large by contemporary astronomical standards, is not intractable. This renders the survey amenable to the methods of astroinformatics. The enormous dynamic range of objects, from the giant elliptical galaxy M87 at M(B) = -21.6, to the faintest dwarf ellipticals at M(B) ~ -6, combined with photometry in 5 broad bands (u* g' r' i' z'), and unprecedented depth revealing many previously unseen structures, creates new challenges in object detection and classification. We present results from ongoing work on the survey, including photometric redshifts, Virgo cluster membership, and the implementation of fast data mining algorithms on the infrastructure of the Canadian Astronomy Data Centre, as part of the Canadian Advanced Network for Astronomical Research (CANFAR).Comment: 8 pages, 2 figures. Accepted for the Joint Workshop and Summer School: Astrostatistics and Data Mining in Large Astronomical Databases, La Palma, May 30th - June 3rd 2011. A higher resolution version is available at http://sites.google.com/site/nickballastronomer/publication

Data Deluge in Astrophysics: Photometric Redshifts as a Template Use Case

Astronomy has entered the big data era and Machine Learning based methods have found widespread use in a large variety of astronomical applications. This is demonstrated by the recent huge increase in the number of publications making use of this new approach. The usage of machine learning methods, however is still far from trivial and many problems still need to be solved. Using the evaluation of photometric redshifts as a case study, we outline the main problems and some ongoing efforts to solve them.Comment: 13 pages, 3 figures, Springer's Communications in Computer and Information Science (CCIS), Vol. 82

Astroinformatics, data mining and the future of astronomical research

Astronomy, as many other scientific disciplines, is facing a true data deluge which is bound to change both the praxis and the methodology of every day research work. The emerging field of astroinformatics, while on the one end appears crucial to face the technological challenges, on the other is opening new exciting perspectives for new astronomical discoveries through the implementation of advanced data mining procedures. The complexity of astronomical data and the variety of scientific problems, however, call for innovative algorithms and methods as well as for an extreme usage of ICT technologies.Comment: To appear in the Proceedings of the 2-nd International Conference on Frontiers on diagnostic technologie

Data Driven Discovery in Astrophysics

We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data rates, and an ever-increasing complexity, now entering the Petascale regime. Telescopes and observatories from both ground and space, covering a full range of wavelengths, feed the data via processing pipelines into dedicated archives, where they can be accessed for scientific analysis. Most of the large archives are connected through the Virtual Observatory framework, that provides interoperability standards and services, and effectively constitutes a global data grid of astronomy. Making discoveries in this overabundance of data requires applications of novel, machine learning tools. We describe some of the recent examples of such applications.Comment: Keynote talk in the proceedings of ESA-ESRIN Conference: Big Data from Space 2014, Frascati, Italy, November 12-14, 2014, 8 pages, 2 figure