Mining Knowledge in Astrophysical Massive Data Sets

Modern scientific data mainly consist of huge datasets gathered by a very large number of techniques and stored in very diversified and often incompatible data repositories. More in general, in the e-science environment, it is considered as a critical and urgent requirement to integrate services across distributed, heterogeneous, dynamic "virtual organizations" formed by different resources within a single enterprise. In the last decade, Astronomy has become an immensely data rich field due to the evolution of detectors (plates to digital to mosaics), telescopes and space instruments. The Virtual Observatory approach consists into the federation under common standards of all astronomical archives available worldwide, as well as data analysis, data mining and data exploration applications. The main drive behind such effort being that once the infrastructure will be completed, it will allow a new type of multi-wavelength, multi-epoch science which can only be barely imagined. Data Mining, or Knowledge Discovery in Databases, while being the main methodology to extract the scientific information contained in such MDS (Massive Data Sets), poses crucial problems since it has to orchestrate complex problems posed by transparent access to different computing environments, scalability of algorithms, reusability of resources, etc. In the present paper we summarize the present status of the MDS in the Virtual Observatory and what is currently done and planned to bring advanced Data Mining methodologies in the case of the DAME (DAta Mining & Exploration) project.Comment: Pages 845-849 1rs International Conference on Frontiers in Diagnostics Technologie

Understanding Institutions: A Multi-Dimensional Approach

With the rise of nativist policies throughout the world, the growing dangers posed by climate change and rising income inequality, and ever-increasing threats to the rule of law, many turn to the institutions of democracy to achieve desired policy goals. Indeed, if one seeks to address climate change, preserve the rule of law, or reduce income inequality, functioning institutions are needed to further such objectives. But the ability to leverage institutions to achieve legal and policy goals presupposes a common understanding of institutions as well as an appreciation for the ways in which they can and may function. Traditional comparative institutional analysis uses this functional understanding to identify which institutional setting—typically the political process, the markets, or the courts—is the preferred means of achieving one’s chosen legal or policy goals. This Article argues that merely differentiating between these institutional settings is insufficient to conduct a meaningful comparative analysis. Such a narrow view of institutional settings, what I will call institutional systems and the institutions they contain, leaves much to be desired, particularly as the scale and complexity of both problems and proposed solutions continue to grow. Indeed, this monolithic, one-dimensional view of institutions is ill-equipped to address the scale and scope of contemporary, collective-action problems. This Article develops an approach to comparative institutional analysis that recognizes the rich, multi-dimensional aspects of not only the characteristics of institutions but also the problems institutions are asked to solve. By embracing a robust and comprehensive view of institutions, this new approach to comparative institutional analysis offers a more meaningful and informative foundation upon which to pursue solutions to the complex societal problems of today and those that will emerge in the future

Astroinformatics, data mining and the future of astronomical research

Astronomy, as many other scientific disciplines, is facing a true data deluge which is bound to change both the praxis and the methodology of every day research work. The emerging field of astroinformatics, while on the one end appears crucial to face the technological challenges, on the other is opening new exciting perspectives for new astronomical discoveries through the implementation of advanced data mining procedures. The complexity of astronomical data and the variety of scientific problems, however, call for innovative algorithms and methods as well as for an extreme usage of ICT technologies.Comment: To appear in the Proceedings of the 2-nd International Conference on Frontiers on diagnostic technologie

Statistical analysis of the trigger algorithm for the NEMO project

We discuss the performances of a trigger implemented for the planned neutrino telescope NEMO. This trigger seems capable to discriminate between the signal and the strong background introduced by atmospheric muons and by the beta decay of the K-40 nuclei present in the water. The performances of the trigger, as evaluated on simulated data are analyzed in detail.Comment: Published in the Proceedings of the "I Workshop of Astronomy and Astrophysics for Students", Eds. N.R. Napolitano & M. Paolillo, Naples, 19-20 April 2006 (astro-ph/0701577

Automated physical classification in the SDSS DR10. A catalogue of candidate Quasars

We discuss whether modern machine learning methods can be used to characterize the physical nature of the large number of objects sampled by the modern multi-band digital surveys. In particular, we applied the MLPQNA (Multi Layer Perceptron with Quasi Newton Algorithm) method to the optical data of the Sloan Digital Sky Survey - Data Release 10, investigating whether photometric data alone suffice to disentangle different classes of objects as they are defined in the SDSS spectroscopic classification. We discuss three groups of classification problems: (i) the simultaneous classification of galaxies, quasars and stars; (ii) the separation of stars from quasars; (iii) the separation of galaxies with normal spectral energy distribution from those with peculiar spectra, such as starburst or starforming galaxies and AGN. While confirming the difficulty of disentangling AGN from normal galaxies on a photometric basis only, MLPQNA proved to be quite effective in the three-class separation. In disentangling quasars from stars and galaxies, our method achieved an overall efficiency of 91.31% and a QSO class purity of ~95%. The resulting catalogue of candidate quasars/AGNs consists of ~3.6 million objects, of which about half a million are also flagged as robust candidates, and will be made available on CDS VizieR facility.Comment: Accepted for publication by MNRAS, 13 pages, 6 figure

Data-Rich Astronomy: Mining Sky Surveys with PhotoRApToR

In the last decade a new generation of telescopes and sensors has allowed the production of a very large amount of data and astronomy has become a data-rich science. New automatic methods largely based on machine learning are needed to cope with such data tsunami. We present some results in the fields of photometric redshifts and galaxy classification, obtained using the MLPQNA algorithm available in the DAMEWARE (Data Mining and Web Application Resource) for the SDSS galaxies (DR9 and DR10). We present PhotoRApToR (Photometric Research Application To Redshift): a Java based desktop application capable to solve regression and classification problems and specialized for photo-z estimation.Comment: proceedings of the IAU Symposium, Vol. 306, Cambridge University Pres

Implementation of the trigger algorithm for the NEMO project

We describe the implementation of trigger algorithm specifically tailored on the characteristics of the neutrino telescope NEMO. Extensive testing against realistic simulations shows that, by making use of the uncorrelated nature of the noise produced mainly by the decay of K-40 beta-decay, this trigger is capable to discriminate among different types of muonic events.Comment: Published in the Proceedings of the "I Workshop of Astronomy and Astrophysics for Students", Eds. N.R. Napolitano & M. Paolillo, Naples, 19-20 April 2006 (astro-ph/0701577

Food demand elasticities in Argentina, Paraguay and Bolivia. Econometric estimation from household surveys

This paper presents the methodology and estimation of food demand elasticities for Argentina, Paraguay and Bolivia using household survey data. The paper reviews the theoretical and empirical approach behind the applied food demand estimation. The empirical approach consists in the estimation of a censored corrected LinQuad incomplete demand system using microdata from national household surveys. The empirical implementation and results are consistent with the state of the art in applied demand estimations using censored cross sectional data.Fil: Lema, Daniel. Instituto Nacional de Tecnología Agropecuaria (INTA). Instituto de Economía y Sociología; Argentina.Fil: Brescia, Víctor. Instituto Nacional de Tecnología Agropecuaria (INTA). Instituto de Economía y Sociología; Argentina.Fil: Berges, Miriam. Universidad Nacional de Mar del Plata. Facultad de Ciencias Económicas y Sociales; Argentina.Fil: Casellas, Karina. Universidad Nacional de Mar del Plata. Facultad de Ciencias Económicas y Sociales; Argentina

Photometric redshifts with Quasi Newton Algorithm (MLPQNA). Results in the PHAT1 contest

Context. Since the advent of modern multiband digital sky surveys, photometric redshifts (photo-z's) have become relevant if not crucial to many fields of observational cosmology, from the characterization of cosmic structures, to weak and strong lensing. Aims. We describe an application to an astrophysical context, namely the evaluation of photometric redshifts, of MLPQNA, a machine learning method based on Quasi Newton Algorithm. Methods. Theoretical methods for photo-z's evaluation are based on the interpolation of a priori knowledge (spectroscopic redshifts or SED templates) and represent an ideal comparison ground for neural networks based methods. The MultiLayer Perceptron with Quasi Newton learning rule (MLPQNA) described here is a computing effective implementation of Neural Networks for the first time exploited to solve regression problems in the astrophysical context and is offered to the community through the DAMEWARE (DAta Mining & ExplorationWeb Application REsource) infrastructure. Results. The PHAT contest (Hildebrandt et al. 2010) provides a standard dataset to test old and new methods for photometric redshift evaluation and with a set of statistical indicators which allow a straightforward comparison among different methods. The MLPQNA model has been applied on the whole PHAT1 dataset of 1984 objects after an optimization of the model performed by using as training set the 515 available spectroscopic redshifts. When applied to the PHAT1 dataset, MLPQNA obtains the best bias accuracy (0.0006) and very competitive accuracies in terms of scatter (0.056) and outlier percentage (16.3%), scoring as the second most effective empirical method among those which have so far participated to the contest. MLPQNA shows better generalization capabilities than most other empirical methods especially in presence of underpopulated regions of the Knowledge Base.Comment: Accepted for publication in Astronomy & Astrophysics; 9 pages, 2 figure

PhotoRaptor - Photometric Research Application To Redshifts

Due to the necessity to evaluate photo-z for a variety of huge sky survey data sets, it seemed important to provide the astronomical community with an instrument able to fill this gap. Besides the problem of moving massive data sets over the network, another critical point is that a great part of astronomical data is stored in private archives that are not fully accessible on line. So, in order to evaluate photo-z it is needed a desktop application that can be downloaded and used by everyone locally, i.e. on his own personal computer or more in general within the local intranet hosted by a data center. The name chosen for the application is PhotoRApToR, i.e. Photometric Research Application To Redshift (Cavuoti et al. 2015, 2014; Brescia 2014b). It embeds a machine learning algorithm and special tools dedicated to preand post-processing data. The ML model is the MLPQNA (Multi Layer Perceptron trained by the Quasi Newton Algorithm), which has been revealed particularly powerful for the photo-z calculation on the base of a spectroscopic sample (Cavuoti et al. 2012; Brescia et al. 2013, 2014a; Biviano et al. 2013). The PhotoRApToR program package is available, for different platforms, at the official website (http://dame.dsf.unina.it/dame_photoz.html#photoraptor).Comment: User Manual of the PhotoRaptor tool, 54 pages. arXiv admin note: substantial text overlap with arXiv:1501.0650