Towards a semantic and statistical selection of association rules

The increasing growth of databases raises an urgent need for more accurate methods to better understand the stored data. In this scope, association rules were extensively used for the analysis and the comprehension of huge amounts of data. However, the number of generated rules is too large to be efficiently analyzed and explored in any further process. Association rules selection is a classical topic to address this issue, yet, new innovated approaches are required in order to provide help to decision makers. Hence, many interesting- ness measures have been defined to statistically evaluate and filter the association rules. However, these measures present two major problems. On the one hand, they do not allow eliminating irrelevant rules, on the other hand, their abun- dance leads to the heterogeneity of the evaluation results which leads to confusion in decision making. In this paper, we propose a two-winged approach to select statistically in- teresting and semantically incomparable rules. Our statis- tical selection helps discovering interesting association rules without favoring or excluding any measure. The semantic comparability helps to decide if the considered association rules are semantically related i.e comparable. The outcomes of our experiments on real datasets show promising results in terms of reduction in the number of rules

Virtual Astronomy, Information Technology, and the New Scientific Methodology

All sciences, including astronomy, are now entering the era of information abundance. The exponentially increasing volume and complexity of modern data sets promises to transform the scientific practice, but also poses a number of common technological challenges. The Virtual Observatory concept is the astronomical community's response to these challenges: it aims to harness the progress in information technology in the service of astronomy, and at the same time provide a valuable testbed for information technology and applied computer science. Challenges broadly fall into two categories: data handling (or "data farming"), including issues such as archives, intelligent storage, databases, interoperability, fast networks, etc., and data mining, data understanding, and knowledge discovery, which include issues such as automated clustering and classification, multivariate correlation searches, pattern recognition, visualization in highly hyperdimensional parameter spaces, etc., as well as various applications of machine learning in these contexts. Such techniques are forming a methodological foundation for science with massive and complex data sets in general, and are likely to have a much broather impact on the modern society, commerce, information economy, security, etc. There is a powerful emerging synergy between the computationally enabled science and the science-driven computing, which will drive the progress in science, scholarship, and many other venues in the 21st century

Data Driven Discovery in Astrophysics

We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data rates, and an ever-increasing complexity, now entering the Petascale regime. Telescopes and observatories from both ground and space, covering a full range of wavelengths, feed the data via processing pipelines into dedicated archives, where they can be accessed for scientific analysis. Most of the large archives are connected through the Virtual Observatory framework, that provides interoperability standards and services, and effectively constitutes a global data grid of astronomy. Making discoveries in this overabundance of data requires applications of novel, machine learning tools. We describe some of the recent examples of such applications.Comment: Keynote talk in the proceedings of ESA-ESRIN Conference: Big Data from Space 2014, Frascati, Italy, November 12-14, 2014, 8 pages, 2 figure

Proceedings of the 2011 New York Workshop on Computer, Earth and Space Science

The purpose of the New York Workshop on Computer, Earth and Space Sciences is to bring together the New York area's finest Astronomers, Statisticians, Computer Scientists, Space and Earth Scientists to explore potential synergies between their respective fields. The 2011 edition (CESS2011) was a great success, and we would like to thank all of the presenters and participants for attending. This year was also special as it included authors from the upcoming book titled "Advances in Machine Learning and Data Mining for Astronomy". Over two days, the latest advanced techniques used to analyze the vast amounts of information now available for the understanding of our universe and our planet were presented. These proceedings attempt to provide a small window into what the current state of research is in this vast interdisciplinary field and we'd like to thank the speakers who spent the time to contribute to this volume.Comment: Author lists modified. 82 pages. Workshop Proceedings from CESS 2011 in New York City, Goddard Institute for Space Studie

Search for high-amplitude Delta Scuti and RR Lyrae stars in Sloan Digital Sky Survey Stripe 82 using principal component analysis

We propose a robust principal component analysis (PCA) framework for the exploitation of multi-band photometric measurements in large surveys. Period search results are improved using the time series of the first principal component due to its optimized signal-to-noise ratio.The presence of correlated excess variations in the multivariate time series enables the detection of weaker variability. Furthermore, the direction of the largest variance differs for certain types of variable stars. This can be used as an efficient attribute for classification. The application of the method to a subsample of Sloan Digital Sky Survey Stripe 82 data yielded 132 high-amplitude Delta Scuti variables. We found also 129 new RR Lyrae variables, complementary to the catalogue of Sesar et al., 2010, extending the halo area mapped by Stripe 82 RR Lyrae stars towards the Galactic bulge. The sample comprises also 25 multiperiodic or Blazhko RR Lyrae stars.Comment: 23 pages, 17 figure

ASIME 2018 White Paper. In-Space Utilisation of Asteroids: Asteroid Composition -- Answers to Questions from the Asteroid Miners

In keeping with the Luxembourg government's initiative to support the future use of space resources, ASIME 2018 was held in Belval, Luxembourg on April 16-17, 2018. The goal of ASIME 2018: Asteroid Intersections with Mine Engineering, was to focus on asteroid composition for advancing the asteroid in-space resource utilisation domain. What do we know about asteroid composition from remote-sensing observations? What are the potential caveats in the interpretation of Earth-based spectral observations? What are the next steps to improve our knowledge on asteroid composition by means of ground-based and space-based observations and asteroid rendez-vous and sample return missions? How can asteroid mining companies use this knowledge? ASIME 2018 was a two-day workshop of almost 70 scientists and engineers in the context of the engineering needs of space missions with in-space asteroid utilisation. The 21 Questions from the asteroid mining companies were sorted into the four asteroid science themes: 1) Potential Targets, 2) Asteroid-Meteorite Links, 3) In-Situ Measurements and 4) Laboratory Measurements. The Answers to those Questions were provided by the scientists with their conference presentations and collected by A. Graps or edited directly into an open-access collaborative Google document or inserted by A. Graps using additional reference materials. During the ASIME 2018, first day and second day Wrap-Ups, the answers to the questions were discussed further. New readers to the asteroid mining topic may find the Conversation boxes and the Mission Design discussions especially interesting.Comment: Outcome from the ASIME 2018: Asteroid Intersections with Mine Engineering, Luxembourg. April 16-17, 2018. 65 Pages. arXiv admin note: substantial text overlap with arXiv:1612.0070

Multi-Level Visual Alphabets

A central debate in visual perception theory is the argument for indirect versus direct perception; i.e., the use of intermediate, abstract, and hierarchical representations versus direct semantic interpretation of images through interaction with the outside world. We present a content-based representation that combines both approaches. The previously developed Visual Alphabet method is extended with a hierarchy of representations, each level feeding into the next one, but based on features that are not abstract but directly relevant to the task at hand. Explorative benchmark experiments are carried out on face images to investigate and explain the impact of the key parameters such as pattern size, number of prototypes, and distance measures used. Results show that adding an additional middle layer improves results, by encoding the spatial co-occurrence of lower-level pattern prototypes

Collaboration in an Open Data eScience: A Case Study of Sloan Digital Sky Survey

Current science and technology has produced more and more publically accessible scientific data. However, little is known about how the open data trend impacts a scientific community, specifically in terms of its collaboration behaviors. This paper aims to enhance our understanding of the dynamics of scientific collaboration in the open data eScience environment via a case study of co-author networks of an active and highly cited open data project, called Sloan Digital Sky Survey. We visualized the co-authoring networks and measured their properties over time at three levels: author, institution, and country levels. We compared these measurements to a random network model and also compared results across the three levels. The study found that 1) the collaboration networks of the SDSS community transformed from random networks to small-world networks; 2) the number of author-level collaboration instances has not changed much over time, while the number of collaboration instances at the other two levels has increased over time; 3) pairwise institutional collaboration become common in recent years. The open data trend may have both positive and negative impacts on scientific collaboration.Comment: iConference 201