Meta learning of bounds on the Bayes classifier error

Meta learning uses information from base learners (e.g. classifiers or estimators) as well as information about the learning problem to improve upon the performance of a single base learner. For example, the Bayes error rate of a given feature space, if known, can be used to aid in choosing a classifier, as well as in feature selection and model selection for the base classifiers and the meta classifier. Recent work in the field of f-divergence functional estimation has led to the development of simple and rapidly converging estimators that can be used to estimate various bounds on the Bayes error. We estimate multiple bounds on the Bayes error using an estimator that applies meta learning to slowly converging plug-in estimators to obtain the parametric convergence rate. We compare the estimated bounds empirically on simulated data and then estimate the tighter bounds on features extracted from an image patch analysis of sunspot continuum and magnetogram images.Comment: 6 pages, 3 figures, to appear in proceedings of 2015 IEEE Signal Processing and SP Education Worksho

Image patch analysis of sunspots and active regions. II. Clustering via matrix factorization

Separating active regions that are quiet from potentially eruptive ones is a key issue in Space Weather applications. Traditional classification schemes such as Mount Wilson and McIntosh have been effective in relating an active region large scale magnetic configuration to its ability to produce eruptive events. However, their qualitative nature prevents systematic studies of an active region's evolution for example. We introduce a new clustering of active regions that is based on the local geometry observed in Line of Sight magnetogram and continuum images. We use a reduced-dimension representation of an active region that is obtained by factoring the corresponding data matrix comprised of local image patches. Two factorizations can be compared via the definition of appropriate metrics on the resulting factors. The distances obtained from these metrics are then used to cluster the active regions. We find that these metrics result in natural clusterings of active regions. The clusterings are related to large scale descriptors of an active region such as its size, its local magnetic field distribution, and its complexity as measured by the Mount Wilson classification scheme. We also find that including data focused on the neutral line of an active region can result in an increased correspondence between our clustering results and other active region descriptors such as the Mount Wilson classifications and the $R$ value. We provide some recommendations for which metrics, matrix factorization techniques, and regions of interest to use to study active regions.Comment: Accepted for publication in the Journal of Space Weather and Space Climate (SWSC). 33 pages, 12 figure

Big Universe, Big Data: Machine Learning and Image Analysis for Astronomy

Astrophysics and cosmology are rich with data. The advent of wide-area digital cameras on large aperture telescopes has led to ever more ambitious surveys of the sky. Data volumes of entire surveys a decade ago can now be acquired in a single night and real-time analysis is often desired. Thus, modern astronomy requires big data know-how, in particular it demands highly efficient machine learning and image analysis algorithms. But scalability is not the only challenge: Astronomy applications touch several current machine learning research questions, such as learning from biased data and dealing with label and measurement noise. We argue that this makes astronomy a great domain for computer science research, as it pushes the boundaries of data analysis. In the following, we will present this exciting application area for data scientists. We will focus on exemplary results, discuss main challenges, and highlight some recent methodological advancements in machine learning and image analysis triggered by astronomical applications

Framework for near real time feature detection from the atmospheric imaging assembly images of the solar dynamics observatory

The study of the variability of the solar corona and the monitoring of its traditional regions (Coronal Holes, Quiet Sun and Active Regions) are of great importance in astrophysics as well as in view of the Space Weather applications. The Atmospheric Imaging Assembly (AIA) of the Solar Dynamics Observatory (SDO) provides high resolution images of the sun imaged at different wavelengths at a rate of approximately one every 10 seconds, a great resource for solar monitoring . Today, the process of identifying features and estimating their properties is applied manually in an iterative fashion to verify the detection results. We introduce a complete, automated image-processing pipeline, starting with raw data and ending with quantitative data of high level feature parameters. We implement two multichannel unsupervised algorithms that automatically segments EUV AIA solar images into Coronal Holes, Quiet Sun and Active Regions in near real time. We also develop a method of post processing to deal with fragments in a segmented image by spatial validity based compact clustering. The segmentation results are consistent with well-known algorithms and databases. The parameters extracted from the segments like area closely follow the solar activity pattern. Moreover, the methods developed within the proposed framework are generic enough to allow the study of any solar feature (e.g. Coronal Bright points) provided that the feature can be deduced from AIA images

Automatic Short-Term Solar Flare Prediction Using Machine Learning and Sunspot Associations.

YesIn this paper, a machine-learning-based system that could provide automated short-term solar flare prediction is presented. This system accepts two sets of inputs: McIntosh classification of sunspot groups and solar cycle data. In order to establish a correlation between solar flares and sunspot groups, the system explores the publicly available solar catalogues from the National Geophysical Data Center to associate sunspots with their corresponding flares based on their timing and NOAA numbers. The McIntosh classification for every relevant sunspot is extracted and converted to a numerical format that is suitable for machine learning algorithms. Using this system we aim to predict whether a certain sunspot class at a certain time is likely to produce a significant flare within six hours time and if so whether this flare is going to be an X or M flare. Machine learning algorithms such as Cascade-Correlation Neural Networks (CCNNs), Support Vector Machines (SVMs) and Radial Basis Function Networks (RBFN) are optimised and then compared to determine the learning algorithm that would provide the best prediction performance. It is concluded that SVMs provide the best performance for predicting whether a McIntosh classified sunspot group is going to flare or not but CCNNs are more capable of predicting the class of the flare to erupt. A hybrid system that combines a SVM and a CCNN is suggested for future use.EPSR

Fully Automated Sunspot Detection and Classification Using SDO HMI Imagery in MATLAB

An automatic sunspot detection and classification method is developed combining HMII and HMIM imagery procured from the Solar Dynamics Observatory. Iterative global thresholding methods are employed for detecting sunspots. Groups are selected based on heliographic distance between sunspots via area-based grouping lengths. Classifications are applied through logical operators adhering to the standard McIntosh classification system. Calculated sunspot parameters and classifications are validated in three way comparisons between code output, Holloman AFB and the Space Weather Prediction Center. Accuracy is achieved within the margin of difference between Holloman and SWPC reports for sunspot area, number of groups, number of spots, and McIntosh classification using data spanning 6 July 2012 to 29 June 2013: SWPC/Holloman (33.38%,57.48%,87.67%), SWPC/SDO (20.22%,51.25%,83.80%), and SDO/Holloman (24.54%,50.91%,80.65%). The automatic classification system is used to evaluate bias inherent in Holloman classification methods. Parameters are altered to reach optimal match percentages with Holloman, indicating differences between computed parameter values and hand-calculated counterparts. Group length cutoffs are shown to differ by 2.5°, eccentricity is quantified at 0.8, and penumbra length cutoffs are shown to exceed differences of 1.4° from McIntosh values