Improvements on coronal hole detection in SDO/AIA images using supervised classification

We demonstrate the use of machine learning algorithms in combination with segmentation techniques in order to distinguish coronal holes and filaments in SDO/AIA EUV images of the Sun. Based on two coronal hole detection techniques (intensity-based thresholding, SPoCA), we prepared data sets of manually labeled coronal hole and filament channel regions present on the Sun during the time range 2011 - 2013. By mapping the extracted regions from EUV observations onto HMI line-of-sight magnetograms we also include their magnetic characteristics. We computed shape measures from the segmented binary maps as well as first order and second order texture statistics from the segmented regions in the EUV images and magnetograms. These attributes were used for data mining investigations to identify the most performant rule to differentiate between coronal holes and filament channels. We applied several classifiers, namely Support Vector Machine, Linear Support Vector Machine, Decision Tree, and Random Forest and found that all classification rules achieve good results in general, with linear SVM providing the best performances (with a true skill statistic of ~0.90). Additional information from magnetic field data systematically improves the performance across all four classifiers for the SPoCA detection. Since the calculation is inexpensive in computing time, this approach is well suited for applications on real-time data. This study demonstrates how a machine learning approach may help improve upon an unsupervised feature extraction method.Comment: in press for SWS

Investigating the Kinematics of Coronal Mass Ejections with the Automated CORIMP Catalog

Studying coronal mass ejections (CMEs) in coronagraph data can be challenging due to their diffuse structure and transient nature, compounded by the variations in their dynamics, morphology, and frequency of occurrence. The large amounts of data available from missions like the Solar and Heliospheric Observatory (SOHO) make manual cataloging of CMEs tedious and prone to human error, and so a robust method of detection and analysis is required and often preferred. A new coronal image processing catalog called CORIMP has been developed in an effort to achieve this, through the implementation of a dynamic background separation technique and multiscale edge detection. These algorithms together isolate and characterise CME structure in the field-of-view of the Large Angle Spectrometric Coronagraph (LASCO) onboard SOHO. CORIMP also applies a Savitzky-Golay filter, along with quadratic and linear fits, to the height-time measurements for better revealing the true CME speed and acceleration profiles across the plane-of-sky. Here we present a sample of new results from the CORIMP CME catalog, and directly compare them with the other automated catalogs of Computer Aided CME Tracking (CACTus) and Solar Eruptive Events Detection System (SEEDS), as well as the manual CME catalog at the Coordinated Data Analysis Workshop (CDAW) Data Center and a previously published study of the sample events. We further investigate a form of unsupervised machine learning by using a k-means clustering algorithm to distinguish detections of multiple CMEs that occur close together in space and time. While challenges still exist, this investigation and comparison of results demonstrates the reliability and robustness of the CORIMP catalog, proving its effectiveness at detecting and tracking CMEs throughout the LASCO dataset.Comment: 23 pages, 11 figures, 1 tabl

Image patch analysis of sunspots and active regions. II. Clustering via matrix factorization

Separating active regions that are quiet from potentially eruptive ones is a key issue in Space Weather applications. Traditional classification schemes such as Mount Wilson and McIntosh have been effective in relating an active region large scale magnetic configuration to its ability to produce eruptive events. However, their qualitative nature prevents systematic studies of an active region's evolution for example. We introduce a new clustering of active regions that is based on the local geometry observed in Line of Sight magnetogram and continuum images. We use a reduced-dimension representation of an active region that is obtained by factoring the corresponding data matrix comprised of local image patches. Two factorizations can be compared via the definition of appropriate metrics on the resulting factors. The distances obtained from these metrics are then used to cluster the active regions. We find that these metrics result in natural clusterings of active regions. The clusterings are related to large scale descriptors of an active region such as its size, its local magnetic field distribution, and its complexity as measured by the Mount Wilson classification scheme. We also find that including data focused on the neutral line of an active region can result in an increased correspondence between our clustering results and other active region descriptors such as the Mount Wilson classifications and the $R$ value. We provide some recommendations for which metrics, matrix factorization techniques, and regions of interest to use to study active regions.Comment: Accepted for publication in the Journal of Space Weather and Space Climate (SWSC). 33 pages, 12 figure

The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.Comment: under revie

State-of-the-art on research and applications of machine learning in the building life cycle

Fueled by big data, powerful and affordable computing resources, and advanced algorithms, machine learning has been explored and applied to buildings research for the past decades and has demonstrated its potential to enhance building performance. This study systematically surveyed how machine learning has been applied at different stages of building life cycle. By conducting a literature search on the Web of Knowledge platform, we found 9579 papers in this field and selected 153 papers for an in-depth review. The number of published papers is increasing year by year, with a focus on building design, operation, and control. However, no study was found using machine learning in building commissioning. There are successful pilot studies on fault detection and diagnosis of HVAC equipment and systems, load prediction, energy baseline estimate, load shape clustering, occupancy prediction, and learning occupant behaviors and energy use patterns. None of the existing studies were adopted broadly by the building industry, due to common challenges including (1) lack of large scale labeled data to train and validate the model, (2) lack of model transferability, which limits a model trained with one data-rich building to be used in another building with limited data, (3) lack of strong justification of costs and benefits of deploying machine learning, and (4) the performance might not be reliable and robust for the stated goals, as the method might work for some buildings but could not be generalized to others. Findings from the study can inform future machine learning research to improve occupant comfort, energy efficiency, demand flexibility, and resilience of buildings, as well as to inspire young researchers in the field to explore multidisciplinary approaches that integrate building science, computing science, data science, and social science

Prediction of Extreme Ultraviolet Variability Experiment (EVE)/Extreme Ultraviolet Spectro-Photometer (ESP) Irradiance from Solar Dynamics Observatory (SDO)/Atmospheric Imaging Assembly (AIA) Images Using Fuzzy Image Processing and Machine Learning

YesThe cadence and resolution of solar images have been increasing dramatically with the launch of new spacecraft such as STEREO and SDO. This increase in data volume provides new opportunities for solar researchers, but the efficient processing and analysis of these data create new challenges. We introduce a fuzzy-based solar feature-detection system in this article. The proposed system processes SDO/AIA images using fuzzy rules to detect coronal holes and active regions. This system is fast and it can handle different size images. It is tested on six months of solar data (1 October 2010 to 31 March 2011) to generate filling factors (ratio of area of solar feature to area of rest of the solar disc) for active regions and coronal holes. These filling factors are then compared to SDO/EVE/ESP irradiance measurements. The correlation between active-region filling factors and irradiance measurements is found to be very high, which has encouraged us to design a time-series prediction system using Radial Basis Function Networks to predict ESP irradiance measurements from our generated filling factors