An empirical comparison of Bayesian network parameter learning algorithms for continuous data streams

We compare three approaches to learning numerical parameters of Bayesian networks from continuous data streams: (1) the EM algorithm applied to all data, (2) the EM algorithm applied to data increments, and (3) the online EM algorithm. Our results show that learning from all data at each step, whenever feasible, leads to the highest parameter accuracy and model classification accuracy. When facing computational limitations, incremental learning approaches are a reasonable alternative. Of these, online EM is reasonably fast, and similar to the incremental EM algorithm in terms of accuracy. For small data sets, incremental EM seems to lead to better accuracy. When the data size gets large, online EM tends to be more accurate. Copyright © 2013, Association for the Advancement of Artificial Intelligence. All rights reserved

THEORETICAL AND PRACTICAL ASPECTS OF DECISION SUPPORT SYSTEMS BASED ON THE PRINCIPLES OF QUERY-BASED DIAGNOSTICS

Diagnosis has been traditionally one of the most successful applications of Bayesian networks. The main bottleneck in applying Bayesian networks to diagnostic problems seems to be model building, which is typically a complex and time consuming task. Query-based diagnostics offers passive, incremental construction of diagnostic models that rest on the interaction between a diagnostician and a computer-based diagnostic system. Every case, passively observed by the system, adds information and, in the long run, leads to construction of a usable model. This approach minimizes knowledge engineering in model building. This dissertation focuses on theoretical and practical aspects of building systems based on the idea of query-based diagnostics. Its main contributions are an investigation of the optimal approach to learning parameters of Bayesian networks from continuous data streams, dealing with structural complexity in building Bayesian networks through removal of the weakest arcs, and a practical evaluation of the idea of query-based diagnostics. One of the main problems of query-based diagnostic systems is dealing with complexity. As data comes in, the models constructed may become too large and too densely connected. I address this problem in two ways. First, I present an empirical comparison of Bayesian network parameter learning algorithms. This study provides the optimal solutions for the system when dealing with continuous data streams. Second, I conduct a series of experiments testing control of the growth of a model by means of removing its weakest arcs. The results show that removing up to 20 percent of the weakest arcs in a network has minimal effect on its classification accuracy, and reduces the amount of memory taken by the clique tree and by this the amount of computation needed to perform inference. An empirical evaluation of query-based diagnostic systems shows that the diagnostic accuracy reaches reasonable levels after merely tens of cases and continues to increase with the number of cases, comparing favorably to state of the art approaches based on learning

One-Class Classification: Taxonomy of Study and Review of Techniques

One-class classification (OCC) algorithms aim to build classification models when the negative class is either absent, poorly sampled or not well defined. This unique situation constrains the learning of efficient classifiers by defining class boundary just with the knowledge of positive class. The OCC problem has been considered and applied under many research themes, such as outlier/novelty detection and concept learning. In this paper we present a unified view of the general problem of OCC by presenting a taxonomy of study for OCC problems, which is based on the availability of training data, algorithms used and the application domains applied. We further delve into each of the categories of the proposed taxonomy and present a comprehensive literature review of the OCC algorithms, techniques and methodologies with a focus on their significance, limitations and applications. We conclude our paper by discussing some open research problems in the field of OCC and present our vision for future research.Comment: 24 pages + 11 pages of references, 8 figure

The Neural Particle Filter

The robust estimation of dynamically changing features, such as the position of prey, is one of the hallmarks of perception. On an abstract, algorithmic level, nonlinear Bayesian filtering, i.e. the estimation of temporally changing signals based on the history of observations, provides a mathematical framework for dynamic perception in real time. Since the general, nonlinear filtering problem is analytically intractable, particle filters are considered among the most powerful approaches to approximating the solution numerically. Yet, these algorithms prevalently rely on importance weights, and thus it remains an unresolved question how the brain could implement such an inference strategy with a neuronal population. Here, we propose the Neural Particle Filter (NPF), a weight-less particle filter that can be interpreted as the neuronal dynamics of a recurrently connected neural network that receives feed-forward input from sensory neurons and represents the posterior probability distribution in terms of samples. Specifically, this algorithm bridges the gap between the computational task of online state estimation and an implementation that allows networks of neurons in the brain to perform nonlinear Bayesian filtering. The model captures not only the properties of temporal and multisensory integration according to Bayesian statistics, but also allows online learning with a maximum likelihood approach. With an example from multisensory integration, we demonstrate that the numerical performance of the model is adequate to account for both filtering and identification problems. Due to the weightless approach, our algorithm alleviates the 'curse of dimensionality' and thus outperforms conventional, weighted particle filters in higher dimensions for a limited number of particles

The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.Comment: under revie