258 research outputs found
Self-Organizing Neural Network for Optimum Supervised Learning
This work introduces a new algorithm called the Self-Organizing Neural Network (SONN), and demonstrates its use in a system identification task. The algorithm constructs a network, chooses the neuron functions, and adjusts the weights. Here, it is compared to the Back-Propagation algorithm in the identification of the chaotic time series. The results show that SONN constructs a simpler, more accurate model, requiring less training data and epochs. The algorithm can also be applied as a classifier
An Evolutionary Algorithmic Approach to Learning a Bayesian Network from Complete Data
Discovering relationships between variables is crucial for interpreting data from large databases. Relationships between variables can be modeled using a Bayesian network. The challenge of learning a Bayesian network from a complete dataset grows exponentially with the number of variables in the database and the number of states in each variable. It therefore becomes important to identify promising heuristics for exploring the space of possible networks. This paper utilizes an evolutionary algorithmic approach, Particle Swarm Optimization (PSO) to perform this search. A fundamental problem with a search for a Bayesian network is that of handling cyclic networks, which are not allowed. This paper explores the PSO approach, handling cyclic networks in two different ways. Results of network extraction for the well-studied ALARM network are presented for PSO simulations where cycles are broken heuristically at each step of the optimization and where networks with cycles are allowed to exist as candidate solutions, but are assigned a poor fitness. The results of the two approaches are compared and it is found that allowing cyclic networks to exist in the particle swarm of candidate solutions can dramatically reduce the number of objective function evaluations required to converge to a target fitness value
Bayesian belief networks : from construction to inference
In het dagelijks leven is het redeneren met onzekerheden gebruikelijker dan het redeneren zonder. Bayesiaanse belief netwerken bieden een wiskundig correct formalisme om onzekerheid te representeren en op efficiëte wijze mee te redeneren.
Een Bayesiaanse belief netwerk bestaat uit twee delen.
Ten eerste bestaat een belief netwerk uit een een gerichte graaf zonder lussen: de netwerkstructuur. Voor elke variabele waarmee we willen redeneren is er een knoop in de graaf. We zullen de termen knoop en variabele dan ook door elkaar gebruiken.
Figuur 0.1 laat een eenvoudig belief netwerk zien voor een klein medisch domein met daarin de leeftijd van een patient (a), de behoefte aan een bril (g), of het zicht beter wordt als de patient knippert (v) en of de patient klachten heeft over zijn zicht (s). Als er een directe afhankelijkheid tussen twee knopen is, dan zijn deze knopen
verbonden met een pijl. Intuitief geeft de richting van de pijl een causale invloed aan. Bijvoorbeeld in Figuur 0.1 geeft de pijl van a naar g weer dat de leeftijd een indicatie is dat de patient een bril nodig heeft
Model Selection for Stochastic Block Models
As a flexible representation for complex systems, networks (graphs) model entities and their interactions as nodes and edges. In many real-world networks, nodes divide naturally into functional communities, where nodes in the same group connect to the rest of the network in similar ways. Discovering such communities is an important part of modeling networks, as community structure offers clues to the processes which generated the graph. The stochastic block model is a popular network model based on community structures. It splits nodes into blocks, within which all nodes are stochastically equivalent in terms of how they connect to the rest of the network. As a generative model, it has a well-defined likelihood function with consistent parameter estimates. It is also highly flexible, capable of modeling a wide variety of community structures, including degree specific and overlapping communities. Performance of different block models vary under different scenarios. Picking the right model is crucial for successful network modeling. A good model choice should balance the trade-off between complexity and fit. The task of model selection is to automatically choose such a model given the data and the inference task. As a problem of wide interest, numerous statistical model selection techniques have been developed for classic independent data. Unfortunately, it has been a common mistake to use these techniques in block models without rigorous examinations of their derivations, ignoring the fact that some of the fundamental assumptions has been violated by moving into the domain of relational data sets such as networks. In this dissertation, I thoroughly exam the literature of statistical model selection techniques, including both Frequentist and Bayesian approaches. My goal is to develop principled statistical model selection criteria for block models by adapting classic methods for network data. I do this by running bootstrapping simulations with an efficient algorithm, and correcting classic model selection theories for block models based on the simulation data. The new model selection methods are verified by both synthetic and real world data sets
Learning Bayesian network structure from massive datasets: The ”sparse candidate” algorithm
Learning Bayesian networks is often cast as an optimization problem, where the computational task is to find a structure that maximizes a statistically motivated score. By and large, existing learning tools address this optimization problem using standard heuristic search techniques. Since the search space is extremely large, such search procedures can spend most of the time examining candidates that are extremely unreasonable. This problem becomes critical when we deal with data sets that are large both in the number of instances, and the number of attributes. In this paper, we introduce an algorithm that achieves faster learning by restricting the search space. This iterative algorithm restricts the parents of each variable to belong to a small subset of candidates. We then search for a network that satisfies these constraints. The learned network is then used for selecting better candidates for the next iteration. We evaluate this algorithm both on synthetic and real-life data. Our results show that it is significantly faster than alternative search procedures without loss of quality in the learned structures.
- …