Boolean Dynamics with Random Couplings

This paper reviews a class of generic dissipative dynamical systems called N-K models. In these models, the dynamics of N elements, defined as Boolean variables, develop step by step, clocked by a discrete time variable. Each of the N Boolean elements at a given time is given a value which depends upon K elements in the previous time step. We review the work of many authors on the behavior of the models, looking particularly at the structure and lengths of their cycles, the sizes of their basins of attraction, and the flow of information through the systems. In the limit of infinite N, there is a phase transition between a chaotic and an ordered phase, with a critical phase in between. We argue that the behavior of this system depends significantly on the topology of the network connections. If the elements are placed upon a lattice with dimension d, the system shows correlations related to the standard percolation or directed percolation phase transition on such a lattice. On the other hand, a very different behavior is seen in the Kauffman net in which all spins are equally likely to be coupled to a given spin. In this situation, coupling loops are mostly suppressed, and the behavior of the system is much more like that of a mean field theory. We also describe possible applications of the models to, for example, genetic networks, cell differentiation, evolution, democracy in social systems and neural networks.Comment: 69 pages, 16 figures, Submitted to Springer Applied Mathematical Sciences Serie

Attraction Basins as Gauges of Robustness against Boundary Conditions in Biological Complex Systems

One fundamental concept in the context of biological systems on which researches have flourished in the past decade is that of the apparent robustness of these systems, i.e., their ability to resist to perturbations or constraints induced by external or boundary elements such as electromagnetic fields acting on neural networks, micro-RNAs acting on genetic networks and even hormone flows acting both on neural and genetic networks. Recent studies have shown the importance of addressing the question of the environmental robustness of biological networks such as neural and genetic networks. In some cases, external regulatory elements can be given a relevant formal representation by assimilating them to or modeling them by boundary conditions. This article presents a generic mathematical approach to understand the influence of boundary elements on the dynamics of regulation networks, considering their attraction basins as gauges of their robustness. The application of this method on a real genetic regulation network will point out a mathematical explanation of a biological phenomenon which has only been observed experimentally until now, namely the necessity of the presence of gibberellin for the flower of the plant Arabidopsis thaliana to develop normally

Neural Networks and Their Applications for the Oil Industry

Neural Networks can be used in many different areas of problems related to Petroleum Exploration and Production. There already exist well defined classes of applications, together with appropriate Neural Networks architectures. Detailed theoretical results allow to monitor and evaluate the results obtained by Neural Networks. Sophisticated applications will certainly require the use of multi-modular architectures

Scaling laws for the attractors of Hopfield networks

Networks of threshold automata are random dynamical systems with a large number of attractors, which J. Hopfield proposed to use as associative memories. We establish the scaling laws relating the maximum number of « useful » attractors and the radius of the attraction basin to the number of automata. A by-product of our analysis is a better choice for thresholds which doubles the performances in terms of the maximum number of « useful » attractors.Les réseaux d'automates à seuil sont des systèmes dynamiques à structure aléatoire semblables aux verres de spins dont J. Hopfield a proposé l'application comme mémoires associatives. Nous établissons les lois d'échelles reliant le nombre maximum d'attracteurs utiles et la distance d'attraction, au nombre des automates du réseau. Notre approche permet aussi un meilleur choix des seuils, ce qui double les performances du réseau en nombre d'attracteurs

Complexity Control Of Image Processing Network Architectures Through Regularization

this paper, we will talk only about OCR applications: while our approach is general to image processing, OCR serves as a perfect test bed for new algorithms, because of the extensive litterature and "universal" data bases (the NIST data base) available. and also of how big the learning set will have to be (i.e. how many data will be needed to match the unknown variables). Thus reducing the complexity of a NN is a means to reduce costs, and also, as we just saw, to increase performances: really a very good deal indeed ! As a consequence of this, a lot of work has been devoted, along the years, to designing networks with carefully controlled complexity. In particular, Time Delay Neural Networks-TDNN- [24], have been used to great success for OCR [12]: a TDNN is an architecture where complexity is controlled through connectivity (connections are local) and weights (which are shared). This dramatically reduces the TDNN complexity as compared to a fully connected architecture: for example the TDNN in [12] has 100 000 connections but only 2 500 weights, while a fully connected NN with 100 000 connections would have that many weights! Finding the optimal architecture though is very much of an art: there does not seem to exist any systematic approach to design the appropriate pattern of connections, local or shared weights. Basically, one tests, by-trial and-error, various choices for receptive field sizes and overlaps), and finally selects that architecture which yields the best performances on a validation set. This process is, of course, extremely time consuming and one could always fear that the final architecture is not optimal, but only the best one among those tested. This problem is often considered as a major weakness of#the NN approach. We need a principled way to d..

Automata networks in computer science: theory and applications

International audienceno abstrac

Local PSOMs and Chebyshev PSOMs -Improving the Parametrised Self-Organizing Maps

Walter JA, Ritter H. Local PSOMs and Chebyshev PSOMs -Improving the Parametrised Self-Organizing Maps. In: Fogelman-Soulie F, ed. 1. ICANN '95, NEURONIMES 95. Vol 1. Paris; 1995: 95-102

Bivariate Negotiations as a Problem of Stochastic Terminal Control

A mathematical model is developed for two players negotiating on two negotiation (operational goal) dimensions. Bivanate utilities are not assumed. Rather at any stage the payoff is expressed as a payoff probability distribution stating the probability of a player obtaining various amounts of each of the two variables. The preferred (optimum) payoff distribution is not fixed but changes in the course of negotiations. The model treats concession making as a problem of stochastic terminal control which can be formulated and solved by dynamic programming to yield normative recommendations as to concession making (control). The model is illustrated by numerical example. The present research generalizes work by Rao and Shakun (Rao, A. G., M. F. Shaxun. 1974. A normative model for negotiations. Management Sci. 20 (10, June).) on a single negotiation variable. It models mathematically in the two-player, two-dimensional case negotiation aspects of a general approach to conflict resolution and design of purposeful systems discussed by Shakun (Shakun, M. F. 1981. Formalizing conflict resolution in policy making. Internal. J. Gen. Systems 7 (3); Shakun, M. F. 1981. Policy making and meaning as design of purposeful systems. Internal J. Gen. Systems 7 (4).).group decisions/bargaining

Using Regularization to Derive Optimally Connected Architectures

There are several ways to design neural networks so that they would generalize better. Specifying an architecture with local connections is one of these, but the choice for the size and location of the neighborhoods involved in this method always seems to be ad hoc. We propose to add an extra term to the error function, which forces a fully connected network to find optimal local connections or locally shared connections. We also propose methods to optimize the hyperparameters involved in this extra term. Results are shown on a compression task of handwritten digits with an auto-encoder MLP. 1 Introduction The number of free parameters in artificial neural networks is a well known factor of generalization performance [Baum and Haussler 89, Moody 92]. Constraining this number is thus necessary to reach good performance. One can do so in basically two different ways : imposing a priori constraints on the network, or letting the data impose the constraints (through learning). The first m..