Phase transitions in soft-committee machines

Equilibrium statistical physics is applied to layered neural networks with differentiable activation functions. A first analysis of off-line learning in soft-committee machines with a finite number (K) of hidden units learning a perfectly matching rule is performed. Our results are exact in the limit of high training temperatures. For K=2 we find a second order phase transition from unspecialized to specialized student configurations at a critical size P of the training set, whereas for K > 2 the transition is first order. Monte Carlo simulations indicate that our results are also valid for moderately low temperatures qualitatively. The limit K to infinity can be performed analytically, the transition occurs after presenting on the order of N K examples. However, an unspecialized metastable state persists up to P= O (N K^2).Comment: 8 pages, 4 figure

Design and Implementation of a Deterministic and Nondeterministic Finite Automaton Simulator

The purpose of this project is to assist students in visualizing and understanding the structure and operation of deterministic and nondeterministic finite automata. This software achieves this purpose by providing students with the ability to build, modify, and test automata in an intuitive environment. This enables a simple and efficient avenue for experimentation, which upholds the Cal Poly ideal of Learning by Doing. Readers of this report should be familiar with basic concepts in the theory of finite state machines; a general understanding of object-oriented programming is also necessary

Testing and Active Learning of Resettable Finite-State Machines

This thesis proposes novel active-learning algorithms and testing methods for deterministic finite-state machines that (i) have a specified transition from every state on each input of the (fixed) alphabet and (ii) can be reliably reset to the initial state on request. These algorithms rely on the novel methods of construction of separating sequences. Extensive evaluation demonstrates that the described testing and learning methods are the most efficient in terms of the amount of interaction by a tester with the system under test