Deterministic Annealing and Nonlinear Assignment

For combinatorial optimization problems that can be formulated as Ising or Potts spin systems, the Mean Field (MF) approximation yields a versatile and simple ANN heuristic, Deterministic Annealing. For assignment problems the situation is more complex -- the natural analog of the MF approximation lacks the simplicity present in the Potts and Ising cases. In this article the difficulties associated with this issue are investigated, and the options for solving them discussed. Improvements to existing Potts-based MF-inspired heuristics are suggested, and the possibilities for defining a proper variational approach are scrutinized.Comment: 15 pages, 3 figure

Generalized Hopfield networks for constrained optimization

A twofold generalization of the classical continuous Hopfield neural network for modelling constrained optimization problems is proposed. On the one hand, non-quadratic cost functions are admitted corresponding to non-linear output summation functions in the neurons. On the other hand it is shown under which conditions various (new) types of constraints can be incorporated directly. The stability properties of several relaxation schemes are shown. If a direct incorporation of the constraints appears to be impossible, the Hopfield-Lagrange model can be applied, the stability properties of which are analyzed as well. Another good way to deal with constraints is by means of dynamic penalty terms, using mean field annealing in order to end up in a feasible solution. A famous example in this context is the elastic net, although it seems impossible - contrary to what is suggested in the literature - to derive the architecture of this network from a constrained Hopfield model. Furthermore, a non-equidistant elastic net is proposed and its stability properties are compared to those of the classical elastic network. In addition to certain simulation results as known from the literature, most theoretical statements of this paper are validated with simulations of toy problems while in some cases, more sophisticated combinatorial optimization problems have been tried as well. In the final section, we discuss the possibilities of applying the various models in the area of constrained optimization. It is also demonstrated how the new ideas as inspired by the analysis of generalized continuous Hopfield models, can be transferred to discrete stochastic Hopfield models. By doing so, simulating annealing can be exploited in other to improve the quality of solutions. The transfer also opens new avenues for continued theoretical research