A linear theory for control of non-linear stochastic systems

We address the role of noise and the issue of efficient computation in stochastic optimal control problems. We consider a class of non-linear control problems that can be formulated as a path integral and where the noise plays the role of temperature. The path integral displays symmetry breaking and there exist a critical noise value that separates regimes where optimal control yields qualitatively different solutions. The path integral can be computed efficiently by Monte Carlo integration or by Laplace approximation, and can therefore be used to solve high dimensional stochastic control problems.Comment: 5 pages, 3 figures. Accepted to PR

Path integrals and symmetry breaking for optimal control theory

This paper considers linear-quadratic control of a non-linear dynamical system subject to arbitrary cost. I show that for this class of stochastic control problems the non-linear Hamilton-Jacobi-Bellman equation can be transformed into a linear equation. The transformation is similar to the transformation used to relate the classical Hamilton-Jacobi equation to the Schr\"odinger equation. As a result of the linearity, the usual backward computation can be replaced by a forward diffusion process, that can be computed by stochastic integration or by the evaluation of a path integral. It is shown, how in the deterministic limit the PMP formalism is recovered. The significance of the path integral approach is that it forms the basis for a number of efficient computational methods, such as MC sampling, the Laplace approximation and the variational approximation. We show the effectiveness of the first two methods in number of examples. Examples are given that show the qualitative difference between stochastic and deterministic control and the occurrence of symmetry breaking as a function of the noise.Comment: 21 pages, 6 figures, submitted to JSTA

Survey propagation at finite temperature: application to a Sourlas code as a toy model

In this paper we investigate a finite temperature generalization of survey propagation, by applying it to the problem of finite temperature decoding of a biased finite connectivity Sourlas code for temperatures lower than the Nishimori temperature. We observe that the result is a shift of the location of the dynamical critical channel noise to larger values than the corresponding dynamical transition for belief propagation, as suggested recently by Migliorini and Saad for LDPC codes. We show how the finite temperature 1-RSB SP gives accurate results in the regime where competing approaches fail to converge or fail to recover the retrieval state

Replica symmetry breaking in the `small world' spin glass

We apply the cavity method to a spin glass model on a `small world' lattice, a random bond graph super-imposed upon a 1-dimensional ferromagnetic ring. We show the correspondence with a replicated transfer matrix approach, up to the level of one step replica symmetry breaking (1RSB). Using the scheme developed by M\'ezard & Parisi for the Bethe lattice, we evaluate observables for a model with fixed connectivity and $\pm J$ long range bonds. Our results agree with numerical simulations significantly better than the replica symmetric (RS) theory.Comment: 21 pages, 3 figure

The Supersymmetric Particle Spectrum

We examine the spectrum of supersymmetric particles predicted by grand unified theoretical (GUT) models where the electroweak symmetry breaking is accomplished radiatively. We evolve the soft supersymmetry breaking parameters according to the renormalization group equations (RGE). The minimization of the Higgs potential is conveniently described by means of tadpole diagrams. We present complete one-loop expressions for these minimization conditions, including contributions from the matter and the gauge sectors. We concentrate on the low $\tan \beta$ fixed point region (that provides a natural explanation of a large top quark mass) for which we find solutions to the RGE satisfying both experimental bounds and fine-tuning criteria. We also find that the constraint from the consideration of the lightest supersymmetric particle as the dark matter of the universe is accommodated in much of parameter space where the lightest neutralino is predominantly gaugino. The supersymmetric mass spectrum displays correlations that are model-independent over much of the GUT parameter space.Comment: 62 pages + 10 PS figures included (uuencoded), MAD/PH/80

Effect of coupling asymmetry on mean-field solutions of direct and inverse Sherrington-Kirkpatrick model

We study how the degree of symmetry in the couplings influences the performance of three mean field methods used for solving the direct and inverse problems for generalized Sherrington-Kirkpatrick models. In this context, the direct problem is predicting the potentially time-varying magnetizations. The three theories include the first and second order Plefka expansions, referred to as naive mean field (nMF) and TAP, respectively, and a mean field theory which is exact for fully asymmetric couplings. We call the last of these simply MF theory. We show that for the direct problem, nMF performs worse than the other two approximations, TAP outperforms MF when the coupling matrix is nearly symmetric, while MF works better when it is strongly asymmetric. For the inverse problem, MF performs better than both TAP and nMF, although an ad hoc adjustment of TAP can make it comparable to MF. For high temperatures the performance of TAP and MF approach each other

Critical phenomena in complex networks

The combination of the compactness of networks, featuring small diameters, and their complex architectures results in a variety of critical effects dramatically different from those in cooperative systems on lattices. In the last few years, researchers have made important steps toward understanding the qualitatively new critical phenomena in complex networks. We review the results, concepts, and methods of this rapidly developing field. Here we mostly consider two closely related classes of these critical phenomena, namely structural phase transitions in the network architectures and transitions in cooperative models on networks as substrates. We also discuss systems where a network and interacting agents on it influence each other. We overview a wide range of critical phenomena in equilibrium and growing networks including the birth of the giant connected component, percolation, k-core percolation, phenomena near epidemic thresholds, condensation transitions, critical phenomena in spin models placed on networks, synchronization, and self-organized criticality effects in interacting systems on networks. We also discuss strong finite size effects in these systems and highlight open problems and perspectives.Comment: Review article, 79 pages, 43 figures, 1 table, 508 references, extende

Optimal control as a graphical model inference problem

We reformulate a class of non-linear stochastic optimal control problems introduced by Todorov (2007) as a Kullback-Leibler (KL) minimization problem. As a result, the optimal control computation reduces to an inference computation and approximate inference methods can be applied to efficiently compute approximate optimal controls. We show how this KL control theory contains the path integral control method as a special case. We provide an example of a block stacking task and a multi-agent cooperative game where we demonstrate how approximate inference can be successfully applied to instances that are too complex for exact computation. We discuss the relation of the KL control approach to other inference approaches to control.Comment: 26 pages, 12 Figures; Machine Learning Journal (2012

The future of sovereignty in multilevel governance Europe: a constructivist reading

Multilevel governance presents a depiction of contemporary structures in EU Europe as consisting of overlapping authorities and competing competencies. By focusing on emerging non-anarchical structures in the international system, hence moving beyond the conventional hierarchy/anarchy dichotomy to distinguish domestic and international arenas, this seems a radical transformation of the familiar Westphalian system and to undermine state sovereignty. Paradoxically, however, the principle of sovereignty proves to be resilient despite its alleged empirical decline. This article argues that social constructivism can explain the paradox, by considering sovereign statehood as a process-dependent institutional fact, and by showing that multilevel governance can feed into this process