The geometry of nonlinear least squares with applications to sloppy models and optimization

Parameter estimation by nonlinear least squares minimization is a common problem with an elegant geometric interpretation: the possible parameter values of a model induce a manifold in the space of data predictions. The minimization problem is then to find the point on the manifold closest to the data. We show that the model manifolds of a large class of models, known as sloppy models, have many universal features; they are characterized by a geometric series of widths, extrinsic curvatures, and parameter-effects curvatures. A number of common difficulties in optimizing least squares problems are due to this common structure. First, algorithms tend to run into the boundaries of the model manifold, causing parameters to diverge or become unphysical. We introduce the model graph as an extension of the model manifold to remedy this problem. We argue that appropriate priors can remove the boundaries and improve convergence rates. We show that typical fits will have many evaporated parameters. Second, bare model parameters are usually ill-suited to describing model behavior; cost contours in parameter space tend to form hierarchies of plateaus and canyons. Geometrically, we understand this inconvenient parametrization as an extremely skewed coordinate basis and show that it induces a large parameter-effects curvature on the manifold. Using coordinates based on geodesic motion, these narrow canyons are transformed in many cases into a single quadratic, isotropic basin. We interpret the modified Gauss-Newton and Levenberg-Marquardt fitting algorithms as an Euler approximation to geodesic motion in these natural coordinates on the model manifold and the model graph respectively. By adding a geodesic acceleration adjustment to these algorithms, we alleviate the difficulties from parameter-effects curvature, improving both efficiency and success rates at finding good fits.Comment: 40 pages, 29 Figure

A phenomenological approach to normal form modeling: a case study in laser induced nematodynamics

An experimental setting for the polarimetric study of optically induced dynamical behavior in nematic liquid crystal films has allowed to identify most notably some behavior which was recognized as gluing bifurcations leading to chaos. This analysis of the data used a comparison with a model for the transition to chaos via gluing bifurcations in optically excited nematic liquid crystals previously proposed by G. Demeter and L. Kramer. The model of these last authors, proposed about twenty years before, does not have the central symmetry which one would expect for minimal dimensional models for chaos in nematics in view of the time series. What we show here is that the simplest truncated normal forms for gluing, with the appropriate symmetry and minimal dimension, do exhibit time signals that are embarrassingly similar to the ones found using the above mentioned experimental settings. The gluing bifurcation scenario itself is only visible in limited parameter ranges and substantial aspect of the chaos that can be observed is due to other factors. First, out of the immediate neighborhood of the homoclinic curve, nonlinearity can produce expansion leading to chaos when combined with the recurrence induced by the homoclinic behavior. Also, pairs of symmetric homoclinic orbits create extreme sensitivity to noise, so that when the noiseless approach contains a rich behavior, minute noise can transform the complex damping into sustained chaos. Leonid Shil'nikov taught us that combining global considerations and local spectral analysis near critical points is crucial to understand the phenomenology associated to homoclinic bifurcations. Here this helps us construct a phenomenological approach to modeling experiments in nonlinear dissipative contexts.Comment: 25 pages, 9 figure

On the spectral distribution of large weighted random regular graphs

McKay proved that the limiting spectral measures of the ensembles of $d$-regular graphs with $N$ vertices converge to Kesten's measure as $N\to\infty$. In this paper we explore the case of weighted graphs. More precisely, given a large $d$-regular graph we assign random weights, drawn from some distribution $\mathcal{W}$, to its edges. We study the relationship between $\mathcal{W}$ and the associated limiting spectral distribution obtained by averaging over the weighted graphs. Among other results, we establish the existence of a unique `eigendistribution', i.e., a weight distribution $\mathcal{W}$ such that the associated limiting spectral distribution is a rescaling of $\mathcal{W}$. Initial investigations suggested that the eigendistribution was the semi-circle distribution, which by Wigner's Law is the limiting spectral measure for real symmetric matrices. We prove this is not the case, though the deviation between the eigendistribution and the semi-circular density is small (the first seven moments agree, and the difference in each higher moment is $O(1/d^2)$). Our analysis uses combinatorial results about closed acyclic walks in large trees, which may be of independent interest.Comment: Version 1.0, 19 page

Asymptotic safety in the f(R) approximation

In the asymptotic safety programme for quantum gravity, it is important to go beyond polynomial truncations. Three such approximations have been derived where the restriction is only to a general function f(R) of the curvature R>0. We confront these with the requirement that a fixed point solution be smooth and exist for all non-negative R. Singularities induced by cutoff choices force the earlier versions to have no such solutions. However, we show that the most recent version has a number of lines of fixed points, each supporting a continuous spectrum of eigen-perturbations. We uncover and analyse the first five such lines. Sensible fixed point behaviour may be achieved if one consistently incorporates geometry/topology change. As an exploratory example, we analyse the equations analytically continued to R<0, however we now find only partial solutions.We show how these results are always consistent with, and to some extent can be predicted from, a straightforward analysis of the constraints inherent in the equations.Comment: Latex, 66 pages, published version, typos correcte

Rotational levels in quantum dots

Low energy spectra of isotropic quantum dots are calculated in the regime of low electron densities where Coulomb interaction causes strong correlations. The earlier developed pocket state method is generalized to allow for continuous rotations. Detailed predictions are made for dots of shallow confinements and small particle numbers, including the occurance of spin blockades in transport.Comment: RevTeX, 10 pages, 2 figure

Excitation spectrum of the homogeneous spin liquid

We discuss the excitation spectrum of a disordered, isotropic and translationally invariant spin state in the 2D Heisenberg antiferromagnet. The starting point is the nearest-neighbor RVB state which plays the role of the vacuum of the theory, in a similar sense as the Neel state is the vacuum for antiferromagnetic spin wave theory. We discuss the elementary excitations of this state and show that these are not Fermionic spin-1/2 `spinons' but spin-1 excited dimers which must be modeled by bond Bosons. We derive an effective Hamiltonian describing the excited dimers which is formally analogous to spin wave theory. Condensation of the bond-Bosons at zero temperature into the state with momentum (pi,pi) is shown to be equivalent to antiferromagnetic ordering. The latter is a key ingredient for a microscopic interpretation of Zhang's SO(5) theory of cuprate superconductivityComment: RevTex-file, 16 PRB pages with 13 embedded eps figures. Hardcopies of figures (or the entire manuscript) can be obtained by e-mail request to: [email protected]

Sudden Trust Collapse in Networked Societies

Trust is a collective, self-fulfilling phenomenon that suggests analogies with phase transitions. We introduce a stylized model for the build-up and collapse of trust in networks, which generically displays a first order transition. The basic assumption of our model is that whereas trust begets trust, panic also begets panic, in the sense that a small decrease in trust may be amplified and ultimately lead to a sudden and catastrophic drop of trust. We show, using both numerical simulations and mean-field analytic arguments, that there are extended regions of the parameter space where two equilibrium states coexist: a well-connected network where confidence is high, and a poorly connected network where confidence is low. In these coexistence regions, spontaneous jumps from the well-connected state to the poorly connected state can occur, corresponding to a sudden collapse of trust that is not caused by any major external catastrophe. In large systems, spontaneous crises are replaced by history dependence: whether the system is found in one state or in the other essentially depends on initial conditions. Finally, we document a new phase, in which agents are connected yet distrustful.Comment: 15 pages, 10 figure