Saddles and Barrier in Landscapes of Generalized Search Operators

Barrier trees are a convenient way of representing the structure of complex combinatorial landscapes over graphs. Here we generalize the concept of barrier trees to landscapes defined over general multi-parent search operators based on a suitable notion of topological connectedness that depends explicitly on the search operator. We show that in the case of recombination spaces, path-connectedness coincides with connectedness as defined by the mutation operator alone. In contrast, topological connectedness is more general and depends on the details of the recombination operators as well. Barrier trees can be meaningfully defined for both concepts of connectedness

High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics

In these lecture notes we present different methods and concepts developed in statistical physics to analyze gradient descent dynamics in high-dimensional non-convex landscapes. Our aim is to show how approaches developed in physics, mainly statistical physics of disordered systems, can be used to tackle open questions on high-dimensional dynamics in Machine Learning.Comment: Lectures given by G. Biroli at the 2022 Les Houches Summer School "Statistical Physics and Machine Learning

Landscape Encodings Enhance Optimization

Hard combinatorial optimization problems deal with the search for the minimum cost solutions (ground states) of discrete systems under strong constraints. A transformation of state variables may enhance computational tractability. It has been argued that these state encodings are to be chosen invertible to retain the original size of the state space. Here we show how redundant non-invertible encodings enhance optimization by enriching the density of low-energy states. In addition, smooth landscapes may be established on encoded state spaces to guide local search dynamics towards the ground state

Cover-Encodings of Fitness Landscapes

The traditional way of tackling discrete optimization problems is by using local search on suitably defined cost or fitness landscapes. Such approaches are however limited by the slowing down that occurs when the local minima that are a feature of the typically rugged landscapes encountered arrest the progress of the search process. Another way of tackling optimization problems is by the use of heuristic approximations to estimate a global cost minimum. Here we present a combination of these two approaches by using cover-encoding maps which map processes from a larger search space to subsets of the original search space. The key idea is to construct cover-encoding maps with the help of suitable heuristics that single out near-optimal solutions and result in landscapes on the larger search space that no longer exhibit trapping local minima. We present cover-encoding maps for the problems of the traveling salesman, number partitioning, maximum matching and maximum clique; the practical feasibility of our method is demonstrated by simulations of adaptive walks on the corresponding encoded landscapes which find the global minima for these problems.Comment: 15 pages, 4 figure

Exponential number of equilibria and depinning threshold for a directed polymer in a random potential

By extending the Kac-Rice approach to manifolds of finite internal dimension, we show that the mean number $\left\langle\mathcal{N}_\mathrm{tot}\right\rangle$ of all possible equilibria (i.e. force-free configurations, a.k.a. equilibrium points) of an elastic line (directed polymer), confined in a harmonic well and submitted to a quenched random Gaussian potential in dimension $d=1+1$, grows exponentially $\left\langle\mathcal{N}_\mathrm{tot}\right\rangle\sim\exp{(r\,L)}$ with its length $L$. The growth rate $r$ is found to be directly related to the generalised Lyapunov exponent (GLE) which is a moment-generating function characterising the large-deviation type fluctuations of the solution to the initial value problem associated with the random Schr\"odinger operator of the 1D Anderson localization problem. For strong confinement, the rate $r$ is small and given by a non-perturbative (instanton, Lifshitz tail-like) contribution to GLE. For weak confinement, the rate $r$ is found to be proportional to the inverse Larkin length of the pinning theory. As an application, identifying the depinning with a landscape "topology trivialization" phenomenon, we obtain an upper bound for the depinning threshold $f_c$, in the presence of an applied force, for elastic lines and $d$-dimensional manifolds, expressed through the mean modulus of the spectral determinant of the Laplace operators with a random potential. We also discuss the question of counting of stable equilibria. Finally, we extend the method to calculate the asymptotic number of equilibria at fixed energy (elastic, potential and total), and obtain the (annealed) distribution of the energy density over these equilibria (i.e. force-free configurations). Some connections with the Larkin model are also established.Comment: v1: 6 pages main text + 14 pages supplemental material, 10 figures. v2: LaTeX, 79 pages, 18 eps figures, new material (Sections 6, 10, 11 & Appendices C, E, F, G

Contours in Visualization

This thesis studies the visualization of set collections either via or defines as the relations among contours. In the first part, dynamic Euler diagrams are used to communicate and improve semimanually the result of clustering methods which allow clusters to overlap arbitrarily. The contours of the Euler diagram are rendered as implicit surfaces called blobs in computer graphics. The interaction metaphor is the moving of items into or out of these blobs. The utility of the method is demonstrated on data arising from the analysis of gene expressions. The method works well for small datasets of up to one hundred items and few clusters. In the second part, these limitations are mitigated employing a GPU-based rendering of Euler diagrams and mixing textures and colors to resolve overlapping regions better. The GPU-based approach subdivides the screen into triangles on which it performs a contour interpolation, i.e. a fragment shader determines for each pixel which zones of an Euler diagram it belongs to. The rendering speed is thus increased to allow multiple hundred items. The method is applied to an example comparing different document clustering results. The contour tree compactly describes scalar field topology. From the viewpoint of graph drawing, it is a tree with attributes at vertices and optionally on edges. Standard tree drawing algorithms emphasize structural properties of the tree and neglect the attributes. Adapting popular graph drawing approaches to the problem of contour tree drawing it is found that they are unable to convey this information. Five aesthetic criteria for drawing contour trees are proposed and a novel algorithm for drawing contour trees in the plane that satisfies four of these criteria is presented. The implementation is fast and effective for contour tree sizes usually used in interactive systems and also produces readable pictures for larger trees. Dynamical models that explain the formation of spatial structures of RNA molecules have reached a complexity that requires novel visualization methods to analyze these model\''s validity. The fourth part of the thesis focuses on the visualization of so-called folding landscapes of a growing RNA molecule. Folding landscapes describe the energy of a molecule as a function of its spatial configuration; they are huge and high dimensional. Their most salient features are described by their so-called barrier tree -- a contour tree for discrete observation spaces. The changing folding landscapes of a growing RNA chain are visualized as an animation of the corresponding barrier tree sequence. The animation is created as an adaption of the foresight layout with tolerance algorithm for dynamic graph layout. The adaptation requires changes to the concept of supergraph and it layout. The thesis finishes with some thoughts on how these approaches can be combined and how the task the application should support can help inform the choice of visualization modality

Recent Applications of Dynamical Mean-Field Methods

Rich out of equilibrium collective dynamics of strongly interacting large assemblies emerge in many areas of science. Some intriguing and not fully understood examples are the glassy arrest in atomic, molecular or colloidal systems, flocking in natural or artificial active matter, and the organization and subsistence of ecosystems. The learning process, and ensuing amazing performance, of deep neural networks bears resemblance with some of the before-mentioned examples. Quantum mechanical extensions are also of interest. In exact or approximate manner the evolution of these systems can be expressed in terms of a dynamical mean-field theory which not only captures many of their peculiar effects but also has predictive power. This short review presents a summary of recent developments of this approach with emphasis on applications on the examples mentioned above.Comment: 39p, 6 figs, Annual Review of Condensed Matter Physics (to appear

Control of quantum phenomena: Past, present, and future

Quantum control is concerned with active manipulation of physical and chemical processes on the atomic and molecular scale. This work presents a perspective of progress in the field of control over quantum phenomena, tracing the evolution of theoretical concepts and experimental methods from early developments to the most recent advances. The current experimental successes would be impossible without the development of intense femtosecond laser sources and pulse shapers. The two most critical theoretical insights were (1) realizing that ultrafast atomic and molecular dynamics can be controlled via manipulation of quantum interferences and (2) understanding that optimally shaped ultrafast laser pulses are the most effective means for producing the desired quantum interference patterns in the controlled system. Finally, these theoretical and experimental advances were brought together by the crucial concept of adaptive feedback control, which is a laboratory procedure employing measurement-driven, closed-loop optimization to identify the best shapes of femtosecond laser control pulses for steering quantum dynamics towards the desired objective. Optimization in adaptive feedback control experiments is guided by a learning algorithm, with stochastic methods proving to be especially effective. Adaptive feedback control of quantum phenomena has found numerous applications in many areas of the physical and chemical sciences, and this paper reviews the extensive experiments. Other subjects discussed include quantum optimal control theory, quantum control landscapes, the role of theoretical control designs in experimental realizations, and real-time quantum feedback control. The paper concludes with a prospective of open research directions that are likely to attract significant attention in the future.Comment: Review article, final version (significantly updated), 76 pages, accepted for publication in New J. Phys. (Focus issue: Quantum control