Finding Minima in Complex Landscapes: Annealed, Greedy and Reluctant Algorithms

We consider optimization problems for complex systems in which the cost function has a multivalleyed landscape. We introduce a new class of dynamical algorithms which, using a suitable annealing procedure coupled with a balanced greedy-reluctant strategy drive the systems towards the deepest minimum of the cost function. Results are presented for the Sherrington-Kirkpatrick model of spin-glasses.Comment: 30 pages, 12 figure

Numerical study of ground state energy fluctuations in spin glasses

Using a stochastic algorithm introduced in a previous paper, we study the finite size volume corrections and the fluctuations of the ground state energy in the Sherrington-Kirkpatrick and the Edwards-Anderson models at zero temperature. The algorithm is based on a suitable annealing procedure coupled with a balanced greedy-reluctant strategy that drives the systems towards the deepest minimum of the energy function.Comment: 20 pages, 14 figure

Reduced Order and Surrogate Models for Gravitational Waves

We present an introduction to some of the state of the art in reduced order and surrogate modeling in gravitational wave (GW) science. Approaches that we cover include Principal Component Analysis, Proper Orthogonal Decomposition, the Reduced Basis approach, the Empirical Interpolation Method, Reduced Order Quadratures, and Compressed Likelihood evaluations. We divide the review into three parts: representation/compression of known data, predictive models, and data analysis. The targeted audience is that one of practitioners in GW science, a field in which building predictive models and data analysis tools that are both accurate and fast to evaluate, especially when dealing with large amounts of data and intensive computations, are necessary yet can be challenging. As such, practical presentations and, sometimes, heuristic approaches are here preferred over rigor when the latter is not available. This review aims to be self-contained, within reasonable page limits, with little previous knowledge (at the undergraduate level) requirements in mathematics, scientific computing, and other disciplines. Emphasis is placed on optimality, as well as the curse of dimensionality and approaches that might have the promise of beating it. We also review most of the state of the art of GW surrogates. Some numerical algorithms, conditioning details, scalability, parallelization and other practical points are discussed. The approaches presented are to large extent non-intrusive and data-driven and can therefore be applicable to other disciplines. We close with open challenges in high dimension surrogates, which are not unique to GW science.Comment: Invited article for Living Reviews in Relativity. 93 page

Multicamera 3D Viewpoint Adjustment for Robotic Surgery via Deep Reinforcement Learning

While robot-assisted minimally invasive surgery (RMIS) procedures afford a variety of benefits over open surgery and manual laparoscopic operations (including increased tool dexterity, reduced patient pain, incision size, trauma and recovery time, and lower infection rates [1], lack of spatial awareness remains an issue. Typical laparoscopic imaging can lack sufficient depth cues and haptic feedback, if provided, rarely reflects realistic tissue-tool interactions. This work is part of a larger ongoing research effort to reconstruct 3D surfaces using multiple viewpoints in RMIS to increase visual perception. The manual placement and adjustment of multicamera systems in RMIS are nonideal and prone to error [2], and other autonomous approaches focus on tool tracking and do not consider reconstruction of the surgical scene [3,4,5]. The group\u27s previous work investigated a novel, context-aware autonomous camera positioning method [6], which incorporated both tool location and scene coverage for multiple camera viewpoint adjustments. In this paper, the authors expand upon this prior work by implementing a streamlined deep reinforcement learning approach between optimal viewpoints calculated using the prior method [6] which encourages discovery of otherwise unobserved and additional camera viewpoints. Combining the framework and robustness of the previous work with the efficiency and additional viewpoints of the augmentations presented here results in improved performance and scene coverage promising towards real-time implementation

Application of reinforcement learning methods to computer game dynamics

The dynamics of the game world present both challenges and opportunities for AI to make a useful difference. Learning smart behaviours for game assets is a first step towards realistic conflict or cooperation. The scope this thesis is the application of Reinforcement Learning to moving assets in the game world. Game sessions a generate stream data on asset's performance which must be processed on the fly. The lead objective is to produce fast, lightweight and flexible learning algorithms for run-time embedding. The motivation from current work is to shorten the time to achieve a workable policy solution by investigating the exploration / exploitation balance, overcome the curse of dimensionality of complex systems, and avoid the use of extra endogenous parameters which require multiple data passes and use a simple state aggregation rather than functional approximation. How action selection (AS) contributes to efficient learning is a key issue in RL since is determines the balance between exploiting and confirming the current policy or exploring an early less likely policy which may prove better in the long run. The methodology deploys the simulation of several AS using 10-armed bandit problem averaged over 10000 epochs. The results show a considerable variation in performance in terms of latency and asymptotic direction. The Upper Confidence Bound comes out leader over most of the episode range, especially at about 100. Using insight from action selection order statistics are applied to determine a criterion for the convergence of policy evaluation. The probability that the action of maximum sample mean is indeed the action of maximum population mean (PMSMMPM) is calculated using the 3 armed bandit problem. PMSMMPM reaches 0.988 by play 26 which provides evidence for it as a convergence criterion. An iteration stopping rule is defined using PMSMMPM and it shows plausible properties as the population parameters are varied. A mathematical analysis of the approximation (P21) of just taking the top two actions yields a minimum sampling size for any level of P21. Using the gradient of P21 a selection rule is derived and when combined with UCB a new complete exploratory policy is demonstrated for 3-arm bandit that requires just over half the sample size when compared with pure UCB. The results provide evidence that the augmented UCB selection rule will contribute to faster learning. TD sarsa(0) learning algorithm has been applied to learn a steering policy for the untried caravan reversing problem and for the kerb avoiding steering problem of racing car both using negative rewards on failure and a simple aggregation. The output policy for the caravan is validated as non jack-knifing for a high proportion of start states. The racing car policy has a similar validation outcome for two exploratory polies which are compared and contrasted

Properties of Random Fitness Landscapes and Their Influence on Evolutionary Dynamics. A Journey through the Hypercube

A fitness landscape is a theoretical concept in population genetics where a fitness value, which measures the reproductive success of an organism and is represented by a real number, is assigned to each genotype. Content of this thesis is the analytical and numerical study of stochastic models for fitness landscapes. The focus is on the landscape ruggedness and its influence on evolutionary dynamics. One proxy for the ruggedness is the number of local maxima, i.e., genotypes from which every mutation leads to lowered fitness. Another way to quantify ruggedness is the study of accessible paths, i.e., successions of mutations that increase the fitness monotonically. The question whether accessible paths exist can be interpreted as a kind of percolation problem. One model for evolutionary dynamics that will be used is the adaptive walk. In this model type, populations are treated as single entities that move through the space of genotypes according to certain probabilistic rules. They are closely related to both ruggedness measures as they follow accessible paths and terminate at local maxima. Furthermore, the individual-based Wright-Fisher model is used to study recombination of genotypes, interactions between individuals and the influence of the underlying fitness landscape on these mechanisms

Machine Learning Econometrics

Much of econometrics is based on a tight probabilistic approach to empirical modeling that dates back to Haavelmo (1944). This thesis explores a modern algorithmic view, and by doing so, finds solutions to classic problems while developing new avenues. In the first chapter, Kalman-filter based computations of random walk coefficients are replaced by a closed-form solution only second to least squares in the pantheon of simplicity. In the second chapter, random walk “drifting” coefficients are themselves dismissed. Rather, evolving coefficients are modeled and forecasted with a powerful machine learning algorithm. Conveniently, this generalization of time-varying parameters provides statistical efficiency and interpretability, which off-the-shelf machine learning algorithms cannot easily offer. The third chapter is about the to the fundamental problem of detecting at which point a learner stops learning and starts imitating. It answers “why can’t Random Forest overfit?” The phenomenon is shown to be a surprising byproduct of randomized “greedy” algorithms – often deployed in the face of computational adversity. Then, the insights are utilized to develop new high-performing non-overfitting algorithms