Nesterov Acceleration of Alternating Least Squares for Canonical Tensor Decomposition: Momentum Step Size Selection and Restart Mechanisms

We present Nesterov-type acceleration techniques for Alternating Least Squares (ALS) methods applied to canonical tensor decomposition. While Nesterov acceleration turns gradient descent into an optimal first-order method for convex problems by adding a momentum term with a specific weight sequence, a direct application of this method and weight sequence to ALS results in erratic convergence behaviour. This is so because the tensor decomposition problem is non-convex and ALS is accelerated instead of gradient descent. Instead, we consider various restart mechanisms and suitable choices of momentum weights that enable effective acceleration. Our extensive empirical results show that the Nesterov-accelerated ALS methods with restart can be dramatically more efficient than the stand-alone ALS or Nesterov accelerated gradient methods, when problems are ill-conditioned or accurate solutions are desired. The resulting methods perform competitively with or superior to existing acceleration methods for ALS, including ALS acceleration by NCG, NGMRES, or LBFGS, and additionally enjoy the benefit of being much easier to implement. We also compare with Nesterov-type updates where the momentum weight is determined by a line search, which are equivalent or closely related to existing line search methods for ALS. On a large and ill-conditioned 71$\times$1000$\times$900 tensor consisting of readings from chemical sensors to track hazardous gases, the restarted Nesterov-ALS method shows desirable robustness properties and outperforms any of the existing methods by a large factor. There is clear potential for extending our Nesterov-type acceleration approach to accelerating other optimization algorithms than ALS applied to other non-convex problems, such as Tucker tensor decomposition. Our Matlab code is available at https://github.com/hansdesterck/nonlinear-preconditioning-for-optimization.Comment: This version: journal revision, Nov 30, 201

Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded

Decision trees usefully represent sparse, high dimensional and noisy data. Having learned a function from this data, we may want to thereafter integrate the function into a larger decision-making problem, e.g., for picking the best chemical process catalyst. We study a large-scale, industrially-relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees and penalty functions mitigating risk. This mixed-integer optimization problem with convex penalty terms broadly applies to optimizing pre-trained regression tree models. Decision makers may wish to optimize discrete models to repurpose legacy predictive models, or they may wish to optimize a discrete model that particularly well-represents a data set. We develop several heuristic methods to find feasible solutions, and an exact, branch-and-bound algorithm leveraging structural properties of the gradient-boosted trees and penalty functions. We computationally test our methods on concrete mixture design instance and a chemical catalysis industrial instance

Robust ab initio calculation of condensed matter: transparent convergence through semicardinal multiresolution analysis

We present the first wavelet-based all-electron density-functional calculations to include gradient corrections and the first in a solid. Direct comparison shows this approach to be unique in providing systematic ``transparent'' convergence, convergence with a priori prediction of errors, to beyond chemical (millihartree) accuracy. The method is ideal for exploration of materials under novel conditions where there is little experience with how traditional methods perform and for the development and use of chemically accurate density functionals, which demand reliable access to such precision.Comment: 4 pages, 3 figures, 4 tables. Submitted to Phys. Rev. Lett. (updated to include GGA

Long range and duration underwater localization using molecular messaging

In this paper, we tackle the problem of how to locate a single entity with an unknown location in a vast underwater search space. In under-water channels, traditional wave-based signals suffer from rapid distance- and time-dependent energy attenuation, leading to expensive and lengthy search missions. In view of this, we investigate two molecular messaging methods for location discovery: a Rosenbrock gradient ascent algorithm, and a chemical encoding messaging method. In absence of explicit diffusion channel knowledge and in presence of diffusion noise, the Rosenbrock method is adapted to account for the blind search process and allow the robot to recover in areas of zero gradient. The two chemical methods are found to offer attractive performance trade-offs in complexity and robustness. Compared to conventional acoustic signals, the chemical methods proposed offers significantly longer propagation distance (1000km) and longer signal persistence duration (months)

100th Anniversary of Macromolecular Science Viewpoint: Opportunities in the Physics of Sequence-Defined Polymers

Polymer science has been driven by ever-increasing molecular complexity, as polymer synthesis expands an already-vast palette of chemical and architectural parameter space. Copolymers represent a key example, where simple homopolymers have given rise to random, alternating, gradient, and block copolymers. Polymer physics has provided the insight needed to explore this monomer sequence parameter space. The future of polymer science, however, must contend with further increases in monomer precision, as this class of macromolecules moves ever closer to the sequence-monodisperse polymers that are the workhorses of biology. The advent of sequence-defined polymers gives rise to opportunities for material design, with increasing levels of chemical information being incorporated into long-chain molecules; however, this also raises questions that polymer physics must address. What properties uniquely emerge from sequence-definition? Is this circumstance-dependent? How do we define and think about sequence dispersity? How do we think about a hierarchy of sequence effects? Are more sophisticated characterization methods, as well as theoretical and computational tools, needed to understand this class of macromolecules? The answers to these questions touch on many difficult scientific challenges, setting the stage for a rich future for sequence-defined polymers in polymer physics

Optimization with gradient-boosted trees and risk control

Decision trees effectively represent the sparse, high dimensional and noisy nature of chemical data from experiments. Having learned a function from this data, we may want to thereafter optimize the function, e.g., picking the best chemical process catalyst. In this way, we may repurpose legacy predictive models. This work studies a large-scale, industrially-relevant mixed-integer quadratic optimization problem involving: (i) gradient-boosted pre-trained regression trees modeling catalyst behavior, (ii) penalty functions mitigating risk, and (iii) penalties enforcing composition constraints. We develop heuristic methods and an exact, branch-and-bound algorithm leveraging structural properties of gradient-boosted trees and penalty functions. We numerically test our methods on an industrial instance

Efficient algorithms for rigid body integration using optimized splitting methods and exact free rotational motion

Hamiltonian splitting methods are an established technique to derive stable and accurate integration schemes in molecular dynamics, in which additional accuracy can be gained using force gradients. For rigid bodies, a tradition exists in the literature to further split up the kinetic part of the Hamiltonian, which lowers the accuracy. The goal of this note is to comment on the best combination of optimized splitting and gradient methods that avoids splitting the kinetic energy. These schemes are generally applicable, but the optimal scheme depends on the desired level of accuracy. For simulations of liquid water it is found that the velocity Verlet scheme is only optimal for crude simulations with accuracies larger than 1.5%, while surprisingly a modified Verlet scheme (HOA) is optimal up to accuracies of 0.4% and a fourth order gradient scheme (GIER4) is optimal for even higher accuracies.Comment: 2 pages, 1 figure. Added clarifying comments. Accepted for publication in the Journal of Chemical Physic

Towards quantum-chemical method development for arbitrary basis functions

We present the design of a flexible quantum-chemical method development framework, which supports employing any type of basis function. This design has been implemented in the light-weight program package molsturm, yielding a basis-function-independent self-consistent field scheme. Versatile interfaces, making use of open standards like python, mediate the integration of molsturm with existing third-party packages. In this way both rapid extension of the present set of methods for electronic structure calculations as well as adding new basis function types can be readily achieved. This makes molsturm well-suitable for testing novel approaches for discretising the electronic wave function and allows comparing them to existing methods using the same software stack. This is illustrated by two examples, an implementation of coupled-cluster doubles as well as a gradient-free geometry optimisation, where in both cases, an arbitrary basis functions could be used. molsturm is open-source and can be obtained from https://molsturm.org.Comment: 15 pages and 7 figure

Galactic abundance gradients from Cepheids : On the iron abundance gradient around 10-12 kpc

Context: Classical Cepheids can be adopted to trace the chemical evolution of the Galactic disk since their distances can be estimated with very high accuracy. Aims: Homogeneous iron abundance measurements for 33 Galactic Cepheids located in the outer disk together with accurate distance determinations based on near-infrared photometry are adopted to constrain the Galactic iron gradient beyond 10 kpc. Methods: Iron abundances were determined using high resolution Cepheid spectra collected with three different observational instruments: ESPaDOnS@CFHT, Narval@TBL and [email protected] ESO/MPG telescope. Cepheid distances were estimated using near-infrared (J,H,K-band) period-luminosity relations and data from SAAO and the 2MASS catalog. Results: The least squares solution over the entire data set indicates that the iron gradient in the Galactic disk presents a slope of -0.052+/-0.003 dex/kpc in the 5-17 kpc range. However, the change of the iron abundance across the disk seems to be better described by a linear regime inside the solar circle and a flattening of the gradient toward the outer disk (beyond 10 kpc). In the latter region the iron gradient presents a shallower slope, i.e. -0.012+/-0.014 dex/kpc. In the outer disk (10-12 kpc) we also found that Cepheids present an increase in the spread in iron abundance. Current evidence indicates that the spread in metallicity depends on the Galactocentric longitude. Finally, current data do not support the hypothesis of a discontinuity in the iron gradient at Galactocentric distances of 10-12 kpc. Conclusions: The occurrence of a spread in iron abundance as a function of the Galactocentric longitude indicates that linear radial gradients should be cautiously treated to constrain the chemical evolution across the disk.Comment: 5 tables, 8 figures, Accepted in A&