Adaptive Regularization for Nonconvex Optimization Using Inexact Function Values and Randomly Perturbed Derivatives

A regularization algorithm allowing random noise in derivatives and inexact function values is proposed for computing approximate local critical points of any order for smooth unconstrained optimization problems. For an objective function with Lipschitz continuous $p$-th derivative and given an arbitrary optimality order $q \leq p$, it is shown that this algorithm will, in expectation, compute such a point in at most $O\left(\left(\min_{j\in\{1,\ldots,q\}}\epsilon_j\right)^{-\frac{p+1}{p-q+1}}\right)$ inexact evaluations of $f$ and its derivatives whenever $q\in\{1,2\}$, where $\epsilon_j$ is the tolerance for $j$th order accuracy. This bound becomes at most $O\left(\left(\min_{j\in\{1,\ldots,q\}}\epsilon_j\right)^{-\frac{q(p+1)}{p}}\right)$ inexact evaluations if $q>2$ and all derivatives are Lipschitz continuous. Moreover these bounds are sharp in the order of the accuracy tolerances. An extension to convexly constrained problems is also outlined.Comment: 22 page

Adaptive Regularization Algorithms with Inexact Evaluations for Nonconvex Optimization

A regularization algorithm using inexact function values and inexact derivatives is proposed and its evaluation complexity analyzed. This algorithm is applicable to unconstrained problems and to problems with inexpensive constraints (that is constraints whose evaluation and enforcement has negligible cost) under the assumption that the derivative of highest degree is $\beta$-H\"{o}lder continuous. It features a very flexible adaptive mechanism for determining the inexactness which is allowed, at each iteration, when computing objective function values and derivatives. The complexity analysis covers arbitrary optimality order and arbitrary degree of available approximate derivatives. It extends results of Cartis, Gould and Toint (2018) on the evaluation complexity to the inexact case: if a $q$th order minimizer is sought using approximations to the first $p$ derivatives, it is proved that a suitable approximate minimizer within $\epsilon$ is computed by the proposed algorithm in at most $O(\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ iterations and at most $O(|\log(\epsilon)|\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ approximate evaluations. An algorithmic variant, although more rigid in practice, can be proved to find such an approximate minimizer in $O(|\log(\epsilon)|+\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ evaluations.While the proposed framework remains so far conceptual for high degrees and orders, it is shown to yield simple and computationally realistic inexact methods when specialized to the unconstrained and bound-constrained first- and second-order cases. The deterministic complexity results are finally extended to the stochastic context, yielding adaptive sample-size rules for subsampling methods typical of machine learning.Comment: 32 page

Updating constraint preconditioners for KKT systems in quadratic programming via low-rank corrections

This work focuses on the iterative solution of sequences of KKT linear systems arising in interior point methods applied to large convex quadratic programming problems. This task is the computational core of the interior point procedure and an efficient preconditioning strategy is crucial for the efficiency of the overall method. Constraint preconditioners are very effective in this context; nevertheless, their computation may be very expensive for large-scale problems, and resorting to approximations of them may be convenient. Here we propose a procedure for building inexact constraint preconditioners by updating a "seed" constraint preconditioner computed for a KKT matrix at a previous interior point iteration. These updates are obtained through low-rank corrections of the Schur complement of the (1,1) block of the seed preconditioner. The updated preconditioners are analyzed both theoretically and computationally. The results obtained show that our updating procedure, coupled with an adaptive strategy for determining whether to reinitialize or update the preconditioner, can enhance the performance of interior point methods on large problems.Comment: 22 page

Experimental and Numerical Analysis of Single Phase Flow in a micro T-junction

This paper was presented at the 4th Micro and Nano Flows Conference (MNF2014), which was held at University College, London, UK. The conference was organised by Brunel University and supported by the Italian Union of Thermofluiddynamics, IPEM, the Process Intensification Network, the Institution of Mechanical Engineers, the Heat Transfer Society, HEXAG - the Heat Exchange Action Group, and the Energy Institute, ASME Press, LCN London Centre for Nanotechnology, UCL University College London, UCL Engineering, the International NanoScience Community, www.nanopaprika.eu.In this work the fluid-dynamic behaviour of a micro T-junction has been investigated both numerically and experimentally for low Reynolds numbers (Re<14) with water as working fluid. The velocity profiles within the T-junction has been experimentally determined by using the micro Particle Image Velocimetry (μPIV). The experimental data have been compared with the numerical results obtained by means of a 3D model implemented in Comsol Multiphysics® environment for incompressible, isothermal, laminar flows with constant properties. The comparison between the experimental and the numerical data puts in evidence a perfect agreement among the results. In the central region of the T-junction where the velocity profiles of the inlet branches interact, the maximum difference is less than 5.8% for different flow rates imposed at the inlet (with the ratio 1:2) and less than 4.4% in the case of the same flow rate at the inlets (1:1). Since the estimated uncertainty of the experimental velocity is about 3%, the obtained result can be considered very good and it demonstrates that no significant scaling effects influences the liquid mixing for low Reynolds numbers (Re<14) and the behaviour of the micro T-junction can be considered as conventional. The detailed analysis of the velocity profile evolution within the central region of the mixer has allowed to determine where the fully developed laminar profile is reached (for instance 260 mm far from the centre of the T-junction when a maximum water flow rate of 8 ml/h is considered)

On the Convergence Properties of a Stochastic Trust-Region Method with Inexact Restoration

We study the convergence properties of SIRTR, a stochastic inexact restoration trust-region method suited for the minimization of a finite sum of continuously differentiable functions. This method combines the trust-region methodology with random function and gradient estimates formed by subsampling. Unlike other existing schemes, it forces the decrease of a merit function by combining the function approximation with an infeasibility term, the latter of which measures the distance of the current sample size from its maximum value. In a previous work, the expected iteration complexity to satisfy an approximate first-order optimality condition was given. Here, we elaborate on the convergence analysis of SIRTR and prove its convergence in probability under suitable accuracy requirements on random function and gradient estimates. Furthermore, we report the numerical results obtained on some nonconvex classification test problems, discussing the impact of the probabilistic requirements on the selection of the sample sizes

The strange case of negative reflection

In this paper, we show the phenomenon of negative reflection occurring in a mechanical phononic structure, consisting of a grating of fixed inclusions embedded in a linear elastic matrix. The negative reflection is not due to the introduction of a subwavelength metastructure or materials with negative mechanical properties. Numerical analyses for out-of-plane shear waves demonstrate that there exist frequencies at which most of the incident energy is reflected at negative angles. The effect is symmetric with respect to a line that is not parallel to the normal direction to the grating structure. Simulations at different angles of incidence and computations of the energy fluxes show that negative reflection is achievable in a wide range of loading conditions

Solving Nonlinear Systems of Equations Via Spectral Residual Methods: Stepsize Selection and Applications

Spectral residual methods are derivative-free and low-cost per iteration procedures for solving nonlinear systems of equations. They are generally coupled with a nonmonotone linesearch strategy and compare well with Newton-based methods for large nonlinear systems and sequences of nonlinear systems. The residual vector is used as the search direction and choosing the steplength has a crucial impact on the performance. In this work we address both theoretically and experimentally the steplength selection and provide results on a real application such as a rolling contact problem