Adaptive Regularization for Nonconvex Optimization Using Inexact Function Values and Randomly Perturbed Derivatives

A regularization algorithm allowing random noise in derivatives and inexact function values is proposed for computing approximate local critical points of any order for smooth unconstrained optimization problems. For an objective function with Lipschitz continuous $p$-th derivative and given an arbitrary optimality order $q \leq p$, it is shown that this algorithm will, in expectation, compute such a point in at most $O\left(\left(\min_{j\in\{1,\ldots,q\}}\epsilon_j\right)^{-\frac{p+1}{p-q+1}}\right)$ inexact evaluations of $f$ and its derivatives whenever $q\in\{1,2\}$, where $\epsilon_j$ is the tolerance for $j$th order accuracy. This bound becomes at most $O\left(\left(\min_{j\in\{1,\ldots,q\}}\epsilon_j\right)^{-\frac{q(p+1)}{p}}\right)$ inexact evaluations if $q>2$ and all derivatives are Lipschitz continuous. Moreover these bounds are sharp in the order of the accuracy tolerances. An extension to convexly constrained problems is also outlined.Comment: 22 page

Adaptive Regularization Algorithms with Inexact Evaluations for Nonconvex Optimization

A regularization algorithm using inexact function values and inexact derivatives is proposed and its evaluation complexity analyzed. This algorithm is applicable to unconstrained problems and to problems with inexpensive constraints (that is constraints whose evaluation and enforcement has negligible cost) under the assumption that the derivative of highest degree is $\beta$-H\"{o}lder continuous. It features a very flexible adaptive mechanism for determining the inexactness which is allowed, at each iteration, when computing objective function values and derivatives. The complexity analysis covers arbitrary optimality order and arbitrary degree of available approximate derivatives. It extends results of Cartis, Gould and Toint (2018) on the evaluation complexity to the inexact case: if a $q$th order minimizer is sought using approximations to the first $p$ derivatives, it is proved that a suitable approximate minimizer within $\epsilon$ is computed by the proposed algorithm in at most $O(\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ iterations and at most $O(|\log(\epsilon)|\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ approximate evaluations. An algorithmic variant, although more rigid in practice, can be proved to find such an approximate minimizer in $O(|\log(\epsilon)|+\epsilon^{-\frac{p+\beta}{p-q+\beta}})$ evaluations.While the proposed framework remains so far conceptual for high degrees and orders, it is shown to yield simple and computationally realistic inexact methods when specialized to the unconstrained and bound-constrained first- and second-order cases. The deterministic complexity results are finally extended to the stochastic context, yielding adaptive sample-size rules for subsampling methods typical of machine learning.Comment: 32 page

Updating constraint preconditioners for KKT systems in quadratic programming via low-rank corrections

This work focuses on the iterative solution of sequences of KKT linear systems arising in interior point methods applied to large convex quadratic programming problems. This task is the computational core of the interior point procedure and an efficient preconditioning strategy is crucial for the efficiency of the overall method. Constraint preconditioners are very effective in this context; nevertheless, their computation may be very expensive for large-scale problems, and resorting to approximations of them may be convenient. Here we propose a procedure for building inexact constraint preconditioners by updating a "seed" constraint preconditioner computed for a KKT matrix at a previous interior point iteration. These updates are obtained through low-rank corrections of the Schur complement of the (1,1) block of the seed preconditioner. The updated preconditioners are analyzed both theoretically and computationally. The results obtained show that our updating procedure, coupled with an adaptive strategy for determining whether to reinitialize or update the preconditioner, can enhance the performance of interior point methods on large problems.Comment: 22 page

The GASMEMS network: Rationale, programme and initial results

This paper was presented at the 2nd Micro and Nano Flows Conference (MNF2009), which was held at Brunel University, West London, UK. The conference was organised by Brunel University and supported by the Institution of Mechanical Engineers, IPEM, the Italian Union of Thermofluid dynamics, the Process Intensification Network, HEXAG - the Heat Exchange Action Group and the Institute of Mathematics and its Applications.GASMEMS is an Initial Training Network supported by the European Commission, which aims at training young researchers in the field of rarefied gas flows in MEMS, and at structuring research in Europe in the field of gas microflows in order to improve global fundamental knowledge and enable technological applications to an industrial and commercial level. The partners and the global objectives of this 4 year programme are detailed, and some initial results are presented. First experimental data about the flow of binary gas mixtures through rectangular microchannels are successfully compared with continuum and kinetic models, in the slip flow and early transition regimes. The behaviour of these mixtures has also been simulated in triangular microchannels, for the whole range of the Knudsen number, using a kinetic approach and the McCormack model. Heat transfer in plane microchannels has been numerically investigated, pointing out compressibility and rarefaction effects. The effect of thermal creep has been studied comparing BGK, Smodel and ellipsoidal model with the solution from the full Boltzmann equation. A semi-analytical model of the Knudsen layer has been developed and used to simulate the problem of thermal transpiration in a microchannel. Gaseous flows through rough microchannels have been simulated using kinetic theory and DSMC method, the wall roughness being simulated as a highly porous medium of variable thickness.This study is funded by the European Community's Seventh Framework Programme FP7/2007-2013 under grant agreement ITN GASMEMS n° 215504

On the Convergence Properties of a Stochastic Trust-Region Method with Inexact Restoration

We study the convergence properties of SIRTR, a stochastic inexact restoration trust-region method suited for the minimization of a finite sum of continuously differentiable functions. This method combines the trust-region methodology with random function and gradient estimates formed by subsampling. Unlike other existing schemes, it forces the decrease of a merit function by combining the function approximation with an infeasibility term, the latter of which measures the distance of the current sample size from its maximum value. In a previous work, the expected iteration complexity to satisfy an approximate first-order optimality condition was given. Here, we elaborate on the convergence analysis of SIRTR and prove its convergence in probability under suitable accuracy requirements on random function and gradient estimates. Furthermore, we report the numerical results obtained on some nonconvex classification test problems, discussing the impact of the probabilistic requirements on the selection of the sample sizes

Association between diverticulosis and colonic neoplastic lesions in individuals with a positive faecal immunochemical test

Background The association between diverticulosis and colonic neoplastic lesions has been suggested, but data in literature are conflicting. This study aimed to investigate such a relationship in patients participating in a colorectal cancer screening program who underwent high-quality colonoscopy.Methods Data from consecutive individuals 50-75 years of age with a positive faecal immunological test were considered. Diverticulosis was categorised as present or absent. The prevalence of neoplastic lesions (adenoma, advanced adenoma, and cancer) between individuals with and those without diverticula was compared. A multivariate analysis was performed.Results Overall, data from 970 consecutive individuals were evaluated, and diverticulosis was detected in 354 (36.5%) cases. At least one adenoma was detected in 490 (50.5%) people, at least one advanced adenoma in 264 (27.2%), multiple adenoma in 71 (7.3%), whilst a cancer was diagnosed in 48 (4.9%) cases. At univariate analysis, the adenoma detection rate in patients with diverticula was significantly higher than in controls (55.9% vs 47.4%; p=0.011). At multivariate analysis, presence of diverticulosis was an independent risk factor for both adenoma detection rate (OR=1.58; 95% CI=1.14-2.18; p=0.006) and advanced adenoma (OR=1.57; 95% CI=1.10-2.24; p=0.013), but not for colorectal cancer.Conclusions In a colorectal screening setting, the adenoma detection rate was significantly higher in individuals with diverticulosis than in controls