Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information

We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve $\epsilon$-approximate second-order optimality which have shown to be tight. Our Hessian approximation conditions constitute a major relaxation over the existing ones in the literature. Consequently, we are able to show that such mild conditions allow for the construction of the approximate Hessian through various random sampling methods. In this light, we consider the canonical problem of finite-sum minimization, provide appropriate uniform and non-uniform sub-sampling strategies to construct such Hessian approximations, and obtain optimal iteration complexity for the corresponding sub-sampled trust-region and cubic regularization methods.Comment: 32 page

Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver

We consider a variant of inexact Newton Method, called Newton-MR, in which the least-squares sub-problems are solved approximately using Minimum Residual method. By construction, Newton-MR can be readily applied for unconstrained optimization of a class of non-convex problems known as invex, which subsumes convexity as a sub-class. For invex optimization, instead of the classical Lipschitz continuity assumptions on gradient and Hessian, Newton-MR's global convergence can be guaranteed under a weaker notion of joint regularity of Hessian and gradient. We also obtain Newton-MR's problem-independent local convergence to the set of minima. We show that fast local/global convergence can be guaranteed under a novel inexactness condition, which, to our knowledge, is much weaker than the prior related works. Numerical results demonstrate the performance of Newton-MR as compared with several other Newton-type alternatives on a few machine learning problems.Comment: 35 page

While Others Are Building Castles In The Air : I\u27ll Build A Cottage For Two

https://digitalcommons.library.umaine.edu/mmb-vp/2745/thumbnail.jp

A PAC-Bayesian Perspective on the Interpolating Information Criterion

Deep learning is renowned for its theory-practice gap, whereby principled theory typically fails to provide much beneficial guidance for implementation in practice. This has been highlighted recently by the benign overfitting phenomenon: when neural networks become sufficiently large to interpolate the dataset perfectly, model performance appears to improve with increasing model size, in apparent contradiction with the well-known bias-variance tradeoff. While such phenomena have proven challenging to theoretically study for general models, the recently proposed Interpolating Information Criterion (IIC) provides a valuable theoretical framework to examine performance for overparameterized models. Using the IIC, a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence generalization performance in the interpolating regime. From the provided bound, we quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, optimizer, and parameter-initialization scheme; the spectrum of the empirical neural tangent kernel; curvature of the loss landscape; and noise present in the data.Comment: 9 page

Investigation of Blade-row Flow Distributions in Axial-flow-compressor Stage Consisting of Guide Vanes and Rotor-blade Row

A 30-inch tip-diameter axial-flow compressor stage was investigated with and without rotor to determine individual blade-row performance, interblade-row effects, and outer-wall boundary-layer conditions. Velocity gradients at guide-vane outlet without rotor approximated design assumptions, when the measured variation of leaving angle was considered. With rotor in operation, Mach number and rotor-blade effects changed flow distribution leaving guide vanes and invalidated design assumption of radial equilibrium. Rotor-blade performance correlated interpolated two-dimensional results within 2 degrees, although tip stall was indicated in experimental and not two-dimensional results. Boundary-displacement thickness was less than 1.0 and 1.5 percent of passage height after guide vanes and after rotor, respectively, but increased rapidly after rotor when tip stall occurred

Overcoming the barriers to e-cluster development in a low product complexity business sector

Purpose – The purpose of this paper is to investigate the issues that impact negatively on e-cluster development in a low product complexity industry and identification of key factors to overcome the barriers. Design/methodology/approach – Structured interviews were used to identify perceived value and user expectations from e-clusters. Workshops involving assessment of a prototype e-cluster validated user expectations. A mapping study and best practice review provided a basis for e-cluster application development and assessing potential industry uptake. Findings – Interest and perceived value of e-clusters varied according to size of organisation with smaller organisations primarily interested in e-connectivity to retailers and e-business development. Organisations of all sizes, however, indicated a willingness to learn from each other and partner although level of e-connectivity was average and overall level of sophistication was low. Practical implications – Industrial review and acceptance of a prototype e-cluster that would enable organisations manage several critical aspects of their operations from a single interface. Originality/value – The paper provides new understanding of key issues that impact the operational benefits of e-clusters and, in particular, factors that would underpin the success of e-cluster success in a competitive, insular, low product complexity industry. This presents an informed basis for e-cluster managers and members to successfully manage their initiative

Inducing institutional change through projects? : Three models of projectified governance

The study of short-term projects to implement policy has lately gained ground among scholars of environmental governance and public administration. The increasing reliance on and prevalence of projects, or ‘projectification’, has spurred critical debates on the ability of projects to contribute to long-term goals, including sustainability, as well as institutional change. Yet, the literature on projectification lacks specificity in terms of how projects are understood, how the relationship between projects and permanent organizations looks like, and how projects can influence institutional orders. The aim of this paper is to systematize the literature in order to uncover the process of transforming project outputs into institutional change. Three models of projectified governance – mechanistic, organic, and adaptive – is presented, providing a conceptual apparatus that advances the study of projects in environmental policy and governance. The paper argues that the adaptive model, with its reliance on multi-scalar networks for the coordination of project activities and knowledge, shows most promise in achieving institutional change to address complex environmental problems.Peer reviewe