On the removal of ill conditioning effects in the computation of optimal controls

Ill conditioning effects eliminated in nonlinear programming algorithms for optimal control

Ill-conditioning and multicollinearity

AbstractIt is well known that unstability of solutions to small changes in inputs causes many problems in numerical computations. Existence, uniqueness and stability of solutions are important features of mathematical problems. Problems that fail to satisfy these conditions are called ill-posed. The purpose of this study is to remind briefly some methods of solution to ill-posed problems and to see the impacts or connections of these techniques to some statistical methods

Estimate of convection-diffusion coefficients from modulated perturbative experiments as an inverse problem

The estimate of coefficients of the Convection-Diffusion Equation (CDE) from experimental measurements belongs in the category of inverse problems, which are known to come with issues of ill-conditioning or singularity. Here we concentrate on a particular class that can be reduced to a linear algebraic problem, with explicit solution. Ill-conditioning of the problem corresponds to the vanishing of one eigenvalue of the matrix to be inverted. The comparison with algorithms based upon matching experimental data against numerical integration of the CDE sheds light on the accuracy of the parameter estimation procedures, and suggests a path for a more precise assessment of the profiles and of the related uncertainty. Several instances of the implementation of the algorithm to real data are presented.Comment: Extended version of an invited talk presented at the 2012 EPS Conference. To appear in Plasma Physics and Controlled Fusio

Ridge Regression and Ill-Conditioning

Hoerl and Kennard (1970) suggested the ridge regression estimator as an alternative to the Ordinary Least Squares (OLS) estimator in the presence of multicollinearity. This article proposes new methods for estimating the ridge parameter in case of ordinary ridge regression. A simulation study evaluates the performance of the proposed estimators based on the Mean Squared Error (MSE) criterion and indicates that, under certain conditions, the proposed estimators perform well compared to the OLS estimator and another well-known estimator reviewed

Electrostatic Point Charge Fitting as an Inverse Problem: Revealing the Underlying Ill-Conditioning

Atom-centered point charge model of the molecular electrostatics---a major workhorse of the atomistic biomolecular simulations---is usually parameterized by least-squares (LS) fitting of the point charge values to a reference electrostatic potential, a procedure that suffers from numerical instabilities due to the ill-conditioned nature of the LS problem. Here, to reveal the origins of this ill-conditioning, we start with a general treatment of the point charge fitting problem as an inverse problem, and construct an analytically soluble model with the point charges spherically arranged according to Lebedev quadrature naturally suited for the inverse electrostatic problem. This analytical model is contrasted to the atom-centered point-charge model that can be viewed as an irregular quadrature poorly suited for the problem. This analysis shows that the numerical problems of the point charge fitting are due to the decay of the curvatures corresponding to the eigenvectors of LS sum Hessian matrix. In part, this ill-conditioning is intrinsic to the problem and related to decreasing electrostatic contribution of the higher multipole moments, that are, in the case of Lebedev grid model, directly associated with the Hessian eigenvectors. For the atom-centered model, this association breaks down beyond the first few eigenvectors related to the high-curvature monopole and dipole terms; this leads to even wider spread-out of the Hessian curvature values. Using these insights, it is possible to alleviate the ill-conditioning of the LS point-charge fitting without introducing external restraints and/or constraints. Also, as the analytical Lebedev grid PC model proposed here can reproduce multipole moments up to a given rank, it may provide a promising alternative to including explicit multipole terms in a force field

Condition number analysis and preconditioning of the finite cell method

The (Isogeometric) Finite Cell Method - in which a domain is immersed in a structured background mesh - suffers from conditioning problems when cells with small volume fractions occur. In this contribution, we establish a rigorous scaling relation between the condition number of (I)FCM system matrices and the smallest cell volume fraction. Ill-conditioning stems either from basis functions being small on cells with small volume fractions, or from basis functions being nearly linearly dependent on such cells. Based on these two sources of ill-conditioning, an algebraic preconditioning technique is developed, which is referred to as Symmetric Incomplete Permuted Inverse Cholesky (SIPIC). A detailed numerical investigation of the effectivity of the SIPIC preconditioner in improving (I)FCM condition numbers and in improving the convergence speed and accuracy of iterative solvers is presented for the Poisson problem and for two- and three-dimensional problems in linear elasticity, in which Nitche's method is applied in either the normal or tangential direction. The accuracy of the preconditioned iterative solver enables mesh convergence studies of the finite cell method

Deliberate Ill-Conditioning of Krylov Matrices

This paper starts o with studying simple extrapolation methods for the classical iteration schemes such as Richardson, Jacobi and Gauss-Seidel iteration. The extrapolation procedures can be interpreted as approximate minimal residual methods in a Krylov subspace. It seems therefore logical to consider, conversely, classical methods as pre-processors for Krylov subspace methods, as was done by Ztko (1996) for the Conjugate Gradient method. The observation made by Ipsen (1998) that small residuals necessarily imply an ill-conditioned Krylov matrix, explains the success of such pre-processing schemes: residuals of classical methods are (unscaled) power method iterates, and building a Krylov subspace on such a classical residual will therefore lead to expansion vectors that are at small angle to the previous Krylov vectors. This results in an ill-conditioned Krylov matrix. In this paper, we present a largenumber of experiments that support this claim, and give theoretical interpretations of the pre-processing. The results are mainly of interest in Krylov subspace methods for non-Hermitian matrices based on long recurrences, and in particular for applications with heavy memory limitations. Also, in applications in which minimal residual methods stagnate due to a lack of ill-conditioning, the use of a classical preprocessor can be a cheap and easily parallelizable remedy