Error Analysis of the Quasi-Gram--Schmidt Algorithm

Let the $n{\times}p$ $(n\geq p)$ matrix $X$ have the QR~factorization $X = QR$, where $R$ is an upper triangular matrix of order $p$ and $Q$ is orthonormal. This widely used decomposition has the drawback that $Q$ is not generally sparse even when $X$ is. One cure is to discard $Q$ retaining only $X$ and $R$. Products like a = Q\trp y = R\itp X\trp y can then be formed by computing b = X\trp y and solving the system R\trp a = b. This approach can be used to modify the Gram--Schmidt algorithm for computing $Q$ and $R$ to compute $R$ without forming $Q$ or altering $X$. Unfortunately, this quasi-Gram--Schmidt algorithm can produce inaccurate results. In this paper it is shown that with reorthogonalization the inaccuracies are bounded under certain natural conditions. (UMIACS-TR-2004-17

Condition number analysis and preconditioning of the finite cell method

The (Isogeometric) Finite Cell Method - in which a domain is immersed in a structured background mesh - suffers from conditioning problems when cells with small volume fractions occur. In this contribution, we establish a rigorous scaling relation between the condition number of (I)FCM system matrices and the smallest cell volume fraction. Ill-conditioning stems either from basis functions being small on cells with small volume fractions, or from basis functions being nearly linearly dependent on such cells. Based on these two sources of ill-conditioning, an algebraic preconditioning technique is developed, which is referred to as Symmetric Incomplete Permuted Inverse Cholesky (SIPIC). A detailed numerical investigation of the effectivity of the SIPIC preconditioner in improving (I)FCM condition numbers and in improving the convergence speed and accuracy of iterative solvers is presented for the Poisson problem and for two- and three-dimensional problems in linear elasticity, in which Nitche's method is applied in either the normal or tangential direction. The accuracy of the preconditioned iterative solver enables mesh convergence studies of the finite cell method

Practical implementation and error bounds of integer-type general algorithm for higher order differential equations

In our preceding paper, we have proposed an algorithm for obtaining finite-norm solutions of higher-order linear ordinary differential equations of the Fuchsian type [\sum_m p_m (x) (d/dx)^m] f(x) = 0 (where p_m is a polynomial with rational-number-valued coefficients), by using only the four arithmetical operations on integers, and we proved its validity. For any nonnegative integer k, it is guaranteed mathematically that this method can produce all the solutions satisfying \int |f(x)|^2 (x^2+1)^k dx < \infty, under some conditions. We materialize this algorithm in practical procedures. An interger-type quasi-orthogonalization used there can suppress the explosion of calculations. Moreover, we give an upper limit of the errors. We also give some results of numerical experiments and compare them with the corresponding exact analytical solutions, which show that the proposed algorithm is successful in yielding solutions with high accuracy (using only arithmetical operations on integers).Comment: Comparison with existing method is adde

Evaluating the Impact of SDC on the GMRES Iterative Solver

Increasing parallelism and transistor density, along with increasingly tighter energy and peak power constraints, may force exposure of occasionally incorrect computation or storage to application codes. Silent data corruption (SDC) will likely be infrequent, yet one SDC suffices to make numerical algorithms like iterative linear solvers cease progress towards the correct answer. Thus, we focus on resilience of the iterative linear solver GMRES to a single transient SDC. We derive inexpensive checks to detect the effects of an SDC in GMRES that work for a more general SDC model than presuming a bit flip. Our experiments show that when GMRES is used as the inner solver of an inner-outer iteration, it can "run through" SDC of almost any magnitude in the computationally intensive orthogonalization phase. That is, it gets the right answer using faulty data without any required roll back. Those SDCs which it cannot run through, get caught by our detection scheme

Computing covariant vectors, Lyapunov vectors, Oseledets vectors, and dichotomy projectors: a comparative numerical study

Covariant vectors, Lyapunov vectors, or Oseledets vectors are increasingly being used for a variety of model analyses in areas such as partial differential equations, nonautonomous differentiable dynamical systems, and random dynamical systems. These vectors identify spatially varying directions of specific asymptotic growth rates and obey equivariance principles. In recent years new computational methods for approximating Oseledets vectors have been developed, motivated by increasing model complexity and greater demands for accuracy. In this numerical study we introduce two new approaches based on singular value decomposition and exponential dichotomies and comparatively review and improve two recent popular approaches of Ginelli et al. (2007) and Wolfe and Samelson (2007). We compare the performance of the four approaches via three case studies with very different dynamics in terms of symmetry, spectral separation, and dimension. We also investigate which methods perform well with limited data

Contraction and optimality properties of an adaptive Legendre-Galerkin method: the multi-dimensional case

We analyze the theoretical properties of an adaptive Legendre-Galerkin method in the multidimensional case. After the recent investigations for Fourier-Galerkin methods in a periodic box and for Legendre-Galerkin methods in the one dimensional setting, the present study represents a further step towards a mathematically rigorous understanding of adaptive spectral/$hp$ discretizations of elliptic boundary-value problems. The main contribution of the paper is a careful construction of a multidimensional Riesz basis in $H^1$, based on a quasi-orthonormalization procedure. This allows us to design an adaptive algorithm, to prove its convergence by a contraction argument, and to discuss its optimality properties (in the sense of non-linear approximation theory) in certain sparsity classes of Gevrey type

Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields

We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substantially improves practical performance, and analyze the rate of convergence of the SAGA variant under non-uniform sampling. Our experimental results reveal that our method often significantly outperforms existing methods in terms of the training objective, and performs as well or better than optimally-tuned stochastic gradient methods in terms of test error.Comment: AI/Stats 2015, 24 page