Minimizing Communication for Eigenproblems and the Singular Value Decomposition

Algorithms have two costs: arithmetic and communication. The latter represents the cost of moving data, either between levels of a memory hierarchy, or between processors over a network. Communication often dominates arithmetic and represents a rapidly increasing proportion of the total cost, so we seek algorithms that minimize communication. In \cite{BDHS10} lower bounds were presented on the amount of communication required for essentially all $O(n^3)$-like algorithms for linear algebra, including eigenvalue problems and the SVD. Conventional algorithms, including those currently implemented in (Sca)LAPACK, perform asymptotically more communication than these lower bounds require. In this paper we present parallel and sequential eigenvalue algorithms (for pencils, nonsymmetric matrices, and symmetric matrices) and SVD algorithms that do attain these lower bounds, and analyze their convergence and communication costs.Comment: 43 pages, 11 figure

Sensitivity of Markov chains for wireless protocols

Network communication protocols such as the IEEE 802.11 wireless protocol are currently best modelled as Markov chains. In these situations we have some protocol parameters $\alpha$, and a transition matrix $P(\alpha)$ from which we can compute the steady state (equilibrium) distribution $z(\alpha)$ and hence final desired quantities $q(\alpha)$, which might be for example the throughput of the protocol. Typically the chain will have thousands of states, and a particular example of interest is the Bianchi chain defined later. Generally we want to optimise $q$, perhaps subject to some constraints that also depend on the Markov chain. To do this efficiently we need the gradient of $q$ with respect to $\alpha$, and therefore need the gradient of $z$ and other properties of the chain with respect to $\alpha$. The matrix formulas available for this involve the so-called fundamental matrix, but are there approximate gradients available which are faster and still sufficiently accurate? In some cases BT would like to do the whole calculation in computer algebra, and get a series expansion of the equilibrium $z$ with respect to a parameter in $P$. In addition to the steady state $z$, the same questions arise for the mixing time and the mean hitting times. Two qualitative features that were brought to the Study Group’s attention were: * the transition matrix $P$ is large, but sparse. * the systems of linear equations to be solved are generally singular and need some additional normalisation condition, such as is provided by using the fundamental matrix. We also note a third highly important property regarding applications of numerical linear algebra: * the transition matrix $P$ is asymmetric. A realistic dimension for the matrix $P$ in the Bianchi model described below is 8064×8064, but on average there are only a few nonzero entries per column. Merely storing such a large matrix in dense form would require nearly 0.5GBytes using 64-bit floating point numbers, and computing its LU factorisation takes around 80 seconds on a modern microprocessor. It is thus highly desirable to employ specialised algorithms for sparse matrices. These algorithms are generally divided between those only applicable to symmetric matrices, the most prominent being the conjugate-gradient (CG) algorithm for solving linear equations, and those applicable to general matrices. A similar division is present in the literature on numerical eigenvalue problems

Exact Algorithm for Sampling the 2D Ising Spin Glass

A sampling algorithm is presented that generates spin glass configurations of the 2D Edwards-Anderson Ising spin glass at finite temperature, with probabilities proportional to their Boltzmann weights. Such an algorithm overcomes the slow dynamics of direct simulation and can be used to study long-range correlation functions and coarse-grained dynamics. The algorithm uses a correspondence between spin configurations on a regular lattice and dimer (edge) coverings of a related graph: Wilson's algorithm [D. B. Wilson, Proc. 8th Symp. Discrete Algorithms 258, (1997)] for sampling dimer coverings on a planar lattice is adapted to generate samplings for the dimer problem corresponding to both planar and toroidal spin glass samples. This algorithm is recursive: it computes probabilities for spins along a "separator" that divides the sample in half. Given the spins on the separator, sample configurations for the two separated halves are generated by further division and assignment. The algorithm is simplified by using Pfaffian elimination, rather than Gaussian elimination, for sampling dimer configurations. For n spins and given floating point precision, the algorithm has an asymptotic run-time of O(n^{3/2}); it is found that the required precision scales as inverse temperature and grows only slowly with system size. Sample applications and benchmarking results are presented for samples of size up to n=128^2, with fixed and periodic boundary conditions.Comment: 18 pages, 10 figures, 1 table; minor clarification

Multigrid waveform relaxation for the time-fractional heat equation

In this work, we propose an efficient and robust multigrid method for solving the time-fractional heat equation. Due to the nonlocal property of fractional differential operators, numerical methods usually generate systems of equations for which the coefficient matrix is dense. Therefore, the design of efficient solvers for the numerical simulation of these problems is a difficult task. We develop a parallel-in-time multigrid algorithm based on the waveform relaxation approach, whose application to time-fractional problems seems very natural due to the fact that the fractional derivative at each spatial point depends on the values of the function at this point at all earlier times. Exploiting the Toeplitz-like structure of the coefficient matrix, the proposed multigrid waveform relaxation method has a computational cost of $O(N M \log(M))$ operations, where $M$ is the number of time steps and $N$ is the number of spatial grid points. A semi-algebraic mode analysis is also developed to theoretically confirm the good results obtained. Several numerical experiments, including examples with non-smooth solutions and a nonlinear problem with applications in porous media, are presented