A Fast Minimum Degree Algorithm and Matching Lower Bound

The minimum degree algorithm is one of the most widely-used heuristics for reducing the cost of solving large sparse systems of linear equations. It has been studied for nearly half a century and has a rich history of bridging techniques from data structures, graph algorithms, and scientific computing. In this paper, we present a simple but novel combinatorial algorithm for computing an exact minimum degree elimination ordering in $O(nm)$ time, which improves on the best known time complexity of $O(n^3)$ and offers practical improvements for sparse systems with small values of $m$. Our approach leverages a careful amortized analysis, which also allows us to derive output-sensitive bounds for the running time of $O(\min\{m\sqrt{m^+}, \Delta m^+\} \log n)$, where $m^+$ is the number of unique fill edges and original edges that the algorithm encounters and $\Delta$ is the maximum degree of the input graph. Furthermore, we show there cannot exist an exact minimum degree algorithm that runs in $O(nm^{1-\varepsilon})$ time, for any $\varepsilon > 0$, assuming the strong exponential time hypothesis. This fine-grained reduction goes through the orthogonal vectors problem and uses a new low-degree graph construction called $U$-fillers, which act as pathological inputs and cause any minimum degree algorithm to exhibit nearly worst-case performance. With these two results, we nearly characterize the time complexity of computing an exact minimum degree ordering.Comment: 17 page

Graph Sketching Against Adaptive Adversaries Applied to the Minimum Degree Algorithm

Motivated by the study of matrix elimination orderings in combinatorial scientific computing, we utilize graph sketching and local sampling to give a data structure that provides access to approximate fill degrees of a matrix undergoing elimination in $O(\text{polylog}(n))$ time per elimination and query. We then study the problem of using this data structure in the minimum degree algorithm, which is a widely-used heuristic for producing elimination orderings for sparse matrices by repeatedly eliminating the vertex with (approximate) minimum fill degree. This leads to a nearly-linear time algorithm for generating approximate greedy minimum degree orderings. Despite extensive studies of algorithms for elimination orderings in combinatorial scientific computing, our result is the first rigorous incorporation of randomized tools in this setting, as well as the first nearly-linear time algorithm for producing elimination orderings with provable approximation guarantees. While our sketching data structure readily works in the oblivious adversary model, by repeatedly querying and greedily updating itself, it enters the adaptive adversarial model where the underlying sketches become prone to failure due to dependency issues with their internal randomness. We show how to use an additional sampling procedure to circumvent this problem and to create an independent access sequence. Our technique for decorrelating the interleaved queries and updates to this randomized data structure may be of independent interest.Comment: 58 pages, 3 figures. This is a substantially revised version of arXiv:1711.08446 with an emphasis on the underlying theoretical problem

Posimodular Function Optimization

Given a posimodular function $f: 2^V \to \mathbb{R}$ on a finite set $V$, we consider the problem of finding a nonempty subset $X$ of $V$ that minimizes $f(X)$. Posimodular functions often arise in combinatorial optimization such as undirected cut functions. In this paper, we show that any algorithm for the problem requires $\Omega(2^{\frac{n}{7.54}})$ oracle calls to $f$, where $n=|V|$. It contrasts to the fact that the submodular function minimization, which is another generalization of cut functions, is polynomially solvable. When the range of a given posimodular function is restricted to be $D=\{0,1,...,d\}$ for some nonnegative integer $d$, we show that $\Omega(2^{\frac{d}{15.08}})$ oracle calls are necessary, while we propose an $O(n^dT_f+n^{2d+1})$-time algorithm for the problem. Here, $T_f$ denotes the time needed to evaluate the function value $f(X)$ for a given $X \subseteq V$. We also consider the problem of maximizing a given posimodular function. We show that $\Omega(2^{n-1})$ oracle calls are necessary for solving the problem, and that the problem has time complexity $\Theta(n^{d-1}T_f)$ when $D=\{0,1,..., d\}$ is the range of $f$ for some constant $d$.Comment: 18 page

Maximizing Symmetric Submodular Functions

Symmetric submodular functions are an important family of submodular functions capturing many interesting cases including cut functions of graphs and hypergraphs. Maximization of such functions subject to various constraints receives little attention by current research, unlike similar minimization problems which have been widely studied. In this work, we identify a few submodular maximization problems for which one can get a better approximation for symmetric objectives than the state of the art approximation for general submodular functions. We first consider the problem of maximizing a non-negative symmetric submodular function $f\colon 2^\mathcal{N} \to \mathbb{R}^+$ subject to a down-monotone solvable polytope $\mathcal{P} \subseteq [0, 1]^\mathcal{N}$. For this problem we describe an algorithm producing a fractional solution of value at least $0.432 \cdot f(OPT)$, where $OPT$ is the optimal integral solution. Our second result considers the problem $\max \{f(S) : |S| = k\}$ for a non-negative symmetric submodular function $f\colon 2^\mathcal{N} \to \mathbb{R}^+$. For this problem, we give an approximation ratio that depends on the value $k / |\mathcal{N}|$ and is always at least $0.432$. Our method can also be applied to non-negative non-symmetric submodular functions, in which case it produces $1/e - o(1)$ approximation, improving over the best known result for this problem. For unconstrained maximization of a non-negative symmetric submodular function we describe a deterministic linear-time $1/2$-approximation algorithm. Finally, we give a $[1 - (1 - 1/k)^{k - 1}]$-approximation algorithm for Submodular Welfare with $k$ players having identical non-negative submodular utility functions, and show that this is the best possible approximation ratio for the problem.Comment: 31 pages, an extended abstract appeared in ESA 201

실시간 의복 시뮬레이션을 위한 선형 모델 연구

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2013. 8. 고형석.옷에서 일어나는 변형은 크게 평면 내 변형과 평면 외 변형으로 나눌 수 있다. 인장과 전단이 평면 내 변형, 굽힘이 평면 외 변형에 속한다. 의류 시뮬레이션은 위 세 가지 변형을 모두 포함한다. 본 논문에서는 옷의 변형에 대한 새로운 물리 모델을 제시한다. 본 논문에서 제시하는 모델의 의의는 그것의 수치적 시뮬레이션이 실시간에 이루어질 수 있다는 점과 기존의 실시간 모델에 존재했던 몇가지 결함을 해결함으로써 시뮬레이션 결과에서 보였던 문제점들을 해결했다는 점에 있다. 본 논문이 새로운 물리 모델을 개발함에 있어 주요한 아이디어는 에너지 함수에 존재하는 (x-C)^2 항을 x^* 라는 상수 벡터를 도입하여x-x^*^2 라는 항으로 바꾼 데 있다. 이렇게 함으로써 힘 자코비안 행렬을 상수로 만들고 그에 따라 시스템 행렬 역시 상수로 만든다. 그 결과 시스템 행렬의 역행렬을 시뮬레이션 시작 전 사전 계산 시간에 미리 구할 수 있고, 내연적 시뮬레이션 진행 과정에서 시스템 행렬을 매번 새로 구성하고 해를 구해야 했던 과정을 단순한 행렬과 벡터의 곱셈으로 대체할 수 있다. 본 논문은 이러한 선형 물리 모델을 선분 기반 시스템과 삼각형 기반 시스템에 대해 제시한다. 추가적으로 행렬과 벡터 곱셈 과정의 속도를 향상하기 위해 최신의 희소 촐레스키 분해 방법을 살펴보고 의상 시뮬레이션에 효과적인 적용 방법을 소개한다.Deformations occurring in cloth can be decomposed into two components: the in-plane and the out-of-plane deformations. Stretch and shear are in-plane deformation, and bending is out-of-plane deformation. Clothing simulation involves all the above types of deformations. This paper proposes a new physical model for deformations of clothes. The significance of the proposed models is that (1) their numerical simulation can be done in real-time, and (2) the models fix some flaws that existed in previous real-time models, leading to conspicuous reduction of artifacts. The essential idea in inventing the new models is to replace (-C)^2 in the energy function with^2 for some constant vector x^*. Then, the force jacobian becomes a constant, and so does the system matrix. As a result, its inverse matrix can be pre-computed only once in off-line, so that the on-line semi-implicit integration can be replaced with (the constant) matrix-vector multiplications. This paper develops such simplified physical models for both edge-based and triangle-based systems. In addition, to speed up the process of matrix-vector multiplications, this work reviews the current state-of-the-art in the Sparse Cholesky factorization methods and introduces an effective method for the current purpose.Abstract i Contents iii List of Figures v List of Tables vii 1 Introduction 1 1.1 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Edge-Based Formulation of Stretch Energy and Force . . . . . 4 1.3 Explicit Formulation . . . . . . . . . . . . . . . . . . . . . . . . 5 1.4 Implicit Formulation . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Related Work 9 3 Edge-Based Linear Stretch Model 13 3.1 Conventional Stretch Model . . . . . . . . . . . . . . . . . . . . 14 3.2 Our Stretch Model . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.3 Representation of Shear Deformations . . . . . . . . . . . . . . 20 3.4 A Killer Application of This Model . . . . . . . . . . . . . . . 21 4 Triangle-Based Linear Stretch/Shear Model 22 4.1 Material Space to 3D Space Mapping S . . . . . . . . . . . . . 23 4.2 Conventional Stretch and Shear Model . . . . . . . . . . . . . . 24 4.3 Our Stretch and Shear Model . . . . . . . . . . . . . . . . . . . 24 5 Linear Bending Model 28 5.1 Calculating Bending Vector . . . . . . . . . . . . . . . . . . . . 28 5.2 Applying Bending Force . . . . . . . . . . . . . . . . . . . . . . 30 5.3 Jacobian of the Bending Force . . . . . . . . . . . . . . . . . . 31 6 Sparse Cholesky Factorization 32 6.1 Cholesky Factorization . . . . . . . . . . . . . . . . . . . . . . . 32 6.2 Reordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 7 Experimental Results 48 8 Conclusion 65 Bibliography 67 초록 71Docto

Estimating spatial covariance using penalised likelihood with weighted L1 penalty

In spatial statistics, the estimation of covariance matrices is of great importance because of its role in spatial prediction and design. In this paper, we propose a penalised likelihood approach with weighted L 1 regularisation to estimate the covariance matrix for spatial Gaussian Markov random field models with unspecified neighbourhood structures. A new algorithm for ordering spatial points is proposed such that the corresponding precision matrix can be estimated more effectively. Furthermore, we develop an efficient algorithm to minimise the penalised likelihood via a novel usage of the regularised solution path algorithm, which does not require the use of iterative algorithms. By exploiting the sparsity structure in the precision matrix, we show that the LASSO type of approach gives improved covariance estimators measured by several criteria. Asymptotic properties of our proposed estimator are derived. Both our simulated examples and an application to the rainfall data set show that the proposed method performs competitively

A Schur complement approach to preconditioning sparse linear least-squares problems with some dense rows

The effectiveness of sparse matrix techniques for directly solving large-scale linear least-squares problems is severely limited if the system matrix A has one or more nearly dense rows. In this paper, we partition the rows of A into sparse rows and dense rows (A s and A d ) and apply the Schur complement approach. A potential difficulty is that the reduced normal matrix AsTA s is often rank-deficient, even if A is of full rank. To overcome this, we propose explicitly removing null columns of A s and then employing a regularization parameter and using the resulting Cholesky factors as a preconditioner for an iterative solver applied to the symmetric indefinite reduced augmented system. We consider complete factorizations as well as incomplete Cholesky factorizations of the shifted reduced normal matrix. Numerical experiments are performed on a range of large least-squares problems arising from practical applications. These demonstrate the effectiveness of the proposed approach when combined with either a sparse parallel direct solver or a robust incomplete Cholesky factorization algorithm