A Characterization Theorem and An Algorithm for A Convex Hull Problem

Given $S= \{v_1, \dots, v_n\} \subset \mathbb{R} ^m$ and $p \in \mathbb{R} ^m$, testing if $p \in conv(S)$, the convex hull of $S$, is a fundamental problem in computational geometry and linear programming. First, we prove a Euclidean {\it distance duality}, distinct from classical separation theorems such as Farkas Lemma: $p$ lies in $conv(S)$ if and only if for each $p' \in conv(S)$ there exists a {\it pivot}, $v_j \in S$ satisfying $d(p',v_j) \geq d(p,v_j)$. Equivalently, $p \not \in conv(S)$ if and only if there exists a {\it witness}, $p' \in conv(S)$ whose Voronoi cell relative to $p$ contains $S$. A witness separates $p$ from $conv(S)$ and approximate $d(p, conv(S))$ to within a factor of two. Next, we describe the {\it Triangle Algorithm}: given $\epsilon \in (0,1)$, an {\it iterate}, $p' \in conv(S)$, and $v \in S$, if $d(p, p') < \epsilon d(p,v)$, it stops. Otherwise, if there exists a pivot $v_j$, it replace $v$ with $v_j$ and $p'$ with the projection of $p$ onto the line $p'v_j$. Repeating this process, the algorithm terminates in $O(mn \min \{\epsilon^{-2}, c^{-1}\ln \epsilon^{-1} \})$ arithmetic operations, where $c$ is the {\it visibility factor}, a constant satisfying $c \geq \epsilon^2$ and $\sin (\angle pp'v_j) \leq 1/\sqrt{1+c}$, over all iterates $p'$. Additionally, (i) we prove a {\it strict distance duality} and a related minimax theorem, resulting in more effective pivots; (ii) describe $O(mn \ln \epsilon^{-1})$-time algorithms that may compute a witness or a good approximate solution; (iii) prove {\it generalized distance duality} and describe a corresponding generalized Triangle Algorithm; (iv) prove a {\it sensitivity theorem} to analyze the complexity of solving LP feasibility via the Triangle Algorithm. The Triangle Algorithm is practical and competitive with the simplex method, sparse greedy approximation and first-order methods.Comment: 42 pages, 17 figures, 2 tables. This revision only corrects minor typo

A Family of Iteration Functions for General Linear Systems

We develop novel theory and algorithms for computing approximate solution to $Ax=b$, or to $A^TAx=A^Tb$, where $A$ is an $m \times n$ real matrix of arbitrary rank. First, we describe the {\it Triangle Algorithm} (TA), where given an ellipsoid $E_{A,\rho}=\{Ax: \Vert x \Vert \leq \rho\}$, in each iteration it either computes successively improving approximation $b_k=Ax_k \in E_{A,\rho}$, or proves $b \not \in E_{A, \rho}$. We then extend TA for computing an approximate solution or minimum-norm solution. Next, we develop a dynamic version of TA, the {\it Centering Triangle Algorithm} (CTA), generating residuals $r_k=b - Ax_k$ via iterations of the simple formula, $F_1(r)=r-(r^THr/r^TH^2r)Hr$, where $H=A$ when $A$ is symmetric PSD, otherwise $H=AA^T$ but need not be computed explicitly. More generally, CTA extends to a family of iteration function, $F_t( r)$, $t=1, \dots, m$ satisfying: On the one hand, given $t \leq m$ and $r_0=b-Ax_0$, where $x_0=A^Tw_0$ with $w_0 \in \mathbb{R}^m$ arbitrary, for all $k \geq 1$, $r_k=F_t(r_{k-1})=b-Ax_k$ and $A^Tr_k$ converges to zero. Algorithmically, if $H$ is invertible with condition number $\kappa$, in $k=O( (\kappa/t) \ln \varepsilon^{-1})$ iterations $\Vert r_k \Vert \leq \varepsilon$. If $H$ is singular with $\kappa^+$ the ratio of its largest to smallest positive eigenvalues, in $k =O(\kappa^+/t\varepsilon)$ iterations either $\Vert r_k \Vert \leq \varepsilon$ or $\Vert A^T r_k\Vert= O(\sqrt{\varepsilon})$. If $N$ is the number of nonzero entries of $A$, each iteration take $O(Nt+t^3)$ operations. On the other hand, given $r_0=b-Ax_0$, suppose its minimal polynomial with respect to $H$ has degree $s$. Then $Ax=b$ is solvable if and only if $F_{s}(r_0)=0$. Moreover, exclusively $A^TAx=A^Tb$ is solvable, if and only if $F_{s}(r_0) \not= 0$ but $A^T F_s(r_0)=0$. Additionally, $\{F_t(r_0)\}_{t=1}^s$ is computable in $O(Ns+s^3)$ operations.Comment: 59 pages, 4 figure