On the convergence of the affine-scaling algorithm

Cover title.Includes bibliographical references (p. 20-22).Research partially supported by the National Science Foundation. NSF-ECS-8519058 Research partially supported by the U.S. Army Research Office. DAAL03-86-K-0171 Research partially supported by the Science and Engineering Research Board of McMaster University.by Paul Tseng and Zhi-Quan Luo

Simple and dynamic data structure for pattern matching in texts, A

2011 Summer.Includes bibliographical references.The demand for a pattern matching algorithm is currently on the rise from diverse areas such as string search, image matching, voice recognition and bioinformatics. In particular, string search or matching algorithms have been growing in popularity as they have been applied to areas such as text editors, search engines and bioinformatics. To satisfy these various demands, many string matching methods have been developed to search for substrings (pattern strings) within a text, and several techniques employ the use of tree data structures, deterministic finite automata, and other structures. The problem of string matching is defined by finding all location of a pattern string P within a text T, where preprocessing of T is allowed in order to facilitate the queries. There has been significant success in finding a pattern string in O(m+k) time, where m is the length of the pattern string and k is the number of occurrences, using data structures that can be constructed in O(n) time, where n is the length of T. Suffix trees and directed acyclic word graphs are such data structures. All of these data structures index the searched text in O(m+k) time. However, the difficulty of understanding and programming the construction algorithms is rarely mentioned. Also, they have significant space requirements and take Θ(n) time to update even if one character of T is changed. To solve these problems, we propose the augmented position heap. It can be built in O(n) time, and can be used to search a pattern string in O(m+k) time. Most importantly, when a block of j characters are inserted or deleted, the asymptotic updating it when a text is modified is O((h(T) + j)h(T)), where h(T) is the length of the longest substring X of T that occurs at least ||X|| times in T, where ||X|| is the length of X. For texts arising from practical applications, h(T) is typically slowly growing function of ||T||; for a random text T, its expected value is O(logn). Another issue in data structures that must be addressed is space requirement. The most space efficient data structure for string search is the suffix array, which uses 2n words and supports searches in O(nlogn + m + k). A compact representation of the position heap proposed in this thesis also takes 2n words, but can be updated in O((h(T) + j)h(T)) time, but takes O(m2+k) time for a search. The best bound known bound for updating the suffix array or the directed acyclic word graph is O(n), and they both take considerably more space. A compact representation proposed in this thesis for the augmented position heap takes 4n words, can be updated just as efficiently as the position heap, and takes O(m+k) time for a search

Minkowski Sum Construction and other Applications of Arrangements of Geodesic Arcs on the Sphere

We present two exact implementations of efficient output-sensitive algorithms that compute Minkowski sums of two convex polyhedra in 3D. We do not assume general position. Namely, we handle degenerate input, and produce exact results. We provide a tight bound on the exact maximum complexity of Minkowski sums of polytopes in 3D in terms of the number of facets of the summand polytopes. The algorithms employ variants of a data structure that represents arrangements embedded on two-dimensional parametric surfaces in 3D, and they make use of many operations applied to arrangements in these representations. We have developed software components that support the arrangement data-structure variants and the operations applied to them. These software components are generic, as they can be instantiated with any number type. However, our algorithms require only (exact) rational arithmetic. These software components together with exact rational-arithmetic enable a robust, efficient, and elegant implementation of the Minkowski-sum constructions and the related applications. These software components are provided through a package of the Computational Geometry Algorithm Library (CGAL) called Arrangement_on_surface_2. We also present exact implementations of other applications that exploit arrangements of arcs of great circles embedded on the sphere. We use them as basic blocks in an exact implementation of an efficient algorithm that partitions an assembly of polyhedra in 3D with two hands using infinite translations. This application distinctly shows the importance of exact computation, as imprecise computation might result with dismissal of valid partitioning-motions.Comment: A Ph.D. thesis carried out at the Tel-Aviv university. 134 pages long. The advisor was Prof. Dan Halperi

Primal-dual interior-point algorithms for linear programs with many inequality constraints

Linear programs (LPs) are one of the most basic and important classes of constrained optimization problems, involving the optimization of linear objective functions over sets defined by linear equality and inequality constraints. LPs have applications to a broad range of problems in engineering and operations research, and often arise as subproblems for algorithms that solve more complex optimization problems. ``Unbalanced'' inequality-constrained LPs with many more inequality constraints than variables are an important subclass of LPs. Under a basic non-degeneracy assumption, only a small number of the constraints can be active at the solution--it is only this active set that is critical to the problem description. On the other hand, the additional constraints make the problem harder to solve. While modern ``interior-point'' algorithms have become recognized as some of the best methods for solving large-scale LPs, they may not be recommended for unbalanced problems, because their per-iteration work does not scale well with the number of constraints. In this dissertation, we investigate "constraint-reduced'' interior-point algorithms designed to efficiently solve unbalanced LPs. At each iteration, these methods construct search directions based only on a small working set of constraints, while ignoring the rest. In this way, they significantly reduce their per-iteration work and, hopefully, their overall running time. In particular, we focus on constraint-reduction methods for the highly efficient primal-dual interior-point (PDIP) algorithms. We propose and analyze a convergent constraint-reduced variant of Mehrotra's predictor-corrector PDIP algorithm, the algorithm implemented in virtually every interior-point software package for linear (and convex-conic) programming. We prove global and local quadratic convergence of this algorithm under a very general class of constraint selection rules and under minimal assumptions. We also propose and analyze two regularized constraint-reduced PDIP algorithms (with similar convergence properties) designed to deal directly with a type of degeneracy that constraint-reduced interior-point algorithms are often subject to. Prior schemes for dealing with this degeneracy could end up negating the benefit of constraint-reduction. Finally, we investigate the performance of our algorithms by applying them to several test and application problems, and show that our algorithms often outperform alternative approaches

Cut Techniques in Integer Linear Programming

Higher Educatio

Basis preconditioning in interior point methods

Solving normal equations AAᵀx = b, where A is an m x n matrix, is a common task in numerical optimization. For the efficient use of iterative methods, this thesis studies the class of preconditioners of the form BBᵀ , where B is a nonsingular "basis" matrix composed of m columns of A. It is known that for any matrix A of full row rank B can be chosen so that the entries in [B⁻¹A] are bounded by 1. Such a basis is said to have "maximum volume" and its preconditioner bounds the spectrum of the transformed normal matrix in the interval [1, 1+mn]. The theory is extended to (numerically) rank deficient matrices, yielding a rank revealing variant of Gaussian elimination and a method for computing the minimum norm solution for x from a reduced normal system and a low-rank update. Algorithms for finding a maximum volume basis are discussed. In the linear programming interior point method a sequence of normal equations needs to be solved, in which A changes by a column scaling from one system to the next. A heuristical algorithm is proposed for maintaining a basis of approximate maximum volume by update operations as those in the revised simplex method. Empirical results demonstrate that the approximation means no loss in the effectiveness of the preconditioner, but makes basis selection much more efficient. The implementation of an interior point solver based on the new linear algebra is described. Features of the code include the elimination of free variables during preconditioning and the removal of degenerate variables from the optimization process once sufficiently close to a bound. A crossover method recovers a vertex solution to the linear program, starting from the basis at the end of the interior point solve. A computational study shows that the implementation is robust and of general applicability, and that its average performance is comparable to that of state-of-the-art solvers

Bundle methods for regularized risk minimization with applications to robust learning

Supervised learning in general and regularized risk minimization in particular is about solving optimization problem which is jointly defined by a performance measure and a set of labeled training examples. The outcome of learning, a model, is then used mainly for predicting the labels for unlabeled examples in the testing environment. In real-world scenarios: a typical learning process often involves solving a sequence of similar problems with different parameters before a final model is identified. For learning to be successful, the final model must be produced timely, and the model should be robust to (mild) irregularities in the testing environment. The purpose of this thesis is to investigate ways to speed up the learning process and improve the robustness of the learned model. We first develop a batch convex optimization solver specialized to the regularized risk minimization based on standard bundle methods. The solver inherits two main properties of the standard bundle methods. Firstly, it is capable of solving both differentiable and non-differentiable problems, hence its implementation can be reused for different tasks with minimal modification. Secondly, the optimization is easily amenable to parallel and distributed computation settings; this makes the solver highly scalable in the number of training examples. However, unlike the standard bundle methods, the solver does not have extra parameters which need careful tuning. Furthermore, we prove that the solver has faster convergence rate. In addition to that, the solver is very efficient in computing approximate regularization path and model selection. We also present a convex risk formulation for incorporating invariances and prior knowledge into the learning problem. This formulation generalizes many existing approaches for robust learning in the setting of insufficient or noisy training examples and covariate shift. Lastly, we extend a non-convex risk formulation for binary classification to structured prediction. Empirical results show that the model obtained with this risk formulation is robust to outliers in the training examples

Self-adaptive isogeometric spatial discretisations of the first and second-order forms of the neutron transport equation with dual-weighted residual error measures and diffusion acceleration

As implemented in a new modern-Fortran code, NURBS-based isogeometric analysis (IGA) spatial discretisations and self-adaptive mesh refinement (AMR) algorithms are developed in the application to the first-order and second-order forms of the neutron transport equation (NTE). These AMR algorithms are shown to be computationally efficient and numerically accurate when compared to standard approaches. IGA methods are very competitive and offer certain unique advantages over standard finite element methods (FEM), not least of all because the numerical analysis is performed over an exact representation of the underlying geometry, which is generally available in some computer-aided design (CAD) software description. Furthermore, mesh refinement can be performed within the analysis program at run-time, without the need to revisit any ancillary mesh generator. Two error measures are described for the IGA-based AMR algorithms, both of which can be employed in conjunction with energy-dependent meshes. The first heuristically minimises any local contributions to the global discretisation error, as per some appropriate user-prescribed norm. The second employs duality arguments to minimise important local contributions to the error as measured in some quantity of interest; this is commonly known as a dual-weighted residual (DWR) error measure and it demands the solution to both the forward (primal) and the adjoint (dual) NTE. Finally, convergent and stable diffusion acceleration and generalised minimal residual (GMRes) algorithms, compatible with the aforementioned AMR algorithms, are introduced to accelerate the convergence of the within-group self-scattering sources for scattering-dominated problems for the first and second-order forms of the NTE. A variety of verification benchmark problems are analysed to demonstrate the computational performance and efficiency of these acceleration techniques.Open Acces