Lower Bounds on the Bounded Coefficient Complexity of Bilinear Maps

We prove lower bounds of order $n\log n$ for both the problem to multiply polynomials of degree $n$, and to divide polynomials with remainder, in the model of bounded coefficient arithmetic circuits over the complex numbers. These lower bounds are optimal up to order of magnitude. The proof uses a recent idea of R. Raz [Proc. 34th STOC 2002] proposed for matrix multiplication. It reduces the linear problem to multiply a random circulant matrix with a vector to the bilinear problem of cyclic convolution. We treat the arising linear problem by extending J. Morgenstern's bound [J. ACM 20, pp. 305-306, 1973] in a unitarily invariant way. This establishes a new lower bound on the bounded coefficient complexity of linear forms in terms of the singular values of the corresponding matrix. In addition, we extend these lower bounds for linear and bilinear maps to a model of circuits that allows a restricted number of unbounded scalar multiplications.Comment: 19 page

Min-Rank Conjecture for Log-Depth Circuits

A completion of an m-by-n matrix A with entries in {0,1,*} is obtained by setting all *-entries to constants 0 or 1. A system of semi-linear equations over GF(2) has the form Mx=f(x), where M is a completion of A and f:{0,1}^n --> {0,1}^m is an operator, the i-th coordinate of which can only depend on variables corresponding to *-entries in the i-th row of A. We conjecture that no such system can have more than 2^{n-c\cdot mr(A)} solutions, where c>0 is an absolute constant and mr(A) is the smallest rank over GF(2) of a completion of A. The conjecture is related to an old problem of proving super-linear lower bounds on the size of log-depth boolean circuits computing linear operators x --> Mx. The conjecture is also a generalization of a classical question about how much larger can non-linear codes be than linear ones. We prove some special cases of the conjecture and establish some structural properties of solution sets.Comment: 22 pages, to appear in: J. Comput.Syst.Sci

Flexible Multi-layer Sparse Approximations of Matrices and Applications

The computational cost of many signal processing and machine learning techniques is often dominated by the cost of applying certain linear operators to high-dimensional vectors. This paper introduces an algorithm aimed at reducing the complexity of applying linear operators in high dimension by approximately factorizing the corresponding matrix into few sparse factors. The approach relies on recent advances in non-convex optimization. It is first explained and analyzed in details and then demonstrated experimentally on various problems including dictionary learning for image denoising, and the approximation of large matrices arising in inverse problems

On Matrix Rigidity and the Complexity of Linear Forms

The rigidity function of a matrix is defined as the minimum number of its entries that need to be changed in order to reduce the rank of the matrix to below a given parameter. Proving a strong enough lower bound on the rigidity of a matrix implies a nontrivial lower bound on the complexity of any linear circuit computing the set of linear forms associated with it. However, although it is shown that most matrices are rigid enough, no explicit construction of a rigid family of matrices is known. In this survey report we review the concept of rigidity and some of its interesting variations as well as several notable results related to that. We also show the existence of highly rigid matrices constructed by evaluation of bivariate polynomials over finite fields

Analiza procesnih i računskih iteracija primenom savremenih računarskih aritmetika

The proposed theme relates to the field of application of modern computer arithmetics in the analysis of process performance and computational iterations, where the concept of the modern computer arithmetic applies to multiple-precision arithmetic and interval arithmetic involved in the new standard IEEE 754 in 2008. Advance computer arithmetic, first of all interval arithmetic and multi-precision arithmetic, are employed in the the dissertation for the analysis of matrix models in the design iteration process software and iterative numerical computations. These are important and current research topics from the point at which the application is working in the world. Research in this area have led to the development and analysis of algorithms to control the accuracy, optimality, rate calculations and other performance aspects of various processes such as designing software and hardware, designing industrial products, transportation optimization, modeling systems, the implementation of numerical algorithms and others. The first part of dissertation is devoted to engineering design and development of new products, which are either industrial products, technical innovations, hardware or software, often contain a very complex set of relationships among many coupled tasks. Controlling, redesigning and identifying features of these tasks can be usefully performed by a suitable model based on the design structure matrix in an iteration procedure. The proposed interval matrix model of design iteration controls and predicts slow and rapid convergence of iteration work on tasks within a project. A new model is based on Perron- Frobenius theorem and interval linear algebra where intervals and interval matrices are employed instead of real numbers and real matrices. In this way a more relaxed quantitative estimation of tasks is achieved and the presence of undetermined quantities is allowed to a certain extent. The presented model is demonstrated in the example of simplified software development process. An additional contribution in this dissertation is the ranking of tasks within a design mode using components of eigenvalue vector corresponding to the spectral radius of design structure matrix. The second part deals with computational efficiency of numerical iterative algorithms. Computational cost of iterative procedures for the implementation of basic arithmetic operations in multi-precision arithmetic are studied and later applied for the analysis of computational efficiency of the iterative methods for solving nonlinear equations. A new approach that deals with the weights of employed arithmetic operations, involved in the realization of these algorithms, is proposed. This enables a precise ranking of considered root-finding algorithms of different structure, especially in a regime of variable (dynamic) precision of applied multi-precision arithmetic