Impact of Accuracy Optimization on the Convergence of Numerical Iterative Methods

Among other objectives, rewriting programs serves as a useful technique to improve numerical accuracy. However, this optimization is not intuitive and this is why we switch to automatic transformation techniques. We are interested in the optimization of numerical programs relying on the IEEE754 oating-point arithmetic. In this article, our main contribution is to study the impact of optimizing the numerical accuracy of programs on the time required by numerical iterative methods to converge. To emphasize the usefulness of our tool, we make it optimize several examples of numerical methods such as Jacobi's method, Newton-Raphson's method, etc. We show that significant speedups are obtained in terms of number of iterations, time and ops

Regularized Nonlinear Acceleration

We describe a convergence acceleration technique for unconstrained optimization problems. Our scheme computes estimates of the optimum from a nonlinear average of the iterates produced by any optimization method. The weights in this average are computed via a simple linear system, whose solution can be updated online. This acceleration scheme runs in parallel to the base algorithm, providing improved estimates of the solution on the fly, while the original optimization method is running. Numerical experiments are detailed on classical classification problems

Dissipative Quantum Dynamics and Optimal Control using Iterative Time Ordering: An Application to Superconducting Qubits

We combine a quantum dynamical propagator that explicitly accounts for quantum mechanical time ordering with optimal control theory. After analyzing its performance with a simple model, we apply it to a superconducting circuit under so-called Pythagorean control. Breakdown of the rotating-wave approximation is the main source of the very strong time-dependence in this example. While the propagator that accounts for the time ordering in an iterative fashion proves its numerical efficiency for the dynamics of the superconducting circuit, its performance when combined with optimal control turns out to be rather sensitive to the strength of the time-dependence. We discuss the kind of quantum gate operations that the superconducting circuit can implement including their performance bounds in terms of fidelity and speed.Comment: 16 pages, 11 figure

Hardware Impairments Aware Transceiver Design for Full-Duplex Amplify-and-Forward MIMO Relaying

In this work we study the behavior of a full-duplex (FD) and amplify-and-forward (AF) relay with multiple antennas, where hardware impairments of the FD relay transceiver is taken into account. Due to the inter-dependency of the transmit relay power on each antenna and the residual self-interference in an FD-AF relay, we observe a distortion loop that degrades the system performance when the relay dynamic range is not high. In this regard, we analyze the relay function in presence of the hardware inaccuracies and an optimization problem is formulated to maximize the signal to distortion-plus-noise ratio (SDNR), under relay and source transmit power constraints. Due to the problem complexity, we propose a gradient-projection-based (GP) algorithm to obtain an optimal solution. Moreover, a nonalternating sub-optimal solution is proposed by assuming a rank-1 relay amplification matrix, and separating the design of the relay process into multiple stages (MuStR1). The proposed MuStR1 method is then enhanced by introducing an alternating update over the optimization variables, denoted as AltMuStR1 algorithm. It is observed that compared to GP, (Alt)MuStR1 algorithms significantly reduce the required computational complexity at the expense of a slight performance degradation. Finally, the proposed methods are evaluated under various system conditions, and compared with the methods available in the current literature. In particular, it is observed that as the hardware impairments increase, or for a system with a high transmit power, the impact of applying a distortion-aware design is significant.Comment: Submitted to IEEE Transactions on Wireless Communication

Improving Performance of Iterative Methods by Lossy Checkponting

Iterative methods are commonly used approaches to solve large, sparse linear systems, which are fundamental operations for many modern scientific simulations. When the large-scale iterative methods are running with a large number of ranks in parallel, they have to checkpoint the dynamic variables periodically in case of unavoidable fail-stop errors, requiring fast I/O systems and large storage space. To this end, significantly reducing the checkpointing overhead is critical to improving the overall performance of iterative methods. Our contribution is fourfold. (1) We propose a novel lossy checkpointing scheme that can significantly improve the checkpointing performance of iterative methods by leveraging lossy compressors. (2) We formulate a lossy checkpointing performance model and derive theoretically an upper bound for the extra number of iterations caused by the distortion of data in lossy checkpoints, in order to guarantee the performance improvement under the lossy checkpointing scheme. (3) We analyze the impact of lossy checkpointing (i.e., extra number of iterations caused by lossy checkpointing files) for multiple types of iterative methods. (4)We evaluate the lossy checkpointing scheme with optimal checkpointing intervals on a high-performance computing environment with 2,048 cores, using a well-known scientific computation package PETSc and a state-of-the-art checkpoint/restart toolkit. Experiments show that our optimized lossy checkpointing scheme can significantly reduce the fault tolerance overhead for iterative methods by 23%~70% compared with traditional checkpointing and 20%~58% compared with lossless-compressed checkpointing, in the presence of system failures.Comment: 14 pages, 10 figures, HPDC'1

OSQP: An Operator Splitting Solver for Quadratic Programs

We present a general-purpose solver for convex quadratic programs based on the alternating direction method of multipliers, employing a novel operator splitting technique that requires the solution of a quasi-definite linear system with the same coefficient matrix at almost every iteration. Our algorithm is very robust, placing no requirements on the problem data such as positive definiteness of the objective function or linear independence of the constraint functions. It can be configured to be division-free once an initial matrix factorization is carried out, making it suitable for real-time applications in embedded systems. In addition, our technique is the first operator splitting method for quadratic programs able to reliably detect primal and dual infeasible problems from the algorithm iterates. The method also supports factorization caching and warm starting, making it particularly efficient when solving parametrized problems arising in finance, control, and machine learning. Our open-source C implementation OSQP has a small footprint, is library-free, and has been extensively tested on many problem instances from a wide variety of application areas. It is typically ten times faster than competing interior-point methods, and sometimes much more when factorization caching or warm start is used. OSQP has already shown a large impact with tens of thousands of users both in academia and in large corporations