7,797 research outputs found
Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization
Majorization-minimization algorithms consist of iteratively minimizing a
majorizing surrogate of an objective function. Because of its simplicity and
its wide applicability, this principle has been very popular in statistics and
in signal processing. In this paper, we intend to make this principle scalable.
We introduce a stochastic majorization-minimization scheme which is able to
deal with large-scale or possibly infinite data sets. When applied to convex
optimization problems under suitable assumptions, we show that it achieves an
expected convergence rate of after iterations, and of
for strongly convex functions. Equally important, our scheme almost
surely converges to stationary points for a large class of non-convex problems.
We develop several efficient algorithms based on our framework. First, we
propose a new stochastic proximal gradient method, which experimentally matches
state-of-the-art solvers for large-scale -logistic regression. Second,
we develop an online DC programming algorithm for non-convex sparse estimation.
Finally, we demonstrate the effectiveness of our approach for solving
large-scale structured matrix factorization problems.Comment: accepted for publication for Neural Information Processing Systems
(NIPS) 2013. This is the 9-pages version followed by 16 pages of appendices.
The title has changed compared to the first technical repor
Solving Large-Scale Optimization Problems Related to Bell's Theorem
Impossibility of finding local realistic models for quantum correlations due
to entanglement is an important fact in foundations of quantum physics, gaining
now new applications in quantum information theory. We present an in-depth
description of a method of testing the existence of such models, which involves
two levels of optimization: a higher-level non-linear task and a lower-level
linear programming (LP) task. The article compares the performances of the
existing implementation of the method, where the LPs are solved with the
simplex method, and our new implementation, where the LPs are solved with a
matrix-free interior point method. We describe in detail how the latter can be
applied to our problem, discuss the basic scenario and possible improvements
and how they impact on overall performance. Significant performance advantage
of the matrix-free interior point method over the simplex method is confirmed
by extensive computational results. The new method is able to solve problems
which are orders of magnitude larger. Consequently, the noise resistance of the
non-classicality of correlations of several types of quantum states, which has
never been computed before, can now be efficiently determined. An extensive set
of data in the form of tables and graphics is presented and discussed. The
article is intended for all audiences, no quantum-mechanical background is
necessary.Comment: 19 pages, 7 tables, 1 figur
Large scale optimization of transonic axial compressor rotor blades
[First Paragraphs]
In the present work the Multipoint Approximation Method (MAM) by Toropov et al. (1993) has been applied to
the shape optimization of an existing transonic compressor rotor (NASA rotor 37) as a benchmark case.
Simulations were performed using the Rolls-Royce plc. PADRAM-HYDRA system (Shahpar and Lapworth 2003,
Lapworth and Shahpar 2004) that includes the parameterization of the blade shape, meshing, CFD analysis, postprocessing,
and objective/constraints evaluation. The parameterization approach adopted in this system is very
flexible but can result in a large scale optimization problem.
For this pilot study, a relatively coarse mesh has been used including around 470,000 nodes. The
parameterization was done using 5 engineering blade parameters like axial movement of sections along the engine
axis in mm (XCEN), circumferential movements of sections in degrees (DELT), solid body rotation of sections in
degrees (SKEW), and leading/trailing edge recambering (LEM0/TEMO) in degrees. The design variables were
specified using 6 control points at 0 % (hub), 20%, 40%, 60%, 80%, and 100% (tip) along the span. Thus the total
number of independent design variables N was 30. B-spline interpolation was used through the control points to
generate smooth design perturbations in the radial direction
A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization
We propose a computationally efficient limited memory Covariance Matrix
Adaptation Evolution Strategy for large scale optimization, which we call the
LM-CMA-ES. The LM-CMA-ES is a stochastic, derivative-free algorithm for
numerical optimization of non-linear, non-convex optimization problems in
continuous domain. Inspired by the limited memory BFGS method of Liu and
Nocedal (1989), the LM-CMA-ES samples candidate solutions according to a
covariance matrix reproduced from direction vectors selected during the
optimization process. The decomposition of the covariance matrix into Cholesky
factors allows to reduce the time and memory complexity of the sampling to
, where is the number of decision variables. When is large
(e.g., > 1000), even relatively small values of (e.g., ) are
sufficient to efficiently solve fully non-separable problems and to reduce the
overall run-time.Comment: Genetic and Evolutionary Computation Conference (GECCO'2014) (2014
Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems
In this paper we review basic and emerging models and associated algorithms
for large-scale tensor networks, especially Tensor Train (TT) decompositions
using novel mathematical and graphical representations. We discus the concept
of tensorization (i.e., creating very high-order tensors from lower-order
original data) and super compression of data achieved via quantized tensor
train (QTT) networks. The purpose of a tensorization and quantization is to
achieve, via low-rank tensor approximations "super" compression, and
meaningful, compact representation of structured data. The main objective of
this paper is to show how tensor networks can be used to solve a wide class of
big data optimization problems (that are far from tractable by classical
numerical methods) by applying tensorization and performing all operations
using relatively small size matrices and tensors and applying iteratively
optimized and approximative tensor contractions.
Keywords: Tensor networks, tensor train (TT) decompositions, matrix product
states (MPS), matrix product operators (MPO), basic tensor operations,
tensorization, distributed representation od data optimization problems for
very large-scale problems: generalized eigenvalue decomposition (GEVD),
PCA/SVD, canonical correlation analysis (CCA).Comment: arXiv admin note: text overlap with arXiv:1403.204
Dynamic Factorization in Large-Scale Optimization
Mathematical Programming, 64, pp. 17-51.Factorization of linear programming (LP) models enables a large portion of the LP tableau to be
represented implicitly and generated from the remaining explicit part. Dynamic factorization admits algebraic elements which change in dimension during the course of solution. A unifying mathematical framework for dynamic row factorization is presented with three algorithms which derive from different LP model row structures: generalized upper bound rows, pure network rows,and generalized network TOWS. Each of these structures is a generalization of its predecessors, and each corresponding algorithm exhibits just enough additional richness to accommodate the structure at hand within the
unified framework. Implementation and computational results are presented for a variety of real-world models. These results suggest that each of these algorithms is superior to the traditional, non-factorized approach, with the degree of improvement depending upon the size and quality of the row factorization identified
- …