Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems

Optimization methods are at the core of many problems in signal/image processing, computer vision, and machine learning. For a long time, it has been recognized that looking at the dual of an optimization problem may drastically simplify its solution. Deriving efficient strategies which jointly brings into play the primal and the dual problems is however a more recent idea which has generated many important new contributions in the last years. These novel developments are grounded on recent advances in convex analysis, discrete optimization, parallel processing, and non-smooth optimization with emphasis on sparsity issues. In this paper, we aim at presenting the principles of primal-dual approaches, while giving an overview of numerical methods which have been proposed in different contexts. We show the benefits which can be drawn from primal-dual algorithms both for solving large-scale convex optimization problems and discrete ones, and we provide various application examples to illustrate their usefulness

Massively-Parallel Feature Selection for Big Data

We present the Parallel, Forward-Backward with Pruning (PFBP) algorithm for feature selection (FS) in Big Data settings (high dimensionality and/or sample size). To tackle the challenges of Big Data FS PFBP partitions the data matrix both in terms of rows (samples, training examples) as well as columns (features). By employing the concepts of $p$-values of conditional independence tests and meta-analysis techniques PFBP manages to rely only on computations local to a partition while minimizing communication costs. Then, it employs powerful and safe (asymptotically sound) heuristics to make early, approximate decisions, such as Early Dropping of features from consideration in subsequent iterations, Early Stopping of consideration of features within the same iteration, or Early Return of the winner in each iteration. PFBP provides asymptotic guarantees of optimality for data distributions faithfully representable by a causal network (Bayesian network or maximal ancestral graph). Our empirical analysis confirms a super-linear speedup of the algorithm with increasing sample size, linear scalability with respect to the number of features and processing cores, while dominating other competitive algorithms in its class

AC OPF in Radial Distribution Networks - Parts I,II

The optimal power-flow problem (OPF) has played a key role in the planning and operation of power systems. Due to the non-linear nature of the AC power-flow equations, the OPF problem is known to be non-convex, therefore hard to solve. Most proposed methods for solving the OPF rely on approximations that render the problem convex, but that may yield inexact solutions. Recently, Farivar and Low proposed a method that is claimed to be exact for radial distribution systems, despite no apparent approximations. In our work, we show that it is, in fact, not exact. On one hand, there is a misinterpretation of the physical network model related to the ampacity constraint of the lines' current flows. On the other hand, the proof of the exactness of the proposed relaxation requires unrealistic assumptions related to the unboundedness of specific control variables. We also show that the extension of this approach to account for exact line models might provide physically infeasible solutions. Recently, several contributions have proposed OPF algorithms that rely on the use of the alternating-direction method of multipliers (ADMM). However, as we show in this work, there are cases for which the ADMM-based solution of the non-relaxed OPF problem fails to converge. To overcome the aforementioned limitations, we propose an algorithm for the solution of a non-approximated, non-convex OPF problem in radial distribution systems that is based on the method of multipliers, and on a primal decomposition of the OPF. This work is divided in two parts. In Part I, we specifically discuss the limitations of BFM and ADMM to solve the OPF problem. In Part II, we provide a centralized version and a distributed asynchronous version of the proposed OPF algorithm and we evaluate its performances using both small-scale electrical networks, as well as a modified IEEE 13-node test feeder

Phase-field modeling and simulation of fracture in brittle materials with strongly anisotropic surface energy

Crack propagation in brittle materials with anisotropic surface energy is important in applications involving single crystals, extruded polymers, or geological and organic materials. Furthermore, when this anisotropy is strong, the phenomenology of crack propagation becomes very rich, with forbidden crack propagation directions or complex sawtooth crack patterns. This problem interrogates fundamental issues in fracture mechanics, including the principles behind the selection of crack direction. Here, we propose a variational phase-field model for strongly anisotropic fracture, which resorts to the extended Cahn-Hilliard framework proposed in the context of crystal growth. Previous phase-field models for anisotropic fracture were formulated in a framework only allowing for weak anisotropy. We implement numerically our higher-order phase-field model with smooth local maximum entropy approximants in a direct Galerkin method. The numerical results exhibit all the features of strongly anisotropic fracture and reproduce strikingly well recent experimental observations.Peer ReviewedPostprint (author’s final draft

Efficient Structures for Innovative Social Networks

What lines of communication among members of an organization are most effective in the early, ideation phase of innovation? We investigate this question with a recombination and selection model of knowledge transfer operating through a social network. We measure cost in human time, and seek efficient social network structures in the time--total cost plane (minimize ideation time subject to an upper bound on total cost, or vice versa) and in the time--cost per period plane, with a similar interpretation. Our results suggest that efficiently innovative organizations look nothing like what one intuitively associates with standard formal organizations with strict and unchanging lines of communication, nor do they conform with what one might expect from static social network representations of communication patterns. Rather, ideation is accelerated when people dynamically churn through a large (ideally the entire population) set of conversational partners over time, which naturally begets short path lengths and eliminates information bottlenecks. In organizations with these features group meetings do not help and can hurt the process, because many parallel conversations can achieve the same or better results as one-to-many communications. A family of networks called the complete wheel-stars emerges as an important family on the time-cost efficient frontier. Wheel-star graphs have a completely connected clique of agents at the center, with all other agents connected to the core but not to each other; the star and the complete graph are its extreme elements. We discuss the consequences of these results for organizations and sociometric analyses.http://deepblue.lib.umich.edu/bitstream/2027.42/64992/1/1136_lovejoy.pd