1,606 research outputs found

    Approximate L0 constrained Non-negative Matrix and Tensor Factorization

    Get PDF

    Block Coordinate Descent for Sparse NMF

    Get PDF
    Nonnegative matrix factorization (NMF) has become a ubiquitous tool for data analysis. An important variant is the sparse NMF problem which arises when we explicitly require the learnt features to be sparse. A natural measure of sparsity is the L0_0 norm, however its optimization is NP-hard. Mixed norms, such as L1_1/L2_2 measure, have been shown to model sparsity robustly, based on intuitive attributes that such measures need to satisfy. This is in contrast to computationally cheaper alternatives such as the plain L1_1 norm. However, present algorithms designed for optimizing the mixed norm L1_1/L2_2 are slow and other formulations for sparse NMF have been proposed such as those based on L1_1 and L0_0 norms. Our proposed algorithm allows us to solve the mixed norm sparsity constraints while not sacrificing computation time. We present experimental evidence on real-world datasets that shows our new algorithm performs an order of magnitude faster compared to the current state-of-the-art solvers optimizing the mixed norm and is suitable for large-scale datasets

    Greedy Algorithms for Cone Constrained Optimization with Convergence Guarantees

    Full text link
    Greedy optimization methods such as Matching Pursuit (MP) and Frank-Wolfe (FW) algorithms regained popularity in recent years due to their simplicity, effectiveness and theoretical guarantees. MP and FW address optimization over the linear span and the convex hull of a set of atoms, respectively. In this paper, we consider the intermediate case of optimization over the convex cone, parametrized as the conic hull of a generic atom set, leading to the first principled definitions of non-negative MP algorithms for which we give explicit convergence rates and demonstrate excellent empirical performance. In particular, we derive sublinear (O(1/t)\mathcal{O}(1/t)) convergence on general smooth and convex objectives, and linear convergence (O(et)\mathcal{O}(e^{-t})) on strongly convex objectives, in both cases for general sets of atoms. Furthermore, we establish a clear correspondence of our algorithms to known algorithms from the MP and FW literature. Our novel algorithms and analyses target general atom sets and general objective functions, and hence are directly applicable to a large variety of learning settings.Comment: NIPS 201

    Descent methods for Nonnegative Matrix Factorization

    Full text link
    In this paper, we present several descent methods that can be applied to nonnegative matrix factorization and we analyze a recently developped fast block coordinate method called Rank-one Residue Iteration (RRI). We also give a comparison of these different methods and show that the new block coordinate method has better properties in terms of approximation error and complexity. By interpreting this method as a rank-one approximation of the residue matrix, we prove that it \emph{converges} and also extend it to the nonnegative tensor factorization and introduce some variants of the method by imposing some additional controllable constraints such as: sparsity, discreteness and smoothness.Comment: 47 pages. New convergence proof using damped version of RRI. To appear in Numerical Linear Algebra in Signals, Systems and Control. Accepted. Illustrating Matlab code is included in the source bundl

    Towards Real-Time Simulation Of Hyperelastic Materials

    Get PDF
    We propose a new method for physics-based simulation supporting many different types of hyperelastic materials from mass-spring systems to three-dimensional finite element models, pushing the performance of the simulation towards real-time. Fast simulation methods such as Position Based Dynamics exist, but support only limited selection of materials; even classical materials such as corotated linear elasticity and Neo-Hookean elasticity are not supported. Simulation of these types of materials currently relies on Newton\u27s method, which is slow, even with only one iteration per timestep. In this work, we start from simple material models such as mass-spring systems or as-rigid-as-possible materials. We express the widely used implicit Euler time integration as an energy minimization problem and introduce auxiliary projection variables as extra unknowns. After our reformulation, the minimization problem becomes linear in the node positions, while all the non-linear terms are isolated in individual elements. We then extend this idea to efficiently simulate a more general spatial discretization using finite element method. We show that our reformulation can be interpreted as a quasi-Newton method. This insight enables very efficient simulation of a large class of hyperelastic materials. The quasi-Newton interpretation also allows us to leverage ideas from numerical optimization. In particular, we show that our solver can be further accelerated using L-BFGS updates (Limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm). Our final method is typically more than ten times faster than one iteration of Newton\u27s method without compromising quality. In fact, our result is often more accurate than the result obtained with one iteration of Newton\u27s method. Our method is also easier to implement, implying reduced software development costs

    Sequential Sparse NMF

    Get PDF
    Nonnegative Matrix Factorization (NMF) is a standard tool for data analysis. An important variant is the Sparse NMF problem. A natural measure of sparsity is the L₀ norm, however its optimization is NP-hard. Here, we consider a sparsity measure linear in the ratio of the L₁ and L₂ norms, and propose an efficient algorithm to handle the norm constraints which arise when optimizing this measure. Although algorithms for solving these are available, they are typically inefficient. We present experimental evidence that our new algorithm performs an order of magnitude faster compared to the previous state-of-the-art
    corecore