13,428 research outputs found

    Sparse Plus Low Rank Matrix Decomposition: A Discrete Optimization Approach

    Full text link
    We study the Sparse Plus Low-Rank decomposition problem (SLR), which is the problem of decomposing a corrupted data matrix into a sparse matrix of perturbations plus a low-rank matrix containing the ground truth. SLR is a fundamental problem in Operations Research and Machine Learning which arises in various applications, including data compression, latent semantic indexing, collaborative filtering, and medical imaging. We introduce a novel formulation for SLR that directly models its underlying discreteness. For this formulation, we develop an alternating minimization heuristic that computes high-quality solutions and a novel semidefinite relaxation that provides meaningful bounds for the solutions returned by our heuristic. We also develop a custom branch-and-bound algorithm that leverages our heuristic and convex relaxations to solve small instances of SLR to certifiable (near) optimality. Given an input nn-by-nn matrix, our heuristic scales to solve instances where n=10000n=10000 in minutes, our relaxation scales to instances where n=200n=200 in hours, and our branch-and-bound algorithm scales to instances where n=25n=25 in minutes. Our numerical results demonstrate that our approach outperforms existing state-of-the-art approaches in terms of rank, sparsity, and mean-square error while maintaining a comparable runtime

    Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference

    Full text link
    We propose Conditional Adapter (CoDA), a parameter-efficient transfer learning method that also improves inference efficiency. CoDA generalizes beyond standard adapter approaches to enable a new way of balancing speed and accuracy using conditional computation. Starting with an existing dense pretrained model, CoDA adds sparse activation together with a small number of new parameters and a light-weight training phase. Our experiments demonstrate that the CoDA approach provides an unexpectedly efficient way to transfer knowledge. Across a variety of language, vision, and speech tasks, CoDA achieves a 2x to 8x inference speed-up compared to the state-of-the-art Adapter approach with moderate to no accuracy loss and the same parameter efficiency

    Safe Zeroth-Order Optimization Using Quadratic Local Approximations

    Full text link
    This paper addresses black-box smooth optimization problems, where the objective and constraint functions are not explicitly known but can be queried. The main goal of this work is to generate a sequence of feasible points converging towards a KKT primal-dual pair. Assuming to have prior knowledge on the smoothness of the unknown objective and constraints, we propose a novel zeroth-order method that iteratively computes quadratic approximations of the constraint functions, constructs local feasible sets and optimizes over them. Under some mild assumptions, we prove that this method returns an η\eta-KKT pair (a property reflecting how close a primal-dual pair is to the exact KKT condition) within O(1/η2)O({1}/{\eta^{2}}) iterations. Moreover, we numerically show that our method can achieve faster convergence compared with some state-of-the-art zeroth-order approaches. The effectiveness of the proposed approach is also illustrated by applying it to nonconvex optimization problems in optimal control and power system operation.Comment: arXiv admin note: text overlap with arXiv:2211.0264

    Convergence Rate of Nonconvex Douglas-Rachford splitting via merit functions, with applications to weakly convex constrained optimization

    Full text link
    We analyze Douglas-Rachford splitting techniques applied to solving weakly convex optimization problems. Under mild regularity assumptions, and by the token of a suitable merit function, we show convergence to critical points and local linear rates of convergence. The merit function, comparable to the Moreau envelope in Variational Analysis, generates a descent sequence, a feature that allows us to extend to the non-convex setting arguments employed in convex optimization. A by-product of our approach is a ADMM-like method for constrained problems with weakly convex objective functions. When specialized to multistage stochastic programming, the proposal yields a nonconvex version of the Progressive Hedging algorithm that converges with linear speed. The numerical assessment on a battery of phase retrieval problems shows promising numerical performance of our method, when compared to existing algorithms in the literature.Comment: 24 pages, 1 figur

    Kirchhoff-Love shell representation and analysis using triangle configuration B-splines

    Full text link
    This paper presents the application of triangle configuration B-splines (TCB-splines) for representing and analyzing the Kirchhoff-Love shell in the context of isogeometric analysis (IGA). The Kirchhoff-Love shell formulation requires global C1C^1-continuous basis functions. The nonuniform rational B-spline (NURBS)-based IGA has been extensively used for developing Kirchhoff-Love shell elements. However, shells with complex geometries inevitably need multiple patches and trimming techniques, where stitching patches with high continuity is a challenge. On the other hand, due to their unstructured nature, TCB-splines can accommodate general polygonal domains, have local refinement, and are flexible to model complex geometries with C1C^1 continuity, which naturally fit into the Kirchhoff-Love shell formulation with complex geometries. Therefore, we propose to use TCB-splines as basis functions for geometric representation and solution approximation. We apply our method to both linear and nonlinear benchmark shell problems, where the accuracy and robustness are validated. The applicability of the proposed approach to shell analysis is further exemplified by performing geometrically nonlinear Kirchhoff-Love shell simulations of a pipe junction and a front bumper represented by a single patch of TCB-splines

    Multiscale structural optimisation with concurrent coupling between scales

    Get PDF
    A robust three-dimensional multiscale topology optimisation framework with concurrent coupling between scales is presented. Concurrent coupling ensures that only the microscale data required to evaluate the macroscale model during each iteration of optimisation is collected and results in considerable computational savings. This represents the principal novelty of the framework and permits a previously intractable number of design variables to be used in the parametrisation of the microscale geometry, which in turn enables accessibility to a greater range of mechanical point properties during optimisation. Additionally, the microscale data collected during optimisation is stored in a re-usable database, further reducing the computational expense of subsequent iterations or entirely new optimisation problems. Application of this methodology enables structures with precise functionally-graded mechanical properties over two-scales to be derived, which satisfy one or multiple functional objectives. For all applications of the framework presented within this thesis, only a small fraction of the microstructure database is required to derive the optimised multiscale solutions, which demonstrates a significant reduction in the computational expense of optimisation in comparison to contemporary sequential frameworks. The derivation and integration of novel additive manufacturing constraints for open-walled microstructures within the concurrently coupled multiscale topology optimisation framework is also presented. Problematic fabrication features are discouraged through the application of an augmented projection filter and two relaxed binary integral constraints, which prohibit the formation of unsupported members, isolated assemblies of overhanging members and slender members during optimisation. Through the application of these constraints, it is possible to derive self-supporting, hierarchical structures with varying topology, suitable for fabrication through additive manufacturing processes.Open Acces

    Discovering the hidden structure of financial markets through bayesian modelling

    Get PDF
    Understanding what is driving the price of a financial asset is a question that is currently mostly unanswered. In this work we go beyond the classic one step ahead prediction and instead construct models that create new information on the behaviour of these time series. Our aim is to get a better understanding of the hidden structures that drive the moves of each financial time series and thus the market as a whole. We propose a tool to decompose multiple time series into economically-meaningful variables to explain the endogenous and exogenous factors driving their underlying variability. The methodology we introduce goes beyond the direct model forecast. Indeed, since our model continuously adapts its variables and coefficients, we can study the time series of coefficients and selected variables. We also present a model to construct the causal graph of relations between these time series and include them in the exogenous factors. Hence, we obtain a model able to explain what is driving the move of both each specific time series and the market as a whole. In addition, the obtained graph of the time series provides new information on the underlying risk structure of this environment. With this deeper understanding of the hidden structure we propose novel ways to detect and forecast risks in the market. We investigate our results with inferences up to one month into the future using stocks, FX futures and ETF futures, demonstrating its superior performance according to accuracy of large moves, longer-term prediction and consistency over time. We also go in more details on the economic interpretation of the new variables and discuss the created graph structure of the market.Open Acces

    Sparse Functional Linear Discriminant Analysis

    Get PDF
    Functional linear discriminant analysis offers a simple yet efficient method for classification, with the possibility of achieving a perfect classification. Several methods are proposed in the literature that mostly address the dimensionality of the problem. On the other hand, there is a growing interest in interpretability of the analysis, which favors a simple and sparse solution. In this work, we propose a new approach that incorporates a type of sparsity that identifies nonzero sub-domains in the functional setting, offering a solution that is easier to interpret without compromising performance. With the need to embed additional constraints in the solution, we reformulate the functional linear discriminant analysis as a regularization problem with an appropriate penalty. Inspired by the success of ℓ1-type regularization at inducing zero coefficients for scalar variables, we develop a new regularization method for functional linear discriminant analysis that incorporates an L1-type penalty, ∫ |f|, to induce zero regions. We demonstrate that our formulation has a well-defined solution that contains zero regions, achieving a functional sparsity in the sense of domain selection. In addition, the misclassification probability of the regularized solution is shown to converge to the Bayes error if the data are Gaussian. Our method does not presume that the underlying function has zero regions in the domain, but produces a sparse estimator that consistently estimates the true function whether or not the latter is sparse. Numerical comparisons with existing methods demonstrate this property in finite samples with both simulated and real data examples

    Towards a non-equilibrium thermodynamic theory of ecosystem assembly and development

    Get PDF
    Non-equilibrium thermodynamics has had a significant historic influence on the development of theoretical ecology, even informing the very concept of an ecosystem. Much of this influence has manifested as proposed extremal principles. These principles hold that systems will tend to maximise certain thermodynamic quantities, subject to the other constraints they operate under. A particularly notable extremal principle is the maximum entropy production principle (MaxEPP); that systems maximise their rate of entropy production. However, these principles are not robustly based in physical theory, and suffer from treating complex ecosystems in an extremely coarse manner. To address this gap, this thesis derives a limited but physically justified extremal principle, as well as carrying out a detailed investigation of the impact of non-equilibrium thermodynamic constraints on the assembly of microbial communities. The extremal principle we obtain pertains to the switching between states in simple bistable systems, with switching paths that generate more entropy being favoured. Our detailed investigation into microbial communities involved developing a novel thermodynamic microbial community model, using which we found the rate of ecosystem development to be set by the availability of free-energy. Further investigation was carried out using this model, demonstrating the way that trade-offs emerging from fundamental thermodynamic constraints impact the dynamics of assembling microbial communities. Taken together our results demonstrate that theory can be developed from non-equilibrium thermodynamics, that is both ecologically relevant and physically well grounded. We find that broad extremal principles are unlikely to be obtained, absent significant advances in the field of stochastic thermodynamics, limiting their applicability to ecology. However, we find that detailed consideration of the non-equilibrium thermodynamic mechanisms that impact microbial communities can broaden our understanding of their assembly and functioning.Open Acces
    • …
    corecore