13,428 research outputs found
Sparse Plus Low Rank Matrix Decomposition: A Discrete Optimization Approach
We study the Sparse Plus Low-Rank decomposition problem (SLR), which is the
problem of decomposing a corrupted data matrix into a sparse matrix of
perturbations plus a low-rank matrix containing the ground truth. SLR is a
fundamental problem in Operations Research and Machine Learning which arises in
various applications, including data compression, latent semantic indexing,
collaborative filtering, and medical imaging. We introduce a novel formulation
for SLR that directly models its underlying discreteness. For this formulation,
we develop an alternating minimization heuristic that computes high-quality
solutions and a novel semidefinite relaxation that provides meaningful bounds
for the solutions returned by our heuristic. We also develop a custom
branch-and-bound algorithm that leverages our heuristic and convex relaxations
to solve small instances of SLR to certifiable (near) optimality. Given an
input -by- matrix, our heuristic scales to solve instances where
in minutes, our relaxation scales to instances where in
hours, and our branch-and-bound algorithm scales to instances where in
minutes. Our numerical results demonstrate that our approach outperforms
existing state-of-the-art approaches in terms of rank, sparsity, and
mean-square error while maintaining a comparable runtime
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference
We propose Conditional Adapter (CoDA), a parameter-efficient transfer
learning method that also improves inference efficiency. CoDA generalizes
beyond standard adapter approaches to enable a new way of balancing speed and
accuracy using conditional computation. Starting with an existing dense
pretrained model, CoDA adds sparse activation together with a small number of
new parameters and a light-weight training phase. Our experiments demonstrate
that the CoDA approach provides an unexpectedly efficient way to transfer
knowledge. Across a variety of language, vision, and speech tasks, CoDA
achieves a 2x to 8x inference speed-up compared to the state-of-the-art Adapter
approach with moderate to no accuracy loss and the same parameter efficiency
Safe Zeroth-Order Optimization Using Quadratic Local Approximations
This paper addresses black-box smooth optimization problems, where the
objective and constraint functions are not explicitly known but can be queried.
The main goal of this work is to generate a sequence of feasible points
converging towards a KKT primal-dual pair. Assuming to have prior knowledge on
the smoothness of the unknown objective and constraints, we propose a novel
zeroth-order method that iteratively computes quadratic approximations of the
constraint functions, constructs local feasible sets and optimizes over them.
Under some mild assumptions, we prove that this method returns an -KKT
pair (a property reflecting how close a primal-dual pair is to the exact KKT
condition) within iterations. Moreover, we numerically show
that our method can achieve faster convergence compared with some
state-of-the-art zeroth-order approaches. The effectiveness of the proposed
approach is also illustrated by applying it to nonconvex optimization problems
in optimal control and power system operation.Comment: arXiv admin note: text overlap with arXiv:2211.0264
Convergence Rate of Nonconvex Douglas-Rachford splitting via merit functions, with applications to weakly convex constrained optimization
We analyze Douglas-Rachford splitting techniques applied to solving weakly
convex optimization problems. Under mild regularity assumptions, and by the
token of a suitable merit function, we show convergence to critical points and
local linear rates of convergence. The merit function, comparable to the Moreau
envelope in Variational Analysis, generates a descent sequence, a feature that
allows us to extend to the non-convex setting arguments employed in convex
optimization. A by-product of our approach is a ADMM-like method for
constrained problems with weakly convex objective functions. When specialized
to multistage stochastic programming, the proposal yields a nonconvex version
of the Progressive Hedging algorithm that converges with linear speed. The
numerical assessment on a battery of phase retrieval problems shows promising
numerical performance of our method, when compared to existing algorithms in
the literature.Comment: 24 pages, 1 figur
Kirchhoff-Love shell representation and analysis using triangle configuration B-splines
This paper presents the application of triangle configuration B-splines
(TCB-splines) for representing and analyzing the Kirchhoff-Love shell in the
context of isogeometric analysis (IGA). The Kirchhoff-Love shell formulation
requires global -continuous basis functions. The nonuniform rational
B-spline (NURBS)-based IGA has been extensively used for developing
Kirchhoff-Love shell elements. However, shells with complex geometries
inevitably need multiple patches and trimming techniques, where stitching
patches with high continuity is a challenge. On the other hand, due to their
unstructured nature, TCB-splines can accommodate general polygonal domains,
have local refinement, and are flexible to model complex geometries with
continuity, which naturally fit into the Kirchhoff-Love shell formulation with
complex geometries. Therefore, we propose to use TCB-splines as basis functions
for geometric representation and solution approximation. We apply our method to
both linear and nonlinear benchmark shell problems, where the accuracy and
robustness are validated. The applicability of the proposed approach to shell
analysis is further exemplified by performing geometrically nonlinear
Kirchhoff-Love shell simulations of a pipe junction and a front bumper
represented by a single patch of TCB-splines
Multiscale structural optimisation with concurrent coupling between scales
A robust three-dimensional multiscale topology optimisation framework with concurrent coupling between scales is presented. Concurrent coupling ensures that only the microscale data required to evaluate the macroscale model during each iteration of optimisation is collected and results in considerable computational savings. This represents the principal novelty of the framework and permits a previously intractable number of design variables to be used in the parametrisation of the microscale geometry, which in turn enables accessibility to a greater range of mechanical point properties during optimisation. Additionally, the microscale data collected during optimisation is stored in a re-usable database, further reducing the computational expense of subsequent iterations or entirely new optimisation problems. Application of this methodology enables structures with precise functionally-graded mechanical properties over two-scales to be derived, which satisfy one or multiple functional objectives. For all applications of the framework presented within this thesis, only a small fraction of the microstructure database is required to derive the optimised multiscale solutions, which demonstrates a significant reduction in the computational expense of optimisation in comparison to contemporary sequential frameworks.
The derivation and integration of novel additive manufacturing constraints for open-walled microstructures within the concurrently coupled multiscale topology optimisation framework is also presented. Problematic fabrication features are discouraged through the application of an augmented projection filter and two relaxed binary integral constraints, which prohibit the formation of unsupported members, isolated assemblies of overhanging members and slender members during optimisation. Through the application of these constraints, it is possible to derive self-supporting, hierarchical structures with varying topology, suitable for fabrication through additive manufacturing processes.Open Acces
Discovering the hidden structure of financial markets through bayesian modelling
Understanding what is driving the price of a financial asset is a question that is currently mostly unanswered. In this work we go beyond the classic one step ahead prediction and instead construct models that create new information on the behaviour of these time series. Our aim is to get a better understanding of the hidden structures that drive the moves of each financial time series and thus the market as a whole.
We propose a tool to decompose multiple time series into economically-meaningful variables to explain the endogenous and exogenous factors driving their underlying variability. The methodology we introduce goes beyond the direct model forecast. Indeed, since our model continuously adapts its variables and coefficients, we can study the time series of coefficients and selected variables. We also present a model to construct the causal graph of relations between these time series and include them in the exogenous factors.
Hence, we obtain a model able to explain what is driving the move of both each specific time series and the market as a whole. In addition, the obtained graph of the time series provides new information on the underlying risk structure of this environment. With this deeper understanding of the hidden structure we propose novel ways to detect and forecast risks in the market. We investigate our results with inferences up to one month into the future using stocks, FX futures and ETF futures, demonstrating its superior performance according to accuracy of large moves, longer-term prediction and consistency over time. We also go in more details on the economic interpretation of the new variables and discuss the created graph structure of the market.Open Acces
Sparse Functional Linear Discriminant Analysis
Functional linear discriminant analysis offers a simple yet efficient method for classification, with the possibility of achieving a perfect classification. Several methods are proposed in the literature that mostly address the dimensionality of the problem. On the other hand, there is a growing interest in interpretability of the analysis, which favors a simple and sparse solution. In this work, we propose a new approach that incorporates a type of sparsity that identifies nonzero sub-domains in the functional setting, offering a solution that is easier to interpret without compromising performance. With the need to embed additional constraints in the solution, we reformulate the functional linear discriminant analysis as a regularization problem with an appropriate penalty. Inspired by the success of ℓ1-type regularization at inducing zero coefficients for scalar variables, we develop a new regularization method for functional linear discriminant analysis that incorporates an L1-type penalty, ∫ |f|, to induce zero regions. We demonstrate that our formulation has a well-defined solution that contains zero regions, achieving a functional sparsity in the sense of domain selection. In addition, the misclassification probability of the regularized solution is shown to converge to the Bayes error if the data are Gaussian. Our method does not presume that the underlying function has zero regions in the domain, but produces a sparse estimator that consistently estimates the true function whether or not the latter is sparse. Numerical comparisons with existing methods demonstrate this property in finite samples with both simulated and real data examples
Towards a non-equilibrium thermodynamic theory of ecosystem assembly and development
Non-equilibrium thermodynamics has had a significant historic influence on the development
of theoretical ecology, even informing the very concept of an ecosystem. Much of this influence
has manifested as proposed extremal principles. These principles hold that systems will tend
to maximise certain thermodynamic quantities, subject to the other constraints they operate
under. A particularly notable extremal principle is the maximum entropy production principle
(MaxEPP); that systems maximise their rate of entropy production. However, these principles
are not robustly based in physical theory, and suffer from treating complex ecosystems in
an extremely coarse manner. To address this gap, this thesis derives a limited but physically
justified extremal principle, as well as carrying out a detailed investigation of the impact of
non-equilibrium thermodynamic constraints on the assembly of microbial communities. The extremal
principle we obtain pertains to the switching between states in simple bistable systems,
with switching paths that generate more entropy being favoured. Our detailed investigation
into microbial communities involved developing a novel thermodynamic microbial community
model, using which we found the rate of ecosystem development to be set by the availability
of free-energy. Further investigation was carried out using this model, demonstrating the way
that trade-offs emerging from fundamental thermodynamic constraints impact the dynamics of
assembling microbial communities. Taken together our results demonstrate that theory can be
developed from non-equilibrium thermodynamics, that is both ecologically relevant and physically
well grounded. We find that broad extremal principles are unlikely to be obtained, absent
significant advances in the field of stochastic thermodynamics, limiting their applicability to
ecology. However, we find that detailed consideration of the non-equilibrium thermodynamic
mechanisms that impact microbial communities can broaden our understanding of their assembly
and functioning.Open Acces
- …