176 research outputs found
Physarum Powered Differentiable Linear Programming Layers and Applications
Consider a learning algorithm, which involves an internal call to an
optimization routine such as a generalized eigenvalue problem, a cone
programming problem or even sorting. Integrating such a method as layers within
a trainable deep network in a numerically stable way is not simple -- for
instance, only recently, strategies have emerged for eigendecomposition and
differentiable sorting. We propose an efficient and differentiable solver for
general linear programming problems which can be used in a plug and play manner
within deep neural networks as a layer. Our development is inspired by a
fascinating but not widely used link between dynamics of slime mold (physarum)
and mathematical optimization schemes such as steepest descent. We describe our
development and demonstrate the use of our solver in a video object
segmentation task and meta-learning for few-shot learning. We review the
relevant known results and provide a technical analysis describing its
applicability for our use cases. Our solver performs comparably with a
customized projected gradient descent method on the first task and outperforms
the very recently proposed differentiable CVXPY solver on the second task.
Experiments show that our solver converges quickly without the need for a
feasible initial point. Interestingly, our scheme is easy to implement and can
easily serve as layers whenever a learning procedure needs a fast approximate
solution to a LP, within a larger network
On the Convergence Time of a Natural Dynamics for Linear Programming
We consider a system of nonlinear ordinary differential equations for the solution of linear programming (LP) problems that was first proposed in the mathematical biology literature as a model for the foraging behavior of acellular slime mold Physarum polycephalum, and more recently considered as a method to solve LP instances. We study the convergence time of the continuous Physarum dynamics in the context of the linear programming problem, and derive a new time bound to approximate optimality that depends on the relative entropy between projected versions of the optimal point and of the initial point. The bound scales logarithmically with the LP cost coefficients and linearly with the inverse of the relative accuracy, establishing the efficiency of the dynamics for arbitrary LP instances with positive costs
Two Results on Slime Mold Computations
We present two results on slime mold computations. In wet-lab experiments
(Nature'00) by Nakagaki et al. the slime mold Physarum polycephalum
demonstrated its ability to solve shortest path problems. Biologists proposed a
mathematical model, a system of differential equations, for the slime's
adaption process (J. Theoretical Biology'07). It was shown that the process
convergences to the shortest path (J. Theoretical Biology'12) for all graphs.
We show that the dynamics actually converges for a much wider class of
problems, namely undirected linear programs with a non-negative cost vector.
Combinatorial optimization researchers took the dynamics describing slime
behavior as an inspiration for an optimization method and showed that its
discretization can -approximately solve linear programs with
positive cost vector (ITCS'16). Their analysis requires a feasible starting
point, a step size depending linearly on , and a number of steps
with quartic dependence on , where is
the difference between the smallest cost of a non-optimal basic feasible
solution and the optimal cost ().
We give a refined analysis showing that the dynamics initialized with any
strongly dominating point converges to the set of optimal solutions. Moreover,
we strengthen the convergence rate bounds and prove that the step size is
independent of , and the number of steps depends logarithmically
on and quadratically on
Physarum Inspired Dynamics to Solve Semi-Definite Programs
Physarum Polycephalum is a Slime mold that can solve the shortest path problem. A mathematical model based on the Physarum's behavior, known as the Physarum Directed Dynamics, can solve positive linear programs. In this paper, we will propose a Physarum based dynamic based on the previous work and introduce a new way to solve positive Semi-Definite Programming (SDP) problems, which are more general than positive linear programs. Empirical results suggest that this extension of the dynamic can solve the positive SDP showing that the nature-inspired algorithm can solve one of the hardest problems in the polynomial domain. In this work, we will formulate an accurate algorithm to solve positive and some non-negative SDPs and formally prove some key characteristics of this solver thus inspiring future work to try and refine this method
- …