Physarum Powered Differentiable Linear Programming Layers and Applications

Consider a learning algorithm, which involves an internal call to an optimization routine such as a generalized eigenvalue problem, a cone programming problem or even sorting. Integrating such a method as layers within a trainable deep network in a numerically stable way is not simple -- for instance, only recently, strategies have emerged for eigendecomposition and differentiable sorting. We propose an efficient and differentiable solver for general linear programming problems which can be used in a plug and play manner within deep neural networks as a layer. Our development is inspired by a fascinating but not widely used link between dynamics of slime mold (physarum) and mathematical optimization schemes such as steepest descent. We describe our development and demonstrate the use of our solver in a video object segmentation task and meta-learning for few-shot learning. We review the relevant known results and provide a technical analysis describing its applicability for our use cases. Our solver performs comparably with a customized projected gradient descent method on the first task and outperforms the very recently proposed differentiable CVXPY solver on the second task. Experiments show that our solver converges quickly without the need for a feasible initial point. Interestingly, our scheme is easy to implement and can easily serve as layers whenever a learning procedure needs a fast approximate solution to a LP, within a larger network

Numerical Optimisation Problems in Finance

This thesis consists of four projects regarding numerical optimisation and financial derivative pricing. The first project deals with the calibration of the Heston stochastic volatility model. A method using the Levenberg-Marquardt algorithm with the analytical gradient is developed. It is so far the fastest Heston model calibrator and meets the speed requirement of practical trading. In the second project, a triply-nested iterative method for the implementation of interior-point methods for linear programs is proposed. It is the first time that an interior-point method entirely based on iterative solvers succeeds in solving a fairly large number of linear programming instances from benchmark libraries under the standard stopping criteria. The third project extends the Black-Scholes valuation to a complex volatility parameter and presents its singularities at zero and infinity. Fractals that describe the chaotic nature of the Newton-Raphson calculation of the implied volatility are shown for different moneyness values. Among other things, these fractals visualise dramatically the effect of an existing modification for improving the stability and convergence of the search. The project studies scientifically an interesting problem widespread in the financial industry, while revealing artistic values stemming from mathematics. The fourth project investigates the consistency of a class of stochastic volatility models under spot rate inversion, and hence their suitability in the foreign exchange market. The general formula of the model parameters for the inversion rate is given, which provides basis for further investigation. The result is further extended to the affine stochastic volatility model. The Heston model, among the other members in the stochastic volatility family, is the only one that we found to be consistent under the spot inversion. The conclusion on the Heston model verifies the arbitrage opportunity in the variance swap