Riemannian Optimization for Convex and Non-Convex Signal Processing and Machine Learning Applications

The performance of most algorithms for signal processing and machine learning applications highly depends on the underlying optimization algorithms. Multiple techniques have been proposed for solving convex and non-convex problems such as interior-point methods and semidefinite programming. However, it is well known that these algorithms are not ideally suited for large-scale optimization with a high number of variables and/or constraints. This thesis exploits a novel optimization method, known as Riemannian optimization, for efficiently solving convex and non-convex problems with signal processing and machine learning applications. Unlike most optimization techniques whose complexities increase with the number of constraints, Riemannian methods smartly exploit the structure of the search space, a.k.a., the set of feasible solutions, to reduce the embedded dimension and efficiently solve optimization problems in a reasonable time. However, such efficiency comes at the expense of universality as the geometry of each manifold needs to be investigated individually. This thesis explains the steps of designing first and second-order Riemannian optimization methods for smooth matrix manifolds through the study and design of optimization algorithms for various applications. In particular, the paper is interested in contemporary applications in signal processing and machine learning, such as community detection, graph-based clustering, phase retrieval, and indoor and outdoor location determination. Simulation results are provided to attest to the efficiency of the proposed methods against popular generic and specialized solvers for each of the above applications

Constrained optimization using a conjugate gradient technique, with control applications

Imperial Users onl

Extension of zigzag search algorithms for power system multi-objective optimization

The work presented in this thesis focuses on the application and extension the zigzag search algorithms in power systems. The zigzag search method is a multi-objective algorithm which has recently been applied in multiple engineering fields, such as oil well replacement, with fast computational time and accurate results.Multi-objective optimization algorithms in power systems have been investigated for years. Most of the literatures focus on evolutionary algorithms (EA) such as a non-dominated sorting genetic algorithm (NSGA) or multi-objective particle swarm optimization (MOPSO) for their simplicity and ease of implementation. However, there have been several issues regarding the evolutionary algorithm (EA). For example, the computational time of EA is significant and the parameter configurations are complicated. Other approaches mainly reply on the weight sum method by lumping together different objective functions to form a new single objective function; however, the priority is hard to determine and the characteristic between different objectives may be lost.In order to improve the performance of power system multi-objective optimization problems, this thesis will first introduce the zigzag search algorithm. Second, by modifying the classic zigzag search algorithm, the zigzag interior point method and zigzag genetic algorithm method will both be proposed to broaden the applications of the classic zigzag search method. Also, in order to provide a systematic method for step-size configuration, a zigzag search method with adaptive step-size will be proposed. Thirdly, all algorithms will be applied to several practical power system multi-objective problems to demonstrate their practicability and effectiveness.The case study will be carried out on a modified IEEE 30-bus system and the IEEE 118-bus system. A comparison will be made with classic multiobjective algorithms which have been widely applied in power systems to demonstrate the effectiveness and efficiency of the proposed zigzag search methods

Improved tabu search and simulated annealing methods for nonlinear data assimilation

Nonlinear data assimilation can be a very challenging task. Four local search methods are proposed for nonlinear data assimilation in this paper. The methods work as follows: At each iteration, the observation operator is linearized around the current solution, and a gradient approximation of the three dimensional variational (3D-Var) cost function is obtained. Then, samples along potential steepest descent directions of the 3D-Var cost function are generated, and the acceptance/rejection criteria for such samples are similar to those proposed by the Tabu Search and the Simulated Annealing framework. In addition, such samples can be drawn within certain sub-spaces so as to reduce the computational effort of computing search directions. Once a posterior mode is estimated, matrix-free ensemble Kalman filter approaches can be implemented to estimate posterior members. Furthermore, the convergence of the proposed methods is theoretically proven based on the necessary assumptions and conditions. Numerical experiments have been performed by using the Lorenz-96 model. The numerical results show that the cost function values on average can be reduced by several orders of magnitudes by using the proposed methods. Even more, the proposed methods can converge faster to posterior modes when sub-space approximations are employed to reduce the computational efforts among iterations