Refined MDP-based branch-and-fix algorithm for the Hamiltonian cycle problem

We consider the famous Hamiltonian cycle problem (HCP) embedded in a Markov decision process (MDP). More specifically, we consider the HCP as an optimisation problem over the space of occupation measures induced by the MDP's stationary policies. In recent years, this approach to the HCP has led to a number of alternative formulations and algorithmic approaches. In this paper, we focus on a specific embedding because of the work of Feinberg. We present a "branch-and-fix" type algorithm that solves the HCP. At each branch of the algorithm, only a linear program needs to be solved and the dimensions of the successive linear programs are shrinking rather than expanding. Because the nodes of the branch-and-fix tree correspond to specially structured l-randomised policies, we characterise the latter. This characterisation indicates that the total number of such policies is significantly smaller than the subset of all l-randomised policies. Finally, we present some numerical results

Adaptation of a Branching Algorithm to Solve the Multi-Objective Hamiltonian Cycle Problem

The Hamiltonian cycle problem (HCP) consists of finding a cycle of length N in an N-vertices graph. In this investigation, a graph G is considered with an associated set of matrices, in which each cell in the matrix corresponds to the weight of an arc. Thus, a multi-objective variant of the HCP is addressed and a Pareto set of solutions that minimizes the weights of the arcs for each objective is computed. To solve the HCP problem, the Branch-and-Fix algorithm is employed, a specific branching algorithm that uses the embedding of the problem in a particular stochastic process. To address the multi-objective HCP, the Branch-and-Fix algorithm is extended by computing different Hamiltonian cycles and fathoming the branches of the tree at earlier stages. The introduced anytime algorithm can produce a valid solution at any time of the execution, improving the quality of the Pareto Set as time increases.This project was funded by the ELKARTEK Research Programme of the Basque Government (project KK-2019/00068). This work has been possible thanks to the support of the computing infrastructure of the i2BASQUE academic network. The work of Roberto Santana was funded by the Basque Government (project IT-1244-19), and Spanish Ministry of Economy and Competitiveness MINECO (project TIN2016-78365-R)

Advances in Branch-and-Fix methods to solve the Hamiltonian cycle problem in manufacturing optimization

159 p.Esta tesis parte del problema de la optimización de la ruta de la herramienta donde se contribuye con unsistema de soporte para la toma de decisiones que genera rutas óptimas en la tecnología de FabricaciónAditiva. Esta contribución sirve como punto de partida o inspiración para analizar el problema del cicloHamiltoniano (HCP). El HCP consiste en visitar todos los vértices de un grafo dado una única vez odeterminar que dicho ciclo no existe. Muchos de los métodos propuestos en la literatura sirven paragrafos no dirigidos y los que se enfocan en los grafos dirigidos no han sido implementados ni testeados.Uno de los métodos para resolver el problema es el Branch-and-Fix (BF), un método exacto que utiliza latranformación del HCP a un problema continuo. El BF es un algoritmo de ramificación que consiste enconstruir un árbol de decisión donde en cada vértice dos problemas lineales son resueltos. Este método hasido testeado en grafos de tamaño pequeño y por ello, no se ha estudiado en profundidad las limitacionesque puede presentar. Por ello, en esta tesis se proponen cuatro contribuciones metodológicasrelacionadas con el HCP y el BF: 1) mejorar la enficiencia del BF en diferentes aspectos, 2) proponer unmétodo de ramificación global, 3) proponer un método del BF colapsado, 4) extender el HCP a unescenario multi-objetivo y proponer un método para resolverlo

Hamiltonian cycles and subsets of discounted occupational measures

We study a certain polytope arising from embedding the Hamiltonian cycle problem in a discounted Markov decision process. The Hamiltonian cycle problem can be reduced to finding particular extreme points of a certain polytope associated with the input graph. This polytope is a subset of the space of discounted occupational measures. We characterize the feasible bases of the polytope for a general input graph $G$, and determine the expected numbers of different types of feasible bases when the underlying graph is random. We utilize these results to demonstrate that augmenting certain additional constraints to reduce the polyhedral domain can eliminate a large number of feasible bases that do not correspond to Hamiltonian cycles. Finally, we develop a random walk algorithm on the feasible bases of the reduced polytope and present some numerical results. We conclude with a conjecture on the feasible bases of the reduced polytope.Comment: revised based on referees comment

Machine learning methods for robust quantum optimal control

Quantum technologies have the potential to revolutionize many classical tasks, particularly including sensing and simulation applications. Yet their full potential is limited by the presence of noise, amongst other issues. This thesis addresses the problem of quantum optimal control of a controllable system with noisy dynamics and an uncertain theoretical description. Towards this goal, this thesis makes two contributions. Firstly, it develops a novel robustness measure called the Robustness Infidelity Measure (RIM) for certification of robustness of optimal control schemes, agnostic of the acquisition method. The RIM is a statistical measure and it can be used to compare the robustness of different schemes. Secondly, this thesis develops novel optimization techniques based on Reinforcement Learning (RL) for robust optimization of noisy quantum dynamics with model uncertainties. In particular, a model-based RL algorithm is proposed that is able to improve over direct applications of model-free RL algorithms in terms of experimental resource consumption. This is done via incorporation of partial knowledge of the uncertain model whilst the rest is learned using experimental data. Our approach highlights the potential of extending pure model-free methods towards model-based approaches, with a learnable model, for noisy optimization tasks and brings RL algorithms closer to deployment on near-term quantum devices. We evaluate the RIM and various model-free RL algorithms on a number of benchmark problems. Our results show that the RIM is a valuable tool for assessing the robustness of quantum control schemes. Moreover, we demonstrate that RL algorithms are able to generate robust control schemes which outperform schemes generated using other methods. We also show how learned models of noisy quantum dynamics can be leveraged to increase the optimality of quantum control schemes found by RL algorithms whilst retaining their robustness performance

Optimising arrival management in air traffic control

Efficient landing is a key component of improving air transport, both for passengers' experience and fuel consumption. With just two runways, Heathrow airport was running at 98% capacity before the pandemic. The existing queuing system allows buffer time for the aircraft to land on one runway, but this can add delays to journeys and be fuel inefficient. With the recovery of the travel industry after the 2020 pandemic, improving landing procedures remains a pertinent problem to NATS (who manages all the air traffic in UK airspace). In this thesis, we develop alternative methods to sequence aircraft as they approach for landing at Heathrow. In the first part of this thesis, we cast the arrival management problem in a reinforcement learning framework. We design a basic air traffic model and apply both table representation methods and nonlinear approximation with a neural network. Specifically, we compare the performance of Q-learning, SARSA, and DDPG on this environment. Further we explore dimension reduction/feature representation through path signatures. Finally we design multi-grid inspired neural network structures and see that these lead to faster training but ultimately comparable performance. For the second part, we look at the problem from a different perspective. We take inspiration from the theory of optimal transport and formulate an entropy-regularised optimisation problem. We design an algorithm with block gradient descent-like steps and note that the conflicts in the set-up of our problem introduces non-convexity even when working in the (convex) space of distributions over arrival times. By adding an additional 'considerate' cost, akin to a Pigouvian tax, the performance of the algorithm is enhanced. Finally, the last part of this thesis shows the flexibility of our approach. We adapt our work to apply to a new air traffic design concept being researched at NATS

Optimal GENCO bidding strategy

Electricity industries worldwide are undergoing a period of profound upheaval. The conventional vertically integrated mechanism is being replaced by a competitive market environment. Generation companies have incentives to apply novel technologies to lower production costs, for example: Combined Cycle units. Economic dispatch with Combined Cycle units becomes a non-convex optimization problem, which is difficult if not impossible to solve by conventional methods. Several techniques are proposed here: Mixed Integer Linear Programming, a hybrid method, as well as Evolutionary Algorithms. Evolutionary Algorithms share a common mechanism, stochastic searching per generation. The stochastic property makes evolutionary algorithms robust and adaptive enough to solve a non-convex optimization problem. This research implements GA, EP, and PS algorithms for economic dispatch with Combined Cycle units, and makes a comparison with classical Mixed Integer Linear Programming.;The electricity market equilibrium model not only helps Independent System Operator/Regulator analyze market performance and market power, but also provides Market Participants the ability to build optimal bidding strategies based on Microeconomics analysis. Supply Function Equilibrium (SFE) is attractive compared to traditional models. This research identifies a proper SFE model, which can be applied to a multiple period situation. The equilibrium condition using discrete time optimal control is then developed for fuel resource constraints. Finally, the research discusses the issues of multiple equilibria and mixed strategies, which are caused by the transmission network. Additionally, an advantage of the proposed model for merchant transmission planning is discussed.;A market simulator is a valuable training and evaluation tool to assist sellers, buyers, and regulators to understand market performance and make better decisions. A traditional optimization model may not be enough to consider the distributed, large-scale, and complex energy market. This research compares the performance and searching paths of different artificial life techniques such as Genetic Algorithm (GA), Evolutionary Programming (EP), and Particle Swarm (PS), and look for a proper method to emulate Generation Companies\u27 (GENCOs) bidding strategies.;After deregulation, GENCOs face risk and uncertainty associated with the fast-changing market environment. A profit-based bidding decision support system is critical for GENCOs to keep a competitive position in the new environment. Most past research do not pay special attention to the piecewise staircase characteristic of generator offer curves. This research proposes an optimal bidding strategy based on Parametric Linear Programming. The proposed algorithm is able to handle actual piecewise staircase energy offer curves. The proposed method is then extended to incorporate incomplete information based on Decision Analysis. Finally, the author develops an optimal bidding tool (GenBidding) and applies it to the RTS96 test system

Foundations of Software Science and Computation Structures

This open access book constitutes the proceedings of the 22nd International Conference on Foundations of Software Science and Computational Structures, FOSSACS 2019, which took place in Prague, Czech Republic, in April 2019, held as part of the European Joint Conference on Theory and Practice of Software, ETAPS 2019. The 29 papers presented in this volume were carefully reviewed and selected from 85 submissions. They deal with foundational research with a clear significance for software science