A complexity analysis of Policy Iteration through combinatorial matrices arising from Unique Sink Orientations

Unique Sink Orientations (USOs) are an appealing abstraction of several major optimization problems of applied mathematics such as Linear Programming (LP), Markov Decision Processes (MDPs) or 2-player Turn Based Stochastic Games (2TBSGs). A polynomial time algorithm to find the sink of a USO would translate into a strongly polynomial time algorithm to solve the aforementioned problems—a major quest for all three cases. In the case of an acyclic USO of a cube, a situation that captures both MDPs and 2TBSGs, one can apply the well-known Policy Iteration (PI) algorithm. The study of its complexity is the object of this work. Despite its exponential worst case complexity, the principle of PI is a powerful source of inspiration for other methods. In 2012, Hansen and Zwick introduced a new combinatorial relaxation of the complexity problem for PI resulting in what we call Order-Regular (OR) matrices. They conjectured that the maximum number of rows of such matrices—an upper bound on the number of steps of PI—should follow the Fibonacci sequence. As our first contribution, we disprove the lower bound part of Hansen and Zwick's conjecture. Then, for our second contribution, we (exponentially) improve the Ω(1.4142n) lower bound on the number of steps of PI from Schurr and Szabó in the case of OR matrices and obtain an Ω(1.4269n) bound. © 2017 Elsevier B.V

Upper Bounds for All and Max-gain Policy Iteration Algorithms on Deterministic MDPs

Policy Iteration (PI) is a widely used family of algorithms to compute optimal policies for Markov Decision Problems (MDPs). We derive upper bounds on the running time of PI on Deterministic MDPs (DMDPs): the class of MDPs in which every state-action pair has a unique next state. Our results include a non-trivial upper bound that applies to the entire family of PI algorithms; another to all "max-gain" switching variants; and affirmation that a conjecture regarding Howard's PI on MDPs is true for DMDPs. Our analysis is based on certain graph-theoretic results, which may be of independent interest.Comment: Added new bounds for two state MDP

Anatomy of the Local Optima Level in Combinatorial Optimisation

Many situations in daily life represent complex combinatorial optimisation problems. These include issues such as efficient fuel consumption, nurse scheduling, or distribution of humanitarian aid. There are many algorithms that attempt to solve these problems but the ability to understand their likely performance on a given problem is still lacking. Fitness landscape analysis identifies some of the reasons why metaheuristic algorithms behave in a particular way. The Local Optima Network (LON) model, proposed in 2008, encodes local optima connectivity in fitness landscapes. In this approach, nodes are local optima and edges encode transitions between these optima. A LON provides a static image of the dynamics of algorithm-problem inter- play. Analysing these structures provides insights into the reactions between optimisation problems and metaheuristic search algorithms. This thesis proposes that analysis of the local optima space of combinatorial fitness landscapes encoded using a LON provides important information concerning potential search algorithm performance. It considers the question as to whether or not features of LONs can contribute to explaining or predicting the outcome of trying to optimise an associated combinatorial problem. Topological landscape features of LONs are proposed, analysed and compared. Benchmark and novel problem instances are studied; both types of problem are sampled and in some cases exhaustively-enumerated such that LONs can be extracted for analysis. Investigations into the nature and biases of LON construction algorithms are conducted and compared. Contributions include aligning fractal geometry to the study of LONs; proposals for novel ways to compute fractal dimension from these structures; comparing the power of different LON construction algorithms for explaining algorithm performances; and analysing the interplay between algorithmic operations and infeasible regions in the local optima space using LONs as a tool. Throughout the thesis, large scale structural patterns in fitness landscapes are shown to be strongly linked with metaheuristic algorithm performance. This includes arrangements of local optima funnel structures; spatial and geometric complexity in the LON (measured by their fractal dimensionality) and fitness levels in the space of local optima. These features are demonstrated to have explanatory or predictive ability with respect to algorithm performance for the underlying combinatorial problems. The results presented here indicate that large topological patterns in fitness landscapes are important during metaheuristic search algorithm design. In many cases they are incontrovertibly linked to the success of the algorithm. These results indicate that use of the suggested fitness landscape measures would be highly beneficial when considering the design of search algorithms for a given problem domain

Supersymmetric Field Theories, Scattering Amplitudes and the Grassmannian

In this thesis we carry out a detailed investigation of a class of four-dimensional N=1 gauge theories, known as Bipartite Field Theories (BFTs), and their utility in integrable systems and scattering amplitudes in 4-dimensional N=4 Super-Yang-Mills (SYM). We present powerful combinatorial tools for analyzing the moduli spaces of BFTs, and find an interesting connection with the matching and matroid polytopes, which play a central role in the understanding of the Grassmannian. We use the tools from BFTs to construct (0+1)-dimensional cluster integrable systems, and propose a way of obtaining (1+1)- and (2+1)-dimensional integrable field theories. Using the matching and matroid polytopes of BFTs, we analyze the singularity structure of planar and non-planar on-shell diagrams, which are central to modern developments of scattering amplitudes in N=4 SYM. In so doing, we uncover a new way of obtaining the positroid stratication of the Grassmannian. We use tools from BFTs to understand the boundary structure of the amplituhedron, a recently found geometric object whose volume calculates the integrand of scattering amplitudes in planar N=4 SYM theory. We provide the most comprehensive study of the geometry of the amplituhedron to date. We also present a detailed study of non-planar on-shell diagrams, constructing the on-shell form using two new, independent methods: a non-planar boundary measurement valid for arbitrary non-planar graphs, and a proposal for a combinatorial method to determine the on-shell form directly from the graph