Probabilistic Constrained Programming: A Reduced Gradient Algorithm Implemented on PC

The described solution technique for stochastic linear programs with one joint probability constraint and with multinormal right-hand sides was developed and implemented in the frame of the IIASA contracted study "Modelling of interconnected power systems". Very cautious use of efficient subroutines made it possible to solve such a numerically complicated optimization problem on IBM/PC-XT or AT compatibles

Multiple object tracking with context awareness

[no abstract

The Dual Forest Iteration Method for The Stochastic Transportation Problem

This paper presents a dual forest iteration algorithm for solving the stochastic transportation problem. The algorithm iterates from one dual forest to another with the values of the dual objective function strictly increasing in the nondegenerate case. It therefore converges in a finite number of steps. At each step it is necessary to solve at most two one-dimensional monotone equations. If the computation is interrupted before completion, a primal feasible solution, and upper and lower bounds to the optimal value of the objective function can be obtained. A numerical example is also presented

Optimization algorithms for decision tree induction

Aufgrund der guten Interpretierbarkeit gehören Entscheidungsbäume zu den am häufigsten verwendeten Modellen des maschinellen Lernens zur Lösung von Klassifizierungs- und Regressionsaufgaben. Ihre Vorhersagen sind oft jedoch nicht so genau wie die anderer Modelle. Der am weitesten verbreitete Ansatz zum Lernen von Entscheidungsbäumen ist die Top-Down-Methode, bei der rekursiv neue Aufteilungen anhand eines einzelnen Merkmals eingefuhrt werden, die ein bestimmtes Aufteilungskriterium minimieren. Eine Möglichkeit diese Strategie zu verbessern und kleinere und genauere Entscheidungsbäume zu erzeugen, besteht darin, andere Arten von Aufteilungen zuzulassen, z.B. welche, die mehrere Merkmale gleichzeitig berücksichtigen. Solche zu bestimmen ist allerdings deutlich komplexer und es sind effektive Optimierungsalgorithmen notwendig um optimale Lösungen zu finden. Für numerische Merkmale sind Aufteilungen anhand affiner Hyperebenen eine Alternative zu univariaten Aufteilungen. Leider ist das Problem der optimalen Bestimmung der Hyperebenparameter im Allgemeinen NP-schwer. Inspiriert durch die zugrunde liegende Problemstruktur werden in dieser Arbeit daher zwei Heuristiken zur näherungsweisen Lösung dieses Problems entwickelt. Die erste ist eine Kreuzentropiemethode, die iterativ Stichproben von der von-Mises-Fisher-Verteilung zieht und deren Parameter mithilfe der besten Elemente daraus verbessert. Die zweite ist ein Simulated-Annealing-Verfahren, das eine Pivotstrategie zur Erkundung des Lösungsraums nutzt. Aufgrund der gleichzeitigen Verwendung aller numerischen Merkmale sind generelle Hyperebenenaufteilungen jedoch schwer zu interpretieren. Als Alternative wird in dieser Arbeit daher die Verwendung von bivariaten Hyperebenenaufteilungen vorgeschlagen, die Linien in dem von zwei Merkmalen aufgespannten Unterraum entsprechen. Mit diesen ist es möglich, den Merkmalsraum deutlich effizienter zu unterteilen als mit univariaten Aufteilungen. Gleichzeitig sind sie aufgrund der Beschränkung auf zwei Merkmale gut interpretierbar. Zur optimalen Bestimmung der bivariaten Hyperebenenaufteilungen wird ein Branch-and-Bound-Verfahren vorgestellt. Darüber hinaus wird ein Branch-and-Bound-Verfahren zur Bestimmung optimaler Kreuzaufteilungen entwickelt. Diese können als Kombination von zwei standardmäßigen univariaten Aufteilung betrachtet werden und sind in Situationen nützlich, in denen die Datenpunkte nur schlecht durch einzelne lineare Aufteilungen separiert werden können. Die entwickelten unteren Schranken für verunreinigungsbasierte Aufteilungskriterien motivieren ebenfalls ein einfaches, aber effektives Branch-and-Bound-Verfahren zur Bestimmung optimaler Aufteilungen nominaler Merkmale. Aufgrund der Komplexität des zugrunde liegenden Optimierungsproblems musste man bisher nominale Merkmale mittels Kodierungsschemata in numerische umwandeln oder Heuristiken nutzen, um suboptimale nominale Aufteilungen zu bestimmen. Das vorgeschlagene Branch-and-Bound-Verfahren bietet eine nützliche Alternative für viele praktische Anwendungsfälle. Schließlich wird ein genetischer Algorithmus zur Induktion von Entscheidungsbäumen als Alternative zur Top-Down-Methode vorgestellt.Decision trees are among the most commonly used machine learning models for solving classification and regression tasks due to their major advantage of being easy to interpret. However, their predictions are often not as accurate as those of other models. The most widely used approach for learning decision trees is to build them in a top-down manner by introducing splits on a single variable that minimize a certain splitting criterion. One possibility of improving this strategy to induce smaller and more accurate decision trees is to allow different types of splits which, for example, consider multiple features simultaneously. However, finding such splits is usually much more complex and effective optimization methods are needed to determine optimal solutions. An alternative to univarate splits for numerical features are oblique splits which employ affine hyperplanes to divide the feature space. Unfortunately, the problem of determining such a split optimally is known to be NP-hard in general. Inspired by the underlying problem structure, two new heuristics are developed for finding near-optimal oblique splits. The first one is a cross-entropy optimization method which iteratively samples points from the von Mises-Fisher distribution and updates its parameters based on the best performing samples. The second one is a simulated annealing algorithm that uses a pivoting strategy to explore the solution space. As general oblique splits employ all of the numerical features simultaneously, they are hard to interpret. As an alternative, in this thesis, the usage of bivariate oblique splits is proposed. These splits correspond to lines in the subspace spanned by two features. They are capable of dividing the feature space much more efficiently than univariate splits while also being fairly interpretable due to the restriction to two features only. A branch and bound method is presented to determine these bivariate oblique splits optimally. Furthermore, a branch and bound method to determine optimal cross-splits is presented. These splits can be viewed as combinations of two standard univariate splits on numeric attributes and they are useful in situations where the data points cannot be separated well linearly. The cross-splits can either be introduced directly to induce quaternary decision trees or, which is usually better, they can be used to provide a certain degree of foresight, in which case only the better of the two respective univariate splits is introduced. The developed lower bounds for impurity based splitting criteria also motivate a simple but effective branch and bound algorithm for splits on nominal features. Due to the complexity of determining such splits optimally when the number of possible values for the feature is large, one previously had to use encoding schemes to transform the nominal features into numerical ones or rely on heuristics to find near-optimal nominal splits. The proposed branch and bound method may be a viable alternative for many practical applications. Lastly, a genetic algorithm is proposed as an alternative to the top-down induction strategy

Tools for primal degenerate linear programs: IPS, DCA, and PE

ABSTRACT: This paper describes three recent tools for dealing with primal degeneracy in linear programming. The first one is the improved primal simplex (IPS) algorithm which turns degeneracy into a possible advantage. The constraints of the original problem are dynamically partitioned based on the numerical values of the current basic variables. The idea is to work only with those constraints that correspond to nondegenerate basic variables. This leads to a row-reduced problem which decreases the size of the current working basis. The main feature of IPS is that it provides a nondegenerate pivot at every iteration of the solution process until optimality is reached. To achieve such a result, a negative reduced cost convex combination of the variables at their bounds is selected, if any. This pricing step provides a necessary and sufficient optimality condition for linear programming. The second tool is the dynamic constraint aggregation (DCA), a constructive strategy specifically designed for set partitioning constraints. It heuristically aims to achieve the properties provided by the IPS methodology. We bridge the similarities and differences of IPS and DCA on set partitioning models. The final tool is the positive edge (PE) rule. It capitalizes on the compatibility definition to determine the status of a column vector and the associated variable during the reduced cost computation. Within IPS, the selection of a compatible variable to enter the basis ensures a nondegenerate pivot, hence PE permits a trade-off between strict improvement and high, reduced cost degenerate pivots. This added value is obtained without explicitly computing the updated column components in the simplex tableau. Ultimately, we establish tight bonds between these three tools by going back to the linear algebra framework from which emanates the so-called concept of subspace basis

Experimental analysis of the simplex method for the multicommodity flow problem

This thesis investigates the multicommodity min-cost flow (MMCF) prob- lem. We aim to contribute to the search for a combinatorial algorithm for MMCF. The simplex method is employed to examine the vertices of the poly- hedron of feasible solutions using experimental analysis on a set of publicly available MMCF instances. To achieve this, we develop a solver capable of tracing solutions in each iteration of the algorithm in exact arithmetic, which was not available in existing solvers. Our investigation focuses on the frac- tionality of MMCF problems and the impact of different pivoting rules, par- ticularly whether the fractionality is exponential or polynomial with respect to increasing dimension. Our findings suggest that fractionality exhibits ex- ponential behavior.Tato práce se zabývá problémem multikomoditního toku minimální ceny (MMCF). Naším cílem je přispět k hledání kombinatorického algoritmu pro MMCF. Použili jsme simplexovou metodu k experimentálnímu prozkoumání vrcholů polyedru přípustných řešení na sadě veřejně dostupných instancí MMCF. K dosažení tohoto cíle jsme vyvinuli řešič, který je schopen sledovat řešení v každé iteraci algoritmu v přesné aritmetice; tato funkcionalita ne- byla k dispozici v existujících řešičích. Zaměřujeme se na zlomkovost MMCF instancí a vliv volby pivotovacího pravidla, zejména zda je zlomkovost expo- nenciální nebo polynomiální s ohledem na rostoucí dimenzi problému. Naše zjištění naznačují, že zlomkovost vykazuje exponenciální chování.Informatický ústav Univerzity KarlovyComputer Science Institute of Charles UniversityFaculty of Mathematics and PhysicsMatematicko-fyzikální fakult