Interval propagation and search on directed acyclic graphs for numerical constraint solving

The fundamentals of interval analysis on directed acyclic graphs (DAGs) for global optimization and constraint propagation have recently been proposed in Schichl and Neumaier (J. Global Optim. 33, 541-562, 2005). For representing numerical problems, the authors use DAGs whose nodes are subexpressions and whose directed edges are computational flows. Compared to tree-based representations [Benhamou etal. Proceedings of the International Conference on Logic Programming (ICLP'99), pp. 230-244. Las Cruces, USA (1999)], DAGs offer the essential advantage of more accurately handling the influence of subexpressions shared by several constraints on the overall system during propagation. In this paper we show how interval constraint propagation and search on DAGs can be made practical and efficient by: (1) flexibly choosing the nodes on which propagations must be performed, and (2) working with partial subgraphs of the initial DAG rather than with the entire graph. We propose a new interval constraint propagation technique which exploits the influence of subexpressions on all the constraints together rather than on individual constraints. We then show how the new propagation technique can be integrated into branch-and-prune search to solve numerical constraint satisfaction problems. This algorithm is able to outperform its obvious contenders, as shown by the experiment

Verified global optimization for estimating the parameters of nonlinear models

Nonlinear parameter estimation is usually achieved via the minimization of some possibly non-convex cost function. Interval analysis allows one to derive algorithms for the guaranteed characterization of the set of all global minimizers of such a cost function when an explicit expression for the output of the model is available or when this output is obtained via the numerical solution of a set of ordinary differential equations. However, cost functions involved in parameter estimation are usually challenging for interval techniques, if only because of multi-occurrences of the parameters in the formal expression of the cost. This paper addresses parameter estimation via the verified global optimization of quadratic cost functions. It introduces tools for the minimization of generic cost functions. When an explicit expression of the output of the parametric model is available, significant improvements may be obtained by a new box exclusion test and by careful manipulations of the quadratic cost function. When the model is described by ODEs, some of the techniques available in the previous case may still be employed, provided that sensitivity functions of the model output with respect to the parameters are available

Rigorous solution techniques for numerical constraint satisfaction problems

A constraint satisfaction problem (e.g., a system of equations and inequalities) consists of a finite set of constraints specifying which value combinations from given variable domains are admitted. It is called numerical if its variable domains are continuous. Such problems arise in many applications, but form a difficult problem class since they are NP-hard. Solving a constraint satisfaction problem is to find one or more value combinations satisfying all its constraints. Numerical computations on floating-point numbers in computers often suffer from rounding errors. The rigorous control of rounding errors during numerical computations is highly desired in many applications because it would benefit the quality and reliability of the decisions based on the solutions found by the computations. Various aspects of rigorous numerical computations in solving constraint satisfaction problems are addressed in this thesis: search, constraint propagation, combination of inclusion techniques, and post-processing. The solution of a constraint satisfaction problem is essentially performed by a search. In this thesis, we propose a new complete search technique (i.e., it can find all solutions within a predetermined tolerance) for numerical constraint satisfaction problems. This technique is general and can be used in place of branching steps in most branch-and-prune methods. Moreover, this new technique speeds up the most recent general search strategy (often by an order of magnitude) and provides a concise representation of solutions. To make a constraint satisfaction problem easier to solve, a major approach, called constraint propagation, in the constraint programming1 field is often used to reduce the variable domains (by discarding redundant value combinations from the domains). Basing on directed acyclic graphs, we propose a new constraint propagation technique and a method for coordinating constraint propagation and search. More importantly, we propose a novel generic scheme for combining multiple inclusion techniques2 in numerical constraint propagation. This scheme allows bringing into the constraint propagation framework the strengths of various techniques coming from different fields. To illustrate the flexibility and efficiency of the generic scheme, we base on this scheme and devise several specific combination strategies for rigorous numerical constraint propagation using interval constraint propagation, interval arithmetic, affine arithmetic, and linear programming. Our experiments show that the new propagation techniques outperform previously available methods by 1 to 4 orders of magnitude or more in speed. We also propose several post-processing techniques for the representation of continuums of solutions. Based on connectedness, they allow grouping each cluster of connected solution subsets into a larger subset, thus allowing getting additional grouping information. Potentially, these techniques enable interval-based solution techniques to be alternatives to bounding-volume techniques in applications such as collision detection and interactive graphics. __________________________________________________ 1 Constraint programming is an approach to programming that relies on both reasoning and computing. 2 An inclusion technique is to include a set of interest into enclosures. It is also called an enclosure technique

Moving into higher dimensions of geometric constraint solving

Journal ArticleIn this paper, we present an approach to geometric constraint solving, based on degree of freedom analysis. Any geometric primitive (point, line, circle, plane, etc.) possesses an intrinsic degree of freedom in its embedding space which is usually two or three dimensional. Constraints reduce the degrees of freedom of an object (or a set of objects). We use graph algorithms to determine upper and lower bounds for the degrees of freedom of a set of constrained objects, symbolically. This analysis is then used to establish dependency graphs and evaluation schemes for symbolic or numeric solutions to constraint problems. The approach has been generalized for n-dimensional space, which, among other things, allows for a uniform handling of 2-D and 3-D constraint problems or algebraic constraints between scalar dimension. Also, higher than three dimensional solutions can be interpreted as approaches to over- and under- constrained problems. In this paper, we will present the theoretical background of the approach, and demonstrate how it can be applied within an interactive design environment

An incremental constraint-based approach to support engineering design.

Constraint-based systems are increasingly being used to support the design of products. Several commercial design systems based on constraints allow the geometry of a product to be specified and modified in a more natural and efficient way. However, it is now widely recognised the needs to have a close coupling of geometric constraints (i.e. parallel, tangent, etc) and engineering constraints (Le. performance, costs, weight, etc) to effectively support the preliminary design stages. This is an active research topic which is the subject of this thesis. As the design evolves, the size of the quation set which captures the constraints can get very large depending on the complexity of the product being designed. These constraints are expected to be solved efficiently to guarantee immediate feedback to the designer. Such requirement is also necessary to support constraint-based design within Virtual Environments, where it is necessary to have interactive speed. However, the majority of constraint-based design systems re-satisfy all constraints from scratch after the insertion of a new design constraint. This process is time consuming and therefore hinders interactive design performance when dealing with large constraint sets. This thesis reports research into the investigation of techniques to support interactive constraint-based design. The main focus of this work is on the development of incremental graph-based algorithms for satisfying a coupled set of engineering and geometric constraints. In this research, the design constraints, represented as simultaneous sets of linear and non-linear equations, are stored in a directed graph called Equation Graph. When a new constraint is imposed, local constraint propagation techniques are used to satisfy the constraint and update the current graph solution, incrementally. Constraint cycles are locally identified and satisfied within the Equation Graph. Therefore, these algorithms efiiciently solve large constraint sets to support interactive design. Techniques to support under-constrained geometry are also considered in this research. The concept of soft constraints is introduced to represent the degrees of freedom of the geometric entities. This is used to allow the incremental satisfaction of newly imposed constraints by exploiting under-constrained space. These soft constraints are also used to support direct manipulation of under-constrained geometric entities, enabling the designers to test the kinematic behaviour of the current assembly. A prototype constraint-based design system has been developed to demonstrate the feasibility of these algorithms to support preliminary desig

Business Decision Insight With Causal Bayesian Networks

Causal Bayesian Networks are a widely recognised tool for modelling the uncer- tainty of a wide range of processes, particularly when the nature of how different factors influence each other. The practice of utilising Causal Bayesian Network is now becoming a growing trend for business that want to fully understand the demands im- posed on them, and how best to adapt their business in order to be successful. When designing and building a Causal Bayesian Network, it is often necessary to consult with domain experts for information about the shape of the model but also the defini- tion of how the causal factors influence others. The definition of these influences can require the specification of a large volume of probability distributions, even if a lot of evidential data is available for analysis. Whilst the definition of the structure of the model can be a relatively simple task for a domain expert, providing the probability distributions is a much more difficult task. In this thesis I discuss a method whereby, given a model structure, a domain expert can provide simple descriptive meta-data so that a hypothetical probability distribution can be generated for the discrete model variables

Global optimisation for dynamic systems using novel overestimation reduction techniques

The optimisation of dynamic systems is of high relevance in chemical engineering as many practical systems can be described by ordinary differential equations (ODEs) or differential algebraic equations (DAEs). The current techniques for solving these problems rigorously to global optimality rely mainly on sequential approaches in which a branch and bound framework is used for solving the global optimisation part of the problem and a verified simulator (in which rounding errors are accounted for in the computations) is used for solving the dynamic constraints. The verified simulation part is the main bottleneck since tight bounds are difficult to obtain for high dimensional dynamic systems. Additionally, uncertainty in the form of, for example, intervals is introduced in the parameters of the dynamic constraints which are also the decision variables of the optimisation problem. Nevertheless, in the verified simulation the accumulation of trajectories that do not belong to the exact solution (overestimation) makes the state bounds overconservative and in the worst case they blow up and tend towards ±∞. In this thesis, methods for verified simulation in global optimisation for dynamic systems were investigated. A novel algorithm that uses an interval Taylor series (ITS) method with enhanced overestimation reduction capabilities was developed. These enhancements for the reduction of the overestimation rely on interval contractors (Krawczyk, Newton, ForwardBackward) and model reformulation based on pattern substitution and input scaling. The method with interval contractors was also extended to Taylor Models (TM) for comparison purposes. The two algorithms were tested on several case studies to demonstrate the effectiveness of the methods. The case studies have a different number of state variables and system parameters and they use uncertain amounts in some of the system parameters and initial conditions. Both of the methods were also used in a sequential approach to address the global optimisation for dynamic systems problem subject to uncertainty. The simulation results demonstrated that the ITS method with overestimation reduction techniques provided tighter state bounds with less computational expense than the traditional method. In the case of the forward-backward contractor additional constraints can be introduced that can potentially contribute significantly to the reduction of the overestimation. Similarly, the novel TM method with enhanced overestimation reduction capabilities provided tighter bounds than the TM method alone. On the other hand, the optimisation results showed that the global optimisation algorithm with the novel ITS method with overestimation reduction techniques converged faster to a rigorous solution due to the improved state bounds

Rigorous techniques for continuous constraint satisfaction problems

Diese Arbeit beschäftigt sich mit rigorosen Techniken für das Lösen kontinuierlicher Zulässigkeitsprobleme. Das heißt, wir suchen nach einem oder allen Punkte, genannt zulässige Punkte, die eine Familie von Gleichungen und/oder Ungleichungen erfüllen, die wir im Weiteren Nebenbedingungen nennen werden. Zahlreiche Anwendungen führen auf kontinuierliche Zulässigkeitsprobleme. Neue und bereits existierende moderne Methoden werden präsentiert und integriert in GloptLab, eine neue, leicht bedienbare Test- und Entwicklungsplattform zum Lösen quadratischer Zulässigkeitsprobleme. Der Lösungsalgorithmus beruht auf dem Grundprinzip von Branch-and-Prune und auf Filterung. Filterungsmethoden dienen zur Verkleinerung/Reduktion einer Box, definiert als das kartesische Produkt der Intervalle, die die Schranken an die Variablen festlegen. Um den Verlust zulässiger Punkte zu vermeiden, werden alle Fehlerabschätzungen rigoros mittels Intervallarithmetik und gerichteter Rundung durchgeführt. Das stellt sicher, dass alle Rechnungen auch in Gleitkommaarithmetik gültig sind. In der Doktorarbeit werden die folgenden Themen diskutiert: der mathematische Hintergrund, Algorithmen und Tests für Constraint-Propagation, strikt konvexe Einschließungen, lineare Relaxationen, das Berechnen, korrekte Benutzen und Verifizieren approximativ zulässiger Punkte, optimale Skalierung und diverse Hilfsmethoden. Insbesondere: - Constraint-Propagation basiert auf einer Folge von Schritten, die jeweils eine einzelne Nebenbedingung verwenden. Traditionelle Techniken werden durch eine spezielle quadratische Methode erweitert, die neue Verfahren für die Eliminierung bilinearer Einträge und für das Berechnen optimaler Einschließungen für separable quadratische Ausdrücke verwendet. - Eine quadratische Ungleichungsnebenbedingung, die eine positiv definite Hesse-Matrix besitzt, definiert ein Ellipsoid. Eine spezielle rundungsfehlerkontrollierte Version der Cholesky-Zerlegung wird verwendet, um die strikt konvexe quadratische Nebenbedingungen in Norm-Ungleichungen zu transformieren. Für diese ist es dann einfach, die Intervall-Hülle analytisch zu bestimmen. - Diverse Methoden für die Erzeugung linearer Relaxationen werden diskutiert, kombiniert und erweitert. Teilweise verbesserte, existierende und neue Verfahren für das rigorose Einschließen der Lösungsmenge linearer Systeme werden präsentiert. - Eine Vielzahl von Beispielen demonstrieren, dass die präsentierten Verfahren einander ergänzen. Außerdem zeigen sie, wie man Lösungsstrategien entwickelt, die Zulässigkeitsprobleme global und effizient lösen.This thesis contributes rigorous techniques for solving continuous constraint satisfaction problems, i.e., finding one or all points (called feasible points) satisfying a given family of equations and/or inequalities (called constraints). Many real word problems are continuous constraint satisfaction problems. New and old state of the art methods are presented, integrated in GloptLab, a new easy-to-use testing and development platform for solving quadratic constraint satisfaction problems. The basic solution principle is branch and prune and filtering. Filtering techniques tighten a box -- the Cartesian product of intervals defined by the bounds on the variables. In order to avoid a loss of feasible points, rigorous error estimation using interval arithmetic and directed rounding are used, to take care that all calculations are valid even though the calculations are performed in floating-point arithmetic only. Discussed are the mathematical background, algorithms and tests of constraint propagation, strictly convex enclosures, linear relaxations, finding, using and verifying approximately feasible points, optimal scaling and other auxiliary techniques. In particular: - Constraint propagation is based on a sequence of steps, each using a single constraint only. Traditional techniques are extended by special quadratic constraint propagation methods using new techniques for eliminating bilinear entries and finding optimal enclosures for separable quadratic expressions. - A quadratic inequality constraint with a positive definite Hessian defines an ellipsoid. A rounding error controlled version of the Cholesky factorization is used to transform a strictly convex quadratic constraint into a norm inequality, for which the interval hull is easy to compute analytically. - Different methods for computing linear relaxations are discussed, combined and extended. Partially improved and existing methods, as well as new approaches for rigorously enclosing the solution set of linear systems of inequalities are presented. - Numerous examples show that the above methods complement each other and how to create solution strategies that solve constraint satisfaction problems globally and efficiently