12 research outputs found
Promoting Generalization for Exact Solvers via Adversarial Instance Augmentation
Machine learning has been successfully applied to improve the efficiency of
Mixed-Integer Linear Programming (MILP) solvers. However, the learning-based
solvers often suffer from severe performance degradation on unseen MILP
instances -- especially on large-scale instances from a perturbed environment
-- due to the limited diversity of training distributions. To tackle this
problem, we propose a novel approach, which is called Adversarial Instance
Augmentation and does not require to know the problem type for new instance
generation, to promote data diversity for learning-based branching modules in
the branch-and-bound (B&B) Solvers (AdaSolver). We use the bipartite graph
representations for MILP instances and obtain various perturbed instances to
regularize the solver by augmenting the graph structures with a learned
augmentation policy. The major technical contribution of AdaSolver is that we
formulate the non-differentiable instance augmentation as a contextual bandit
problem and adversarially train the learning-based solver and augmentation
policy, enabling efficient gradient-based training of the augmentation policy.
To the best of our knowledge, AdaSolver is the first general and effective
framework for understanding and improving the generalization of both
imitation-learning-based (IL-based) and reinforcement-learning-based (RL-based)
B&B solvers. Extensive experiments demonstrate that by producing various
augmented instances, AdaSolver leads to a remarkable efficiency improvement
across various distributions
Combining dynamic and static scheduling in high-level synthesis
Field Programmable Gate Arrays (FPGAs) are starting to become mainstream devices for custom computing, particularly deployed in data centres. However, using these FPGA devices requires familiarity with digital design at a low abstraction level. In order to enable software engineers without a hardware background to design custom hardware, high-level synthesis (HLS) tools automatically transform a high-level program, for example in C/C++, into a low-level hardware description.
A central task in HLS is scheduling: the allocation of operations to clock cycles. The classic approach to scheduling is static, in which each operation is mapped to a clock cycle at compile time, but recent years have seen the emergence of dynamic scheduling, in which an operation’s clock cycle is only determined at run-time. Both approaches have their merits: static scheduling can lead to simpler circuitry and more resource sharing, while dynamic scheduling can lead to faster hardware when the computation has a non-trivial control flow.
This thesis proposes a scheduling approach that combines the best of both worlds. My idea is to use existing program analysis techniques in software designs, such as probabilistic analysis and formal verification, to optimize the HLS hardware. First, this thesis proposes a tool named DASS that uses a heuristic-based approach to identify the code regions in the input program that are amenable to static scheduling and synthesises them into statically scheduled components, also known as static islands, leaving the top-level hardware dynamically scheduled. Second, this thesis addresses a problem of this approach: that the analysis of static islands and their dynamically scheduled surroundings are separate, where one treats the other as black boxes. We apply static analysis including dependence analysis between static islands and their dynamically scheduled surroundings to optimize the offsets of static islands for high performance. We also apply probabilistic analysis to estimate the performance of the dynamically scheduled part and use this information to optimize the static islands for high area efficiency. Finally, this thesis addresses the problem of conservatism in using sequential control flow designs which can limit the throughput of the hardware. We show this challenge can be solved by formally proving that certain control flows can be safely parallelised for high performance. This thesis demonstrates how to use automated formal verification to find out-of-order loop pipelining solutions and multi-threading solutions from a sequential program.Open Acces
Understanding and Enhancing CDCL-based SAT Solvers
Modern conflict-driven clause-learning (CDCL) Boolean satisfiability (SAT) solvers routinely
solve formulas from industrial domains with millions of variables and clauses, despite the Boolean
satisfiability problem being NP-complete and widely regarded as intractable in general. At the
same time, very small crafted or randomly generated formulas are often infeasible for CDCL
solvers. A commonly proposed explanation is that these solvers somehow exploit the underlying
structure inherent in industrial instances. A better understanding of the structure of Boolean
formulas not only enables improvements to modern SAT solvers, but also lends insight as to why
solvers perform well or poorly on certain types of instances. Even further, examining solvers
through the lens of these underlying structures can help to distinguish the behavior of different
solving heuristics, both in theory and practice.
The first issue we address relates to the representation of SAT formulas. A given Boolean
satisfiability problem can be represented in arbitrarily many ways, and the type of encoding can
have significant effects on SAT solver performance. Further, in some cases, a direct encoding
to SAT may not be the best choice. We introduce a new system that integrates SAT solving
with computer algebra systems (CAS) to address representation issues for several graph-theoretic
problems. We use this system to improve the bounds on several finitely-verified conjectures
related to graph-theoretic problems. We demonstrate how our approach is more appropriate for
these problems than other off-the-shelf SAT-based tools.
For more typical SAT formulas, a better understanding of their underlying structural properties,
and how they relate to SAT solving, can deepen our understanding of SAT. We perform a largescale
evaluation of many of the popular structural measures of formulas, such as community
structure, treewidth, and backdoors. We investigate how these parameters correlate with CDCL
solving time, and whether they can effectively be used to distinguish formulas from different
domains. We demonstrate how these measures can be used as a means to understand the behavior
of solvers during search. A common theme is that the solver exhibits locality during search
through the lens of these underlying structures, and that the choice of solving heuristic can greatly
influence this locality. We posit that this local behavior of modern SAT solvers is crucial to their
performance.
The remaining contributions dive deeper into two new measures of SAT formulas. We first
consider a simple measure, denoted “mergeability,” which characterizes the proportion of input
clauses pairs that can resolve and merge. We develop a formula generator that takes as input a seed
formula, and creates a sequence of increasingly more mergeable formulas, while maintaining many
of the properties of the original formula. Experiments over randomly-generated industrial-like
instances suggest that mergeability strongly negatively correlates with CDCL solving time, i.e., as
the mergeability of formulas increases, the solving time decreases, particularly for unsatisfiable
instances.
Our final contribution considers whether one of the aforementioned measures, namely backdoor
size, is influenced by solver heuristics in theory. Starting from the notion of learning-sensitive
(LS) backdoors, we consider various extensions of LS backdoors by incorporating different branching
heuristics and restart policies. We introduce learning-sensitive with restarts (LSR) backdoors
and show that, when backjumping is disallowed, LSR backdoors may be exponentially smaller
than LS backdoors. We further demonstrate that the size of LSR backdoors are dependent on the
learning scheme used during search. Finally, we present new algorithms to compute upper-bounds
on LSR backdoors that intrinsically rely upon restarts, and can be computed with a single run of
a SAT solver. We empirically demonstrate that this can often produce smaller backdoors than
previous approaches to computing LS backdoors
Design Space Re-Engineering for Power Minimization in Modern Embedded Systems
Power minimization is a critical challenge for modern embedded system design. Recently, due to the rapid increase of system's complexity and
the power density, there is a growing need for power control techniques at various design levels. Meanwhile, due to technology scaling, leakage power has become a significant part of power dissipation in the CMOS circuits and new techniques are needed to reduce leakage power.
As a result, many new power minimization techniques have been proposed such as voltage island, gate sizing, multiple supply and threshold voltage, power gating and input vector control, etc. These design options further
enlarge the design space and make it prohibitively expensive to explore
for the most energy efficient design solution.
Consequently, heuristic algorithms and randomized
algorithms are frequently used to explore the design space, seeking sub-optimal
solutions to meet the time-to-market requirements. These algorithms are
based on the idea of truncating the design space
and restricting the search in a subset of the original design space. While this approach can effectively reduce the runtime of searching, it may also exclude high-quality design solutions and cause design quality degradation.
When the solution to one problem is used as the base for another problem, such solution quality degradation will accumulate. In modern electronics system design, when several such algorithms are used in series to solve problems in different design levels, the final solution can be far off the optimal one.
In my Ph.D. work, I develop a {\em re-engineering} methodology to
facilitate exploring the design space of power efficient embedded systems design.
The direct goal is to enhance the performance of existing low power
techniques. The methodology is based on the idea that design quality can be improved
via iterative ``re-shaping'' the design space based on the ``bad'' structure
in the obtained design solutions; the searching run-time can be reduced
by the guidance from previous exploration. This approach can be
described in three phases: (1) apply the existing techniques to obtain
a sub-optimal solution; (2) analyze the solution and expand the design
space accordingly; and (3) re-apply the technique to re-explore the
enlarged design space.
We apply this methodology at different levels of embedded system design to
minimize power: (i) switching power reduction in sequential logic synthesis;
(ii) gate-level static leakage current reduction; (iii) dual threshold voltage CMOS
circuits design; and (iv) system-level energy-efficient detection scheme for
wireless sensor networks. An extensive amount of experiments have been conducted
and the results have shown that this methodology can effectively enhance
the power efficiency of the existing embedded system design flows with very little
overhead
Exploiting machine learning for combinatorial problem solving and optimisation
This dissertation presents a number of contributions to the field of solver portfolios, in particular for combinatorial search problems. We propose a novel hierarchical portfolio which does not rely on a single problem representation, but may transform the problem to an alternate representation using a portfolio of encodings, additionally a portfolio of solvers is employed for each of the representations. We extend this multi-representation portfolio for discrete optimisation tasks in the graphical models domain, realising a portfolio which won the UAI 2014 Inference Competition. We identify a fundamental flaw in empirical evaluations of many portfolio and runtime prediction methods. The fact that solvers exhibit a runtime distribution has not been considered in the setting of runtime prediction, solver portfolios, or automated configuration systems, to date these methods have taken a single sample as ground-truth. We demonstrated through a large empirical analysis that the outcome of empirical competitions can vary and provide statistical bounds on such variations. Finally, we consider an elastic solver which capitalises on the runtime distribution of a solver by launching searches in parallel, potentially on thousands of machines. We analyse the impact of the number of cores on not only solution time but also on energy consumption, the challenge being to find a optimal balance between the two. We highlight that although solution time always drops as the number of machines increases, the relation between the number of machines and energy consumption is more complicated. We also develop a prediction model, demonstrating that such insights can be exploited to achieve faster solutions times in a more energy efficient manner
Recommended from our members
Systematic techniques for more effective fault localization and program repair
Debugging faulty code is a tedious process that is often quite expensive and can require much manual effort. Developers typically perform debugging in two key steps: (1) fault localization, i.e., identifying the location of faulty line(s) of code; and (2) program repair, i.e., modifying the code to remove the fault(s). Automating debugging to reduce its cost has been the focus of a number of research projects during the last decade, which have introduced a variety of techniques.
However, existing techniques suffer from two basic limitations. One, they lack accuracy to handle real programs. Two, they focus on automating only one of the two key steps, thereby leaving the other key step to the developer.
Our thesis is that an approach that integrates systematic search based on state-of-the-art constraint solvers with techniques to analyze artifacts that describe application specific properties and behaviors, provides the basis for developing more effective debugging techniques. We focus on faults in programs that operate on structurally complex inputs, such as heap-allocated data or relational databases.
Our approach lays the foundation for a unified framework for localization and repair of faults in programs. We embody our thesis in a suite of integrated techniques based on propositional satisfiability solving, correctness specifications analysis, test-spectra analysis, and rule-learning algorithms from machine learning, implement them as a prototype tool-set, and evaluate them using several subject programs.Electrical and Computer Engineerin
Proceedings of the 22nd Conference on Formal Methods in Computer-Aided Design – FMCAD 2022
The Conference on Formal Methods in Computer-Aided Design (FMCAD) is an annual conference on the theory and applications of formal methods in hardware and system verification. FMCAD provides a leading forum to researchers in academia and industry for presenting and discussing groundbreaking methods, technologies, theoretical results, and tools for reasoning formally about computing systems. FMCAD covers formal aspects of computer-aided system design including verification, specification, synthesis, and testing
Recommended from our members
Formal Analysis of Arithmetic Circuits using Computer Algebra - Verification, Abstraction and Reverse Engineering
Despite a considerable progress in verification and abstraction of random and control logic, advances in formal verification of arithmetic designs have been lagging. This can be attributed mostly to the difficulty in an efficient modeling of arithmetic circuits and datapaths without resorting to computationally expensive Boolean methods, such as Binary Decision Diagrams (BDDs) and Boolean Satisfiability (SAT), that require “bit blasting”, i.e., flattening the design to a bit-level netlist. Approaches that rely on computer algebra and Satisfiability Modulo Theories (SMT) methods are either too abstract to handle the bit-level nature of arithmetic designs or require solving computationally expensive decision or satisfiability problems. The work proposed in this thesis aims at overcoming the limitations of analyzing arithmetic circuits, specifically at the post-synthesized phase. It addresses the verification, abstraction and reverse engineering problems of arithmetic circuits at an algebraic level, treating an arithmetic circuit and its specification as a properly constructed algebraic system. The proposed technique solves these problems by function extraction, i.e., by deriving arithmetic function computed by the circuit from its low-level circuit implementation using computer algebraic rewriting technique. The proposed techniques work on large integer arithmetic circuits and finite field arithmetic circuits, up to 512-bit wide containing millions of logic gates
Proceedings of the 22nd Conference on Formal Methods in Computer-Aided Design – FMCAD 2022
The Conference on Formal Methods in Computer-Aided Design (FMCAD) is an annual conference on the theory and applications of formal methods in hardware and system verification. FMCAD provides a leading forum to researchers in academia and industry for presenting and discussing groundbreaking methods, technologies, theoretical results, and tools for reasoning formally about computing systems. FMCAD covers formal aspects of computer-aided system design including verification, specification, synthesis, and testing