4 research outputs found
PolyARBerNN: A Neural Network Guided Solver and Optimizer for Bounded Polynomial Inequalities
Constraints solvers play a significant role in the analysis, synthesis, and
formal verification of complex embedded and cyber-physical systems. In this
paper, we study the problem of designing a scalable constraints solver for an
important class of constraints named polynomial constraint inequalities (also
known as non-linear real arithmetic theory). In this paper, we introduce a
solver named PolyARBerNN that uses convex polynomials as abstractions for
highly nonlinear polynomials. Such abstractions were previously shown to be
powerful to prune the search space and restrict the usage of sound and complete
solvers to small search spaces. Compared with the previous efforts on using
convex abstractions, PolyARBerNN provides three main contributions namely (i) a
neural network guided abstraction refinement procedure that helps selecting the
right abstraction out of a set of pre-defined abstractions, (ii) a Bernstein
polynomial-based search space pruning mechanism that can be used to compute
tight estimates of the polynomial maximum and minimum values which can be used
as an additional abstraction of the polynomials, and (iii) an optimizer that
transforms polynomial objective functions into polynomial constraints (on the
gradient of the objective function) whose solutions are guaranteed to be close
to the global optima. These enhancements together allowed the PolyARBerNN
solver to solve complex instances and scales more favorably compared to the
state-of-art non-linear real arithmetic solvers while maintaining the soundness
and completeness of the resulting solver. In particular, our test benches show
that PolyARBerNN achieved 100X speedup compared with Z3 8.9, Yices 2.6, and
NASALib (a solver that uses Bernstein expansion to solve multivariate
polynomial constraints) on a variety of standard test benches
Learning-Based Communication Systems
Connecting people offers opportunities to build communities of any size and consequently, brings the world closer together. Conventionally, the connectivity has happened through traditional radio-frequency communication methods. However, the ever-increasing demand for higher data-rate communications and the explosion of advanced wireless applications such as virtual reality, augmented reality, and internet of things, reduce the effectiveness of these method. Therefore, developing next-generation technologies, such as learning-based communication systems, that can satisfy the large data and ultra-high rate communication requirements would be of interest. To address the challenging problem of connectivity, our research focuses on developing a learning-based framework for the next-generation communication systems. These systems can proactively adapt their communication and networking strategies to the dynamic of the environment, thereby maximizing their end-to-end performance in terms of data-rates, energy-efficiency, and link-reliability. Toward this goal, first information-theoretical tools are used to establish the fundamental limits (including bounds on the end-to-end performance). These performance limits are the keys for building reliable and efficient systems. Then, powerful machine learning techniques, such as deep learning, are employed for the implementation of such systems. In particular, a simple and cost-effective system with near-optimal performance can be implemented by merely taking off-the-shelf deep learning models, applying them to communication design problems, and tuning them based on the training data.masters, M.S., Electrical and Computer Engineering -- University of Idaho - College of Graduate Studies, 2019-0
Recommended from our members
Formal Verification of AI-Controlled Cyber-Physical Systems Using Polynomial Approximations: Constraints Solver, Model Checkers, and Applications
The last decade's advancement in machine learning (ML) for controlling cyber-physical systems has heralded a new era in autonomous technology, driving innovations from self-driving cars to smart infrastructure. However, these systems often grapple with challenges related to safety, reliability, and the ability to generalize across different scenarios. This dissertation aims to bridge the gap between the scalability and flexibility of ML-based control systems and the rigorous safety and reliability guarantees provided by formal methods and control theory. It introduces novel methodologies that leverage machine learning to enhance the design, verification, and optimization of AI-controlled cyber-physical systems, ensuring they meet high-level specifications while managing such systems' inherent complexity and non-linearity. The contributions of this thesis are multi-fold. 1) We proposed a highly efficient and parallelizable solver called PolyAR, which aims to solve general multivariate polynomial inequality constraints. PolyAR uses convex polynomials as an abstraction for highly nonlinear polynomials. Such abstractions were previously shown to be powerful to prune the search space and restrict the usage of sound and complete solvers to small search space. We compared the scalability of PolyAR against state-of-the-art solvers such as Z3 8.9 and Yices 2.6 on complex design and verification problems. The experiment results show that the PolyAR solver drastically outperformed Z3 8.9 and Yices 2.6 regarding execution time. 2) We developed PolyARBerNN, an enhancement to PolyAR that employs neural networks (NN) to guide the abstraction refinement procedure that helps to select the right abstraction out of a set of pre-defined abstractions and a Bernstein polynomial-based search space pruning mechanism. These enhancements together made PolyARBerNN capable of solving complex instances and scaling more favorably compared to the state-of-the-art nonlinear real arithmetic solvers while maintaining the soundness and completeness of the resulting solver. In addition, we proposed an efficient optimizer called PolyAROpt that transforms polynomial objective functions into polynomial constraints (on the gradient of the objective function) whose solutions are guaranteed to be close to the global optima. PolyAROpt optimizer uses PolyARBerNN to solve constrained polynomial optimization problems. Numerical results show that PolyAROpt can solve high-dimensional and high-order polynomial optimization problems faster than the built-in optimizer in the Z3 8.9 solver. 3) We proposed an efficient algorithm called BERN-NN that employs polynomial interval arithmetic, where tight over/under approximations of the NN's activation functions are computed using Bernstein polynomials. These polynomials have several interesting mathematical proprieties. One particular property is called the sharpness propriety, which allows us to obtain extremely tight bounds that are tighter than those currently exist in the literature (e.g., interval arithmetic, crowns, linear programming, and many centered forms). Moreover, we exploited the mathematical proprieties of Bernstein polynomials to convert the proposed polynomial interval arithmetic operations into add-and-multiply operations, which are easily implemented using GPUs. Thanks to those GPUs, our tool's execution time is drastically reduced. Experimental results show that our method approximates NN's outputs tighter than state-of-the-art NN verification tools by several orders of magnitude. 4) We proposed BERN-NN-IBF, a significant enhancement of the Bernstein-polynomial-based bound propagation algorithms. BERN-NN-IBF offers three main contributions: (i) a memory-efficient encoding of Bernstein-polynomials to scale the bound propagation algorithms, (ii) optimized tensor operations for the new polynomial encoding to maintain the integrity of the bounds while enhancing computational efficiency, and (iii) tighter under-approximations of the ReLU activation function using quadratic polynomials tailored to minimize approximation errors. Through comprehensive testing, we demonstrate that BERN-NN-IBF achieves tighter bounds and higher computational efficiency than the original BERN-NN and state-of-the-art methods, including linear programming and convex used within the winner of the VNN-COMPETITION