924 research outputs found
DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization
Neural network-based Combinatorial Optimization (CO) methods have shown
promising results in solving various NP-complete (NPC) problems without relying
on hand-crafted domain knowledge. This paper broadens the current scope of
neural solvers for NPC problems by introducing a new graph-based diffusion
framework, namely DIFUSCO. Our framework casts NPC problems as discrete {0,
1}-vector optimization problems and leverages graph-based denoising diffusion
models to generate high-quality solutions. We investigate two types of
diffusion models with Gaussian and Bernoulli noise, respectively, and devise an
effective inference schedule to enhance the solution quality. We evaluate our
methods on two well-studied NPC combinatorial optimization problems: Traveling
Salesman Problem (TSP) and Maximal Independent Set (MIS). Experimental results
show that DIFUSCO strongly outperforms the previous state-of-the-art neural
solvers, improving the performance gap between ground-truth and neural solvers
from 1.76% to 0.46% on TSP-500, from 2.46% to 1.17% on TSP-1000, and from 3.19%
to 2.58% on TSP10000. For the MIS problem, DIFUSCO outperforms the previous
state-of-the-art neural solver on the challenging SATLIB benchmark. Our code is
available at "https://github.com/Edward-Sun/DIFUSCO"
A Neural PDE Solver with Temporal Stencil Modeling
Numerical simulation of non-linear partial differential equations plays a
crucial role in modeling physical science and engineering phenomena, such as
weather, climate, and aerodynamics. Recent Machine Learning (ML) models trained
on low-resolution spatio-temporal signals have shown new promises in capturing
important dynamics in high-resolution signals, under the condition that the
models can effectively recover the missing details. However, this study shows
that significant information is often lost in the low-resolution down-sampled
features. To address such issues, we propose a new approach, namely Temporal
Stencil Modeling (TSM), which combines the strengths of advanced time-series
sequence modeling (with the HiPPO features) and state-of-the-art neural PDE
solvers (with learnable stencil modeling). TSM aims to recover the lost
information from the PDE trajectories and can be regarded as a temporal
generalization of classic finite volume methods such as WENO. Our experimental
results show that TSM achieves the new state-of-the-art simulation accuracy for
2-D incompressible Navier-Stokes turbulent flows: it significantly outperforms
the previously reported best results by 19.9% in terms of the highly-correlated
duration time and reduces the inference latency into 80%. We also show a strong
generalization ability of the proposed method to various out-of-distribution
turbulent flow settings. Our code is available at
"https://github.com/Edward-Sun/TSM-PDE"
Accelerating Diffusion-based Combinatorial Optimization Solvers by Progressive Distillation
Graph-based diffusion models have shown promising results in terms of
generating high-quality solutions to NP-complete (NPC) combinatorial
optimization (CO) problems. However, those models are often inefficient in
inference, due to the iterative evaluation nature of the denoising diffusion
process. This paper proposes to use progressive distillation to speed up the
inference by taking fewer steps (e.g., forecasting two steps ahead within a
single step) during the denoising process. Our experimental results show that
the progressively distilled model can perform inference 16 times faster with
only 0.019% degradation in performance on the TSP-50 dataset
- …