137 research outputs found
Robust Decision Trees Against Adversarial Examples
Although adversarial examples and model robustness have been extensively
studied in the context of linear models and neural networks, research on this
issue in tree-based models and how to make tree-based models robust against
adversarial examples is still limited. In this paper, we show that tree based
models are also vulnerable to adversarial examples and develop a novel
algorithm to learn robust trees. At its core, our method aims to optimize the
performance under the worst-case perturbation of input features, which leads to
a max-min saddle point problem. Incorporating this saddle point objective into
the decision tree building procedure is non-trivial due to the discrete nature
of trees --- a naive approach to finding the best split according to this
saddle point objective will take exponential time. To make our approach
practical and scalable, we propose efficient tree building algorithms by
approximating the inner minimizer in this saddle point problem, and present
efficient implementations for classical information gain based trees as well as
state-of-the-art tree boosting models such as XGBoost. Experimental results on
real world datasets demonstrate that the proposed algorithms can substantially
improve the robustness of tree-based models against adversarial examples
A Monte Carlo packing algorithm for poly-ellipsoids and its comparison with packing generation using Discrete Element Model
Granular material is showing very often in geotechnical engineering,
petroleum engineering, material science and physics. The packings of the
granular material play a very important role in their mechanical behaviors,
such as stress-strain response, stability, permeability and so on. Although
packing is such an important research topic that its generation has been
attracted lots of attentions for a long time in theoretical, experimental, and
numerical aspects, packing of granular material is still a difficult and active
research topic, especially the generation of random packing of non-spherical
particles. To this end, we will generate packings of same particles with same
shapes, numbers, and same size distribution using geometry method and dynamic
method, separately. Specifically, we will extend one of Monte Carlo models for
spheres to ellipsoids and poly-ellipsoids
Bayesian Over-the-Air FedAvg via Channel Driven Stochastic Gradient Langevin Dynamics
The recent development of scalable Bayesian inference methods has renewed
interest in the adoption of Bayesian learning as an alternative to conventional
frequentist learning that offers improved model calibration via uncertainty
quantification. Recently, federated averaging Langevin dynamics (FALD) was
introduced as a variant of federated averaging that can efficiently implement
distributed Bayesian learning in the presence of noiseless communications. In
this paper, we propose wireless FALD (WFALD), a novel protocol that realizes
FALD in wireless systems by integrating over-the-air computation and
channel-driven sampling for Monte Carlo updates. Unlike prior work on wireless
Bayesian learning, WFALD enables (\emph{i}) multiple local updates between
communication rounds; and (\emph{ii}) stochastic gradients computed by
mini-batch. A convergence analysis is presented in terms of the 2-Wasserstein
distance between the samples produced by WFALD and the targeted global
posterior distribution. Analysis and experiments show that, when the
signal-to-noise ratio is sufficiently large, channel noise can be fully
repurposed for Monte Carlo sampling, thus entailing no loss in performance.Comment: 6 pages, 4 figures, 26 references, submitte
Rare Event Probability Learning by Normalizing Flows
A rare event is defined by a low probability of occurrence. Accurate
estimation of such small probabilities is of utmost importance across diverse
domains. Conventional Monte Carlo methods are inefficient, demanding an
exorbitant number of samples to achieve reliable estimates. Inspired by the
exact sampling capabilities of normalizing flows, we revisit this challenge and
propose normalizing flow assisted importance sampling, termed NOFIS. NOFIS
first learns a sequence of proposal distributions associated with predefined
nested subset events by minimizing KL divergence losses. Next, it estimates the
rare event probability by utilizing importance sampling in conjunction with the
last proposal. The efficacy of our NOFIS method is substantiated through
comprehensive qualitative visualizations, affirming the optimality of the
learned proposal distribution, as well as a series of quantitative experiments
encompassing distinct test cases, which highlight NOFIS's superiority over
baseline approaches.Comment: 16 pages, 5 figures, 2 table
Towards Fast Computation of Certified Robustness for ReLU Networks
Verifying the robustness property of a general Rectified Linear Unit (ReLU)
network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer
CAV17]. Although finding the exact minimum adversarial distortion is hard,
giving a certified lower bound of the minimum distortion is possible. Current
available methods of computing such a bound are either time-consuming or
delivering low quality bounds that are too loose to be useful. In this paper,
we exploit the special structure of ReLU networks and provide two
computationally efficient algorithms Fast-Lin and Fast-Lip that are able to
certify non-trivial lower bounds of minimum distortions, by bounding the ReLU
units with appropriate linear functions Fast-Lin, or by bounding the local
Lipschitz constant Fast-Lip. Experiments show that (1) our proposed methods
deliver bounds close to (the gap is 2-3X) exact minimum distortion found by
Reluplex in small MNIST networks while our algorithms are more than 10,000
times faster; (2) our methods deliver similar quality of bounds (the gap is
within 35% and usually around 10%; sometimes our bounds are even better) for
larger networks compared to the methods based on solving linear programming
problems but our algorithms are 33-14,000 times faster; (3) our method is
capable of solving large MNIST and CIFAR networks up to 7 layers with more than
10,000 neurons within tens of seconds on a single CPU core.
In addition, we show that, in fact, there is no polynomial time algorithm
that can approximately find the minimum adversarial distortion of a
ReLU network with a approximation ratio unless
=, where is the number of neurons in the network.Comment: Tsui-Wei Weng and Huan Zhang contributed equall
Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering
We present Tensor4D, an efficient yet effective approach to dynamic scene
modeling. The key of our solution is an efficient 4D tensor decomposition
method so that the dynamic scene can be directly represented as a 4D
spatio-temporal tensor. To tackle the accompanying memory issue, we decompose
the 4D tensor hierarchically by projecting it first into three time-aware
volumes and then nine compact feature planes. In this way, spatial information
over time can be simultaneously captured in a compact and memory-efficient
manner. When applying Tensor4D for dynamic scene reconstruction and rendering,
we further factorize the 4D fields to different scales in the sense that
structural motions and dynamic detailed changes can be learned from coarse to
fine. The effectiveness of our method is validated on both synthetic and
real-world scenes. Extensive experiments show that our method is able to
achieve high-quality dynamic reconstruction and rendering from sparse-view
camera rigs or even a monocular camera. The code and dataset will be released
at https://liuyebin.com/tensor4d/tensor4d.html
Estimated ultimate recovery prediction of fractured horizontal wells in tight oil reservoirs based on deep neural networks
Accurate estimated ultimate recovery prediction of fractured horizontal wells in tight reservoirs is crucial to economic evaluation and oil field development plan formulation. Advances in artificial intelligence and big data have provided a new tool for rapid production prediction of unconventional reservoirs. In this study, the estimated ultimate recovery prediction model based on deep neural networks was established using the data of 58 horizontal wells in Mahu tight oil reservoirs. First, the estimated ultimate recovery of oil wells was calculated based on the stretched exponential production decline model and a five-region flow model. Then, the calculated estimated ultimate recovery, geological attributes, engineering parameters, and production data of each well were used to build a machine learning database. Before the model training, the number of input parameters was reduced from 14 to 9 by feature selection. The prediction accuracy of the model was improved by data normalization, the early stopping technique, and 10-fold cross validation. The optimal activation function, hidden layers, number of neurons in each layer, and learning rate of the deep neural network model were obtained through hyperparameter optimization. The average determination coefficient on the testing set was 0.73. The results indicate that compared with the traditional estimated ultimate recovery prediction methods, the established deep neural network model has the strengths of a simple procedure and low time consumption, and the deep neural network model can be easily updated to improve prediction accuracy when new well information is obtained.Cited as: Luo, S., Ding, C., Cheng, H., Zhang, B., Zhao, Y., Liu, L. Estimated ultimate recovery prediction of fractured horizontal wells in tight oil reservoirs based on deep neural networks. Advances in Geo-Energy Research, 2022, 6(2): 111-122. https://doi.org/10.46690/ager.2022.02.0
Robustness Verification of Tree-based Models
We study the robustness verification problem for tree-based models, including
decision trees, random forests (RFs) and gradient boosted decision trees
(GBDTs). Formal robustness verification of decision tree ensembles involves
finding the exact minimal adversarial perturbation or a guaranteed lower bound
of it. Existing approaches find the minimal adversarial perturbation by a mixed
integer linear programming (MILP) problem, which takes exponential time so is
impractical for large ensembles. Although this verification problem is
NP-complete in general, we give a more precise complexity characterization. We
show that there is a simple linear time algorithm for verifying a single tree,
and for tree ensembles, the verification problem can be cast as a max-clique
problem on a multi-partite graph with bounded boxicity. For low dimensional
problems when boxicity can be viewed as constant, this reformulation leads to a
polynomial time algorithm. For general problems, by exploiting the boxicity of
the graph, we develop an efficient multi-level verification algorithm that can
give tight lower bounds on the robustness of decision tree ensembles, while
allowing iterative improvement and any-time termination. OnRF/GBDT models
trained on 10 datasets, our algorithm is hundreds of times faster than the
previous approach that requires solving MILPs, and is able to give tight
robustness verification bounds on large GBDTs with hundreds of deep trees.Comment: Hongge Chen and Huan Zhang contributed equall
- …