711 research outputs found
Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability
Post-hoc model-agnostic interpretation methods such as partial dependence
plots can be employed to interpret complex machine learning models. While these
interpretation methods can be applied regardless of model complexity, they can
produce misleading and verbose results if the model is too complex, especially
w.r.t. feature interactions. To quantify the complexity of arbitrary machine
learning models, we propose model-agnostic complexity measures based on
functional decomposition: number of features used, interaction strength and
main effect complexity. We show that post-hoc interpretation of models that
minimize the three measures is more reliable and compact. Furthermore, we
demonstrate the application of these measures in a multi-objective optimization
approach which simultaneously minimizes loss and complexity
Quantifying the Preferential Direction of the Model Gradient in Adversarial Training With Projected Gradient Descent
Adversarial training, especially projected gradient descent (PGD), has been
the most successful approach for improving robustness against adversarial
attacks. After adversarial training, gradients of models with respect to their
inputs have a preferential direction. However, the direction of alignment is
not mathematically well established, making it difficult to evaluate
quantitatively. We propose a novel definition of this direction as the
direction of the vector pointing toward the closest point of the support of the
closest inaccurate class in decision space. To evaluate the alignment with this
direction after adversarial training, we apply a metric that uses generative
adversarial networks to produce the smallest residual needed to change the
class present in the image. We show that PGD-trained models have a higher
alignment than the baseline according to our definition, that our metric
presents higher alignment values than a competing metric formulation, and that
enforcing this alignment increases the robustness of models.Comment: Updates for second version: added methods/analysis for multiclass
datasets; added new references found since last submission; removed claims
about interpretability; overall editin
A Mass-Conserving-Perceptron for Machine Learning-Based Modeling of Geoscientific Systems
Although decades of effort have been devoted to building Physical-Conceptual
(PC) models for predicting the time-series evolution of geoscientific systems,
recent work shows that Machine Learning (ML) based Gated Recurrent Neural
Network technology can be used to develop models that are much more accurate.
However, the difficulty of extracting physical understanding from ML-based
models complicates their utility for enhancing scientific knowledge regarding
system structure and function. Here, we propose a physically-interpretable Mass
Conserving Perceptron (MCP) as a way to bridge the gap between PC-based and
ML-based modeling approaches. The MCP exploits the inherent isomorphism between
the directed graph structures underlying both PC models and GRNNs to explicitly
represent the mass-conserving nature of physical processes while enabling the
functional nature of such processes to be directly learned (in an interpretable
manner) from available data using off-the-shelf ML technology. As a proof of
concept, we investigate the functional expressivity (capacity) of the MCP,
explore its ability to parsimoniously represent the rainfall-runoff (RR)
dynamics of the Leaf River Basin, and demonstrate its utility for scientific
hypothesis testing. To conclude, we discuss extensions of the concept to enable
ML-based physical-conceptual representation of the coupled nature of
mass-energy-information flows through geoscientific systems.Comment: 60 pages and 7 figures in the main text. 10 figures, and 10 tables in
the supplementary material
- …