Recent developments in surrogate construction predominantly focused on two
strategies to improve surrogate accuracy. Firstly, component-wise domain
scaling informed by cross-validation. Secondly, regression to construct
response surfaces using additional information in the form of additional
function-values sampled from multi-fidelity models and gradients.
Component-wise domain scaling reliably improves the surrogate quality at low
dimensions but has been shown to suffer from high computational costs for
higher dimensional problems. The second strategy, adding gradients to train
surrogates, typically results in regression surrogates. Counter-intuitively,
these gradient-enhanced regression-based surrogates do not exhibit improved
accuracy compared to surrogates only interpolating function values.
This study empirically establishes three main findings. Firstly, constructing
the surrogate in poorly scaled domains is the predominant cause of
deteriorating response surfaces when regressing with additional gradient
information. Secondly, surrogate accuracy improves if the surrogates are
constructed in a fully transformed domain, by scaling and rotating the original
domain, not just simply scaling the domain. The domain transformation scheme
should be based on the local curvature of the approximation surface and not its
global curvature. Thirdly, the main benefit of gradient information is to
efficiently determine the (near) optimal domain in which to construct the
surrogate. This study proposes a foundational transformation algorithm that
performs near-optimal transformations for lower dimensional problems. The
algorithm consistently outperforms cross-validation-based component-wise domain
scaling for higher dimensional problems. A carefully selected test problem set
that varies between 2 and 16-dimensional problems is used to clearly
demonstrate the three main findings of this study.Comment: 20 pages, 28 figure