Configurable software systems are employed in many important application
domains. Understanding the performance of the systems under all configurations
is critical to prevent potential performance issues caused by misconfiguration.
However, as the number of configurations can be prohibitively large, it is not
possible to measure the system performance under all configurations. Thus, a
common approach is to build a prediction model from a limited measurement data
to predict the performance of all configurations as scalar values. However, it
has been pointed out that there are different sources of uncertainty coming
from the data collection or the modeling process, which can make the scalar
predictions not certainly accurate. To address this problem, we propose a
Bayesian deep learning based method, namely BDLPerf, that can incorporate
uncertainty into the prediction model. BDLPerf can provide both scalar
predictions for configurations' performance and the corresponding confidence
intervals of these scalar predictions. We also develop a novel uncertainty
calibration technique to ensure the reliability of the confidence intervals
generated by a Bayesian prediction model. Finally, we suggest an efficient
hyperparameter tuning technique so as to train the prediction model within a
reasonable amount of time whilst achieving high accuracy. Our experimental
results on 10 real-world systems show that BDLPerf achieves higher accuracy
than existing approaches, in both scalar performance prediction and confidence
interval estimation