Experiences with eliciting probalitities from multiple experts, 14th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems

Bayesian networks are typically designed in collaboration with a single domain expert from a single institute. Since a network is often intended for wider use, its engineering involves verifying whether it appropriately reflects expert knowledge from other institutes. Upon engineering a network intended for use across Europe, we compared the original probability assessments obtained from our Dutch expert with assessments from 38 experts in six countries. While we found large variances among the assessments per probability, very high consistency was found for the qualitative properties embedded in the series of assessments per assessor. The apparent robustness of these properties suggests the importance of enforcing them in a Bayesian network under construction