We present an experimental study of the effects of quantifier alternations on
the evaluation of quantified Boolean formula (QBF) solvers. The number of
quantifier alternations in a QBF in prenex conjunctive normal form (PCNF) is
directly related to the theoretical hardness of the respective QBF
satisfiability problem in the polynomial hierarchy. We show empirically that
the performance of solvers based on different solving paradigms substantially
varies depending on the numbers of alternations in PCNFs. In related
theoretical work, quantifier alternations have become the focus of
understanding the strengths and weaknesses of various QBF proof systems
implemented in solvers. Our results motivate the development of methods to
evaluate orthogonal solving paradigms by taking quantifier alternations into
account. This is necessary to showcase the broad range of existing QBF solving
paradigms for practical QBF applications. Moreover, we highlight the potential
of combining different approaches and QBF proof systems in solvers.Comment: preprint of a paper to be published at CP 2018, LNCS, Springer,
including appendi