Parametrized Quantum Circuits (PQCs) enable a novel method for machine
learning (ML). However, from a computational point of view they present a
challenge to existing eXplainable AI (xAI) methods. On the one hand,
measurements on quantum circuits introduce probabilistic errors which impact
the convergence of these methods. On the other hand, the phase space of a
quantum circuit expands exponentially with the number of qubits, complicating
efforts to execute xAI methods in polynomial time. In this paper we will
discuss the performance of established xAI methods, such as Baseline SHAP and
Integrated Gradients. Using the internal mechanics of PQCs we study ways to
speed up their computation