Physical and numerical limitations of stationary Vlasov-Poisson solvers based on backward Liouville methods are investigated with five solvers that combine different meshes, numerical integrators, and electric field interpolation schemes. Since some of the limitations arise when moving from an integrable to a non-integrable configuration, an elliptical Langmuir probe immersed in a Maxwellian plasma was considered and the eccentricity (ep) of its cross-section used as integrability-breaking parameter. In the cylindrical case, ep=0, the energy and angular momentum are both conserved. The trajectories of the charged particles are regular and the boundaries that separate trapped from non-trapped particles in phase space are smooth curves. However, their computation has to be done carefully because, albeit small, the intrinsic numerical errors of some solvers break these conservation laws. It is shown that an optimum exists for the number of loops around the probe that the solvers need to classify a particle trajectory as trapped. For ep≠0, the angular momentum is not conserved and particle dynamics in phase space is a mix of regular and chaotic orbits. The distribution function is filamented and the boundaries that separate trapped from non-trapped particles in phase space have a fractal geometry. The results were used to make a list of recommendations for the practical implementation of stationary Vlasov-Poisson solvers in a wide range of physical scenarios.This work was supported by the European Union's Horizon 2020 Research and Innovation Programme under grant agreement No 828902 (E.T.PACK project). GSA work is supported by the Ministerio de Ciencia, Innovación of Spain under the Grant RYC-2014-15357. The authors thank the Reviewers for their valuable comments and suggestions about the use of energy-conserving numerical integrators
