Quantum systems have entered a competitive regime where classical computers
must make approximations to represent highly entangled quantum states. However,
in this beyond-classically-exact regime, fidelity comparisons between quantum
and classical systems have so far been limited to digital quantum devices, and
it remains unsolved how to estimate the actual entanglement content of
experiments. Here we perform fidelity benchmarking and mixed-state entanglement
estimation with a 60-atom analog Rydberg quantum simulator, reaching a high
entanglement entropy regime where exact classical simulation becomes
impractical. Our benchmarking protocol involves extrapolation from comparisons
against many approximate classical algorithms with varying entanglement limits.
We then develop and demonstrate an estimator of the experimental mixed-state
entanglement, finding our experiment is competitive with state-of-the-art
digital quantum devices performing random circuit evolution. Finally, we
compare the experimental fidelity against that achieved by various approximate
classical algorithms, and find that only one, which we introduce here, is able
to keep pace with the experiment on the classical hardware we employ. Our
results enable a new paradigm for evaluating the performance of both analog and
digital quantum devices in the beyond-classically-exact regime, and highlight
the evolving divide between quantum and classical systems.Comment: ALS, ZC, and JC contributed equall