Magnetic particle imaging (MPI) data is commonly reconstructed using a system
matrix acquired in a time-consuming calibration measurement. The calibration
approach has the important advantage over model-based reconstruction that it
takes the complex particle physics as well as system imperfections into
account. This benefit comes for the cost that the system matrix needs to be
re-calibrated whenever the scan parameters, particle types or even the particle
environment (e.g. viscosity or temperature) changes. One route for reducing the
calibration time is the sampling of the system matrix at a subset of the
spatial positions of the intended field-of-view and employing system matrix
recovery. Recent approaches used compressed sensing (CS) and achieved
subsampling factors up to 28 that still allowed reconstructing MPI images of
sufficient quality. In this work, we propose a novel framework with a 3d-System
Matrix Recovery Network and demonstrate it to recover a 3d system matrix with a
subsampling factor of 64 in less than one minute and to outperform CS in terms
of system matrix quality, reconstructed image quality, and processing time. The
advantage of our method is demonstrated by reconstructing open access MPI
datasets. The model is further shown to be capable of inferring system matrices
for different particle types