A Transformer-based deep direct sampling method is proposed for a class of
boundary value inverse problems. A real-time reconstruction is achieved by
evaluating the learned inverse operator between carefully designed data and the
reconstructed images. An effort is made to give a specific example to a
fundamental question: whether and how one can benefit from the theoretical
structure of a mathematical problem to develop task-oriented and
structure-conforming deep neural networks? Specifically, inspired by direct
sampling methods for inverse problems, the 1D boundary data in different
frequencies are preprocessed by a partial differential equation-based feature
map to yield 2D harmonic extensions as different input channels. Then, by
introducing learnable non-local kernels, the direct sampling is recast to a
modified attention mechanism. The proposed method is then applied to electrical
impedance tomography, a well-known severely ill-posed nonlinear inverse
problem. The new method achieves superior accuracy over its predecessors and
contemporary operator learners, as well as shows robustness with respect to
noise. This research shall strengthen the insights that the attention
mechanism, despite being invented for natural language processing tasks, offers
great flexibility to be modified in conformity with the a priori mathematical
knowledge, which ultimately leads to the design of more physics-compatible
neural architectures