Ever since the first microscope by Zacharias Janssen in the late 16th
century, scientists have been inventing new types of microscopes for various
tasks. Inventing a novel architecture demands years, if not decades, worth of
scientific experience and creativity. In this work, we introduce Differentiable
Microscopy (∂μ), a deep learning-based design paradigm, to aid
scientists design new interpretable microscope architectures. Differentiable
microscopy first models a common physics-based optical system however with
trainable optical elements at key locations on the optical path. Using
pre-acquired data, we then train the model end-to-end for a task of interest.
The learnt design proposal can then be simplified by interpreting the learnt
optical elements. As a first demonstration, based on the optical 4-f system,
we present an all-optical quantitative phase microscope (QPM) design that
requires no computational post-reconstruction. A follow-up literature survey
suggested that the learnt architecture is similar to the generalized phase
contrast method developed two decades ago. Our extensive experiments on
multiple datasets that include biological samples show that our learnt
all-optical QPM designs consistently outperform existing methods. We
experimentally verify the functionality of the optical 4-f system based QPM
design using a spatial light modulator. Furthermore, we also demonstrate that
similar results can be achieved by an uninterpretable learning based method,
namely diffractive deep neural networks (D2NN). The proposed differentiable
microscopy framework supplements the creative process of designing new optical
systems and would perhaps lead to unconventional but better optical designs