Low-depth parity check (LDPC) codes are a paradigm of error correction that
allow for spatially non-local interactions between (qu)bits, while still
enforcing that each (qu)bit interacts only with finitely many others. On
expander graphs, they can give rise to ``good codes'' that combine a finite
encoding rate with an optimal scaling of the code distance, which governs the
code's robustness against noise. Such codes have garnered much recent attention
due to two breakthrough developments: the construction of good quantum LDPC
codes and good locally testable classical LDPC codes, using similar methods.
Here we explore these developments from a physics lens, establishing
connections between LDPC codes and ordered phases of matter defined for systems
with non-local interactions and on non-Euclidean geometries. We generalize the
physical notions of Kramers-Wannier (KW) dualities and gauge theories to this
context, using the notion of chain complexes as an organizing principle. We
discuss gauge theories based on generic classical LDPC codes and make a
distinction between two classes, based on whether their excitations are
point-like or extended. For the former, we describe KW dualities, analogous to
the 1D Ising model and describe the role played by ``boundary conditions''. For
the latter we generalize Wegner's duality to obtain generic quantum LDPC codes
within the deconfined phase of a Z_2 gauge theory. We show that all known
examples of good quantum LDPC codes are obtained by gauging locally testable
classical codes. We also construct cluster Hamiltonians from arbitrary
classical codes, related to the Higgs phase of the gauge theory, and formulate
generalizations of the Kennedy-Tasaki duality transformation. We use the chain
complex language to discuss edge modes and non-local order parameters for these
models, initiating the study of SPT phases in non-Euclidean geometries