Total mean curvature, scalar curvature, and a variational analog of Brown-York mass

Abstract

We study the supremum of the total mean curvature on the boundary of compact, mean-convex 3-manifolds with nonnegative scalar curvature, and a prescribed boundary metric. We establish an additivity property for this supremum and exhibit rigidity for maximizers assuming the supremum is attained. When the boundary consists of 2-spheres, we demonstrate that the finiteness of the supremum follows from the previous work of Shi-Tam and Wang-Yau on the quasi-local mass problem in general relativity. In turn, we define a variational analog of Brown-York quasi-local mass without assuming that the boundary 2-sphere has positive Gauss curvature.Comment: Final version; incorporates changes made for journa