During embryogenesis, the spherical inner cell mass (ICM) proliferates in the confined environment of a blastocyst. Embryonic stem cells (ESCs) are derived from the ICM, and mimicking embryogenesis <i>in vitro</i>, mouse ESCs (mESCs) are often cultured in hanging droplets. This promotes the formation of a spheroid as the cells sediment and aggregate owing to increased physical confinement and cell-cell interactions. In contrast, mESCs form two-dimensional monolayers on flat substrates and it remains unclear if the difference in organization is owing to a lack of physical confinement or increased cell-substrate versus cell-cell interactions. Employing microfabricated substrates, we demonstrate that a single geometric degree of physical confinement on a surface can also initiate spherogenesis. Experiment and computation reveal that a balance between cell-cell and cell-substrate interactions finely controls the morphology and organization of mESC aggregates. Physical confinement is thus an important regulatory cue in the three-dimensional organization and morphogenesis of developing cells