Performance in everyday spatial orientation tasks (e.g. map reading and navigation) has been considered functionally separate from performance on more abstract object-based spatial abilities (e.g. mental rotation and visualization). However, evidence remains scarce and unsystematic. With a novel gamified battery, we assessed six tests of spatial orientation in a virtual environment and examined their association with ten object-based spatial tests, as well as their links to general cognitive ability (g). We further estimated the role of genetic and environmental factors in underlying variation and covariation in these spatial tests. Participants (N = 2,660) were part of the Twins Early Development Study, aged 19 to 22. The 6 tests of spatial orientation clustered into a single ‘Navigation’ factor that was 64% heritable. Examining the structure of spatial ability across all 16 tests, three factors emerged: Navigation, Object Manipulation and Visualization. These, in turn, loaded strongly onto a general factor of Spatial Ability, which was highly heritable (84%). A large portion (45%) of this high heritability was independent of g. The results from this most comprehensive investigation of spatial abilities to date point towards the existence of a common genetic network that supports all spatial abilities