Honey bee workers have few opportunities for direct reproduction because their ovary development is chemically suppressed by queens and worker-laid eggs are destroyed by workers. While workers with fully developed ovaries are rare in honey bee colonies, we show that partial ovary development is common. Across nine studies, an average of 6% to 43% of workers had partially developed ovaries in queenright colonies with naturally mated queens. This shift by workers toward potential future reproduction is linked to lower productivity, which suggests that even small investments in reproductive physiology by selfish workers reduce cooperation below a theoretical maximum. Furthermore, comparisons across 26 species of bees and wasps revealed that the level of partial ovary development in honey bees is similar to that of other eusocial Hymenoptera where there is reproductive conflict among colony members. Natural variation in the extent of partial ovary development in honey bee colonies calls for an exploration of the genetic and ecological factors that modulate shifts in cooperation within animal societies