Altruism between close relatives can be easily explained. However, paradoxes arise when organisms divert altruism towards more distantly related recipients. In some social insects, workers drift extensively between colonies and help raise less related foreign brood, seemingly reducing inclusive fitness. Since being highlighted by W. D. Hamilton, three hypotheses (bet hedging, indirect reciprocity and diminishing returns to cooperation) have been proposed for this surprising behaviour. Here, using inclusive fitness theory, we show that bet hedging and indirect reciprocity could only drive cooperative drifting under improbable conditions. However, diminishing returns to cooperation create a simple context in which sharing workers is adaptive. Using a longitudinal dataset comprising over a quarter of a million nest cell observations, we quantify cooperative payoffs in the Neotropical wasp Polistes canadensis, for which drifting occurs at high levels. As the worker-to-brood ratio rises in a workerβs home colony, the predicted marginal benefit of a worker for expected colony productivity diminishes. Helping related colonies can allow effort to be focused on related brood that are more in need of care. Finally, we use simulations to show that cooperative drifting evolves under diminishing returns when dispersal is local, allowing altruists to focus their efforts on related recipients. Our results indicate the power of nonlinear fitness effects to shape social organization, and suggest that models of eusocial evolution should be extended to include neglected social interactions within colony networks