During the past five million yrs, benthic d18O records indicate a large range of climates, from warmer than today during the Pliocene Warm Period to considerably colder during glacials. Antarctic ice cores have revealed Pleistocene glacial-interglacial CO2 variability of 60-100 ppm, while sea level fluctuations of typically 125 m are documented by proxy data. However, in the pre-ice core period, CO2 and sea level proxy data are scarce and there is disagreement between different proxies and different records of the same proxy. This hampers comprehensive understanding of the long-term relations between CO2, sea level and climate. Here, we drive a coupled climate-ice sheet model over the past five million years, inversely forced by a stacked benthic d18O record. We obtain continuous simulations of benthic d18O, sea level and CO2 that are mutually consistent. Our model shows CO2 concentrations of 300 to 470 ppm during the Early Pliocene. Furthermore, we simulate strong CO2 variability during the Pliocene and Early Pleistocene. These features are broadly supported by existing and new d11B-based proxy CO2 data, but less by alkenone-based records. The simulated concentrations and variations therein are larger than expected from global mean temperature changes. Our findings thus suggest a smaller Earth System Sensitivity than previously thought. This is explained by a more restricted role of land ice variability in the Pliocene. The largest uncertainty in our simulation arises from the mass balance formulation of East Antarctica, which governs the variability in sea level, but only modestly affects the modeled CO2 concentrations