While in atmosphere models it is already common to define objective metrics to investigate
how well an atmospheric model performs compared to observations, this is not too common
for ocean models. Here we define a simple metrics encompassing the 3D structure of bias and
absolute error to estimate the performance of ocean models and we apply it to the historical
CMIP5 simulations from 1950 to 2005. Ocean model 3D temperature and salinity fields are
compared to the PHC climatology for the major ocean basins. For each 3D grid point of the
PHC dataset bias and absolute error of the model climatology are calculated and then volume-
averaged over each ocean basin. An average CMIP5 model error is calculated for each ocean
basin and used as a reference when investigating a particular model - similarly as has been
done for the atmosphere by Reichler and Kim (2008) for CMIP3 models.
Ocean surface temperature is generally reasonably well simulated by CMIP5 models and mean
absolute errors amount to around 1 K which is comparable to the interannual variability. But
in 500 to 1000 m - depending on the ocean basin and on the model - mean absolute errors
of up to 4 K are detected which clearly exceed the interannual variability of generally below 1
K. For salinity mean absolute errors are in all levels clearly higher than the interannual
variability. For example at the surface the mean absolute error amounts to up to 1 psu while
the interannual variability is below 0.2 psu. Even if investigating biases which allows for
cancelling out of errors within a basin instead of the mean absolute error this statement still
holds in many cases. This means that there is a lot of scope for improvement of the
simulation of the vertical structure of the ocean
