In this paper, we discuss the question whether a physical "simplification" of
a model makes it always easier to study, at least from a mathematical and
numerical point of view. To this end, we give different examples showing that
these simplifications often lead to worse mathematical properties of the
solution to the model. This may affect the existence and uniqueness of
solutions as well as their numerical approximability and other qualitative
properties. In the first part, we consider examples where the addition of a
higher-order term or stochastic noise leads to better mathematical results,
whereas in the second part, we focus on examples showing that also nonlocal
models can often be seen as physically more exact models as they have a close
connection to higher-order models