The classical interpretation of probability together with the Principle of Indifference are formulated in terms of probability measure spaces in which the probability is given by the Haar measure. A notion called Labeling Invariance is defined in the category of Haar probability spaces, it is shown that Labeling Invariance is violated and Bertrand's Paradox is interpreted as the very proof of violation of Labeling Invariance. It is shown that Bangu's attempt (Bangu 2010) to block the emergence of Bertrand's Paradox by requiring the re-labeling of random events to preserve randomness cannot succeed non-trivially. A non-trivial strategy to preserve Labeling Invariance is identified and it is argued that, under the interpretation of Bertrand's Paradox suggested in the paper, the paradox does not undermine either the Principle of Indifference or the classical interpretation and is in complete harmony with how mathematical probability theory is used in the sciences to model phenomena; it is shown in particular that violation of Labeling Invariance does not entail that labeling of random events affects the probabilities of random events.
It also is argued however that the content of the Principle of Indifference cannot be specified in such a way that it can establish the classical interpretation of probability as descriptively accurate or predictively successful
