1 research outputs found
Analytical Bounds between Entropy and Error Probability in Binary Classifications
The existing upper and lower bounds between entropy and error probability are
mostly derived from the inequality of the entropy relations, which could
introduce approximations into the analysis. We derive analytical bounds based
on the closed-form solutions of conditional entropy without involving any
approximation. Two basic types of classification errors are investigated in the
context of binary classification problems, namely, Bayesian and non-Bayesian
errors. We theoretically confirm that Fano's lower bound is an exact lower
bound for any types of classifier in a relation diagram of "error probability
vs. conditional entropy". The analytical upper bounds are achieved with respect
to the minimum prior probability, which are tighter than Kovalevskij's upper
bound.Comment: 7 Pages, 2 Figures, Maple cod