2 research outputs found
Minimum Description Length Principle for Maximum Entropy Model Selection
Model selection is central to statistics, and many learning problems can be
formulated as model selection problems. In this paper, we treat the problem of
selecting a maximum entropy model given various feature subsets and their
moments, as a model selection problem, and present a minimum description length
(MDL) formulation to solve this problem. For this, we derive normalized maximum
likelihood (NML) codelength for these models. Furthermore, we prove that the
minimax entropy principle is a special case of maximum entropy model selection,
where one assumes that complexity of all the models are equal. We apply our
approach to gene selection problem and present simulation results.Comment: 9 pages, 3 figures, 4 tables, submitted to Uncertainty in Artificial
Intelligenc