The work discussed in this paper is motivated by the need of building decision support systems for real-world problem domains. Our goal is to use these systems as a tool for supporting Bayes optimal decision making, where the action maximizing the expected utility, with respect to predicted probabilities of the possible outcomes, should be selected. For this reason, the models used need to be probabilistic in nature --- the output of a model has to be a probability distribution, not just a set of numbers. For the model family, we have chosen the set of simple discrete finite mixture models which have the advantage of being computationally very efficient. In this work, we describe a Bayesian approach for constructing finite mixture models from sample data. Our approach is based on a two-phase unsupervised learning process which can be used both for exploratory analysis and model construction. In the first phase, the selection of a model class, i.e., the number of parameters, is performed by calculating the Cheeseman-Stutz approximation for the model class evidence. In the second phase, the MAP parameters in the selected class are estimated by the EM algorithm. In this framework, the overfitting problem common to many traditional learning approaches can be avoided, as the learning process automatically regulates the complexity of the model. This paper focuses on the model class selection phase and the approach is validated by presenting empirical results with both natural and synthetic data.
Probabilistic inference and learning, finite mixture models, marginal
likelihood, crossvalidation