Learning class probabilities from labeled data

A Bayesian classifier may supply an optimal estimate of the a posteriori class probabilities for classifying stochastic patterns, provided that the underlying statistical model of the problem is known. In the absence of such a priori knowledge, one valuable alternative is the Boltzmann Perceptron Classifier (BPC), a statistical neural based classifier, which was shown to have the capability of Bayesian like decisions. The original learning algorithm of the BPC requires a knowledge of the a posteriori probabilities for the given training set. However, these probabilities are seldom known in advance, and instead, a labeled training data is given for which only the class membership associated with each training sample is known. We introduce a regulated learning scheme which estimates the class probabilities from such labeled data and constructs a classifier that generalizes well for new data.