Improved Targets for Multilayer Perceptron Learning

Categorization tasks for which layered neural networks can be trained from examples are often better characterized by target activities corresponding to probability distributions [1]. If the activity of output neuron j under presentation of input pattern a is to be proportional to the probability that input pattern a belongs to category j, the performance of the network can be measured by the relative entropy of the output to the target probability distributions [2].