Predictions and Generalization in Layered Networks: A Statistical Approach

The problem of learning a general input-output relation using a "layered neural network" is discussed in a statistical mechanical framework. It is well known that training of the network by minimizing the error on a finite training set does not guarantee that the resulting network will give small error for data outside of the training set (generalization). What is required is a method for estimating the actual performance of the resulting network during training. Since layered networks are hard to analyze directly, we use statistical techniques o formulate the learning problem on a statistical ensemble of networks with the same structure.