Statistical Theory of Learning a Rule

The problem of learning from examples in neural networks is studied in a statistical mechanical framework. The particular network we consider is a single layer perceptron trained to infer a rule from noisy examples. Unlike earlier studies on randomly defined rules we focus here on a deterministic one represented by a reference perceptron. By using the replica method, learning and generalization are investigated at zero and finite temperatures, as well as the role of thermal noise in improving the generalization performance. Modeling the noisy rule by the ensemble of networks is also discussed.